Depending on the domain of purchase, such an investment of time and energy can either pay off in spades, or be a huge waste of resources when one could instead just buy some product recommended on Wirecutter, or a well-rated Amazon product or something.
What I am saying is that doing as you suggest very often (and, in fact, increasingly often) ends up being completely misleading—not just “not ideal” or some such, but actually much worse than doing as I suggest.
I usually spend way too much time on ultimately irrelevant product research, so when I see a suggestion to ignore a valid but imperfect source of information because there’s a more effortful way of obtaining better information, I feel compelled to suggest to reverse that advice.
But it’s not a valid source of information. That’s my point!
For example, supposing I wanted to build a new PC, it’s immediately obvious that the more expensive build has a higher performance.
I have built many PCs over the years (there are at this moment six machines that I built, sitting in this room), and I can tell you that this is not correct.
Finally, there’s the problem that one is often insufficiently aware of one’s requirements. …
On this point I agree with you. (But it’s not clear whether this is meant as a counterpoint to anything that I said? It does not seem to be any such thing…)
I have built many PCs over the years (there are at this moment six machines that I built, sitting in this room), and I can tell you that this is not correct.
I’ve also built a smaller number of PCs over the years, maybe 5-ish. I always set a rough budget, then chose the best components in terms of price/performance (and e.g. silence), relative to that budget. I don’t understand how you’re ever supposed to get worse performance by using that algorithm while tripling your budget.
To be clear, I’m not suggesting that a random $3000 PC necessarily has higher performance than a random $1000 PC. But e.g. my original example linked to these PC build guides, and there I would be shocked if a random highly voted $3000 build would lose in performance to even the best $1000 build.
How is that consistent with saying that price is not a valid source of information?
On this point I agree with you. (But it’s not clear whether this is meant as a counterpoint to anything that I said? It does not seem to be any such thing…)
What I meant here was: How am I supposed to follow your advice to do all this effortful research (how many hours are we talking here, anyway?), when the way to find out what I actually need is to try a bunch of versions of the product in question, at which point I immediately realize that some weird new requirement like “smartphone screen must not exceed a certain height” trumps all my other requirements.
First, it would be foolish to suggest that, in any given category, any more expensive thing is always worse than any less expensive thing, and indeed that is not what I claimed. (Note, again, what I said: “It is a fundamental mistake to think that spending more money necessarily gets you more of anything that you value.”—I did not emphasize ‘necessarily’ in my initial comment, but it’s there for a reason!)
Second, an obvious point, but one whose importance is easy to overlook, is that the PC builds you link to, are not single products, but collections of parts, each of which is itself a retail product. The correct question is not “is this PC build better than that PC build”; rather, the correct question is “is this PC case better than that PC case”, “is this graphics card better than that graphics card”, etc.
(Note that this does not apply to all of the parts, but only some of them! So, for example, the cheapest and the most expensive builds you link to have different amounts of RAM: 16 GB vs. 32 GB. Obviously, 32 GB of memory is going to cost you more than 16 GB of memory. On the other hand, you should ask whether DDR5-4800 RAM is actually going to give you superior performance to DDR4-3200 RAM, such that 32 GB of the former justifies a price increase of 3.5x over 16 GB of the latter, instead of only 2x. Note that there is no guarantee whatsoever that there will be any performance difference, nor even that the difference, if any, will be positive.)
And the answers to these more appropriate sub-questions may not be obvious at all! The Antec DF700 Flux ATX Mid Tower Case, from the cheapest build, costs ~$110; the Fractal Design Torrent RGB ATX Mid Tower Case, from the priciest build, costs ~$250. Is going from the former to the latter an improvement at all? Is it possible for a more expensive PC case to be worse, in all relevant ways, than a cheaper PC case? I can answer this one easily: yes, it absolutely is possible; examples abound.
On the other hand, the MSI Radeon RX 6600 XT 8 GB MECH 2X OC Video Card, from the cheapest build, costs ~$400, while the Zotac GeForce RTX 3090 24 GB GAMING AMP Core Holo Video Card, from the priciest build, costs ~$1500. Is the latter a better graphics card? Yes. (Whether the performance difference is relevant to your needs, or worth the difference in price, is another matter entirely, outside the scope of our discussion.) But this gets us into my next point…
Third, PC components such as RAM, graphics cards, CPUs, etc., are products whose performance is, compared to most consumer products, very easy to quantify and measure. (But note that the danger of Goodhart’s law lurks behind every performance benchmark! The history of the consumer/enthusiast PC market is rife with such cases…) Because these components are routinely used to produce measurable output (whether that be “Bitcoins mined per hour” or “rendered frames per second” or whatever else), there is a quantitative check on evaluated quality, which is easily measurable, more or less publicly accessible, and therefore tied closely to demand and thus price.
And it is therefore not surprising that it’s precisely such components of a build where this is not true, such as cases (see above), that are where it is most likely that you can find yourself paying more for less (or, at best, for nothing).
Fourth, the PC enthusiast community is a source of high-quality data about the market and the products therein. We build PCs, we test them, we write detailed reviews, etc. That you linked to PC Part Picker is not a coincidence. There is no Microwave Picker, Toothbrush Picker, Trash Can Picker, etc. (Amazon, you say? But Amazon sells computers, too—why didn’t you link to there? Because, of course, Amazon is an extremely low-quality signal, where the dominant optimization pressure is not toward accuracy, but toward deception. The Wirecutter? As I’ve noted before, The Wirecutter is notorious for its inaccurate and misleading claims.)
For most consumer products, there is no such high-quality data source, and consequently there is no reason to expect that the pricier product is always, or even almost always, going to be “better” in any meaningful way.
In summary: the “PC build” example is extremely non-representative, because (a) each build is a collection of individual retail product categories, in each of which relevant quality may or may not be strongly correlated with price; (b) computer components have performance that is much more easily quantifiable and measurable than is that of most consumer products; (c) there is a PC enthusiast community which generates high-quality data about the products in question, on which basis it is possible to evaluate the builds—as is not the case for the great majority of consumer products.
What I meant here was: How am I supposed to follow your advice to do all this effortful research (how many hours are we talking here, anyway?), when the way to find out what I actually need is to try a bunch of versions of the product in question, at which point I immediately realize that some weird new requirement like “smartphone screen must not exceed a certain height” trumps all my other requirements.
Here I will only note that the standard advice in such cases is “buy the cheapest version of whatever it is; use it until it breaks (or no longer satisfies your needs); at that point you will have gotten a great deal of data to inform subsequent purchases, if any”.
Doing a great deal of research first, before taking any action whatsoever, is certainly a typical (or perhaps stereotypical) “rationalist” failing, and I do not by any means endorse it. (Indeed you may note that I never said anything about “do[ing] all this effortful research”; there is no particular reason why the sort of thought process to which I alluded in my initial post should take all that long. The difficult part—apparently—is the insight, in the first place, that such an approach is correct.)
What I am saying is that doing as you suggest very often (and, in fact, increasingly often) ends up being completely misleading—not just “not ideal” or some such, but actually much worse than doing as I suggest.
But it’s not a valid source of information. That’s my point!
I have built many PCs over the years (there are at this moment six machines that I built, sitting in this room), and I can tell you that this is not correct.
On this point I agree with you. (But it’s not clear whether this is meant as a counterpoint to anything that I said? It does not seem to be any such thing…)
I’ve also built a smaller number of PCs over the years, maybe 5-ish. I always set a rough budget, then chose the best components in terms of price/performance (and e.g. silence), relative to that budget. I don’t understand how you’re ever supposed to get worse performance by using that algorithm while tripling your budget.
To be clear, I’m not suggesting that a random $3000 PC necessarily has higher performance than a random $1000 PC. But e.g. my original example linked to these PC build guides, and there I would be shocked if a random highly voted $3000 build would lose in performance to even the best $1000 build.
How is that consistent with saying that price is not a valid source of information?
What I meant here was: How am I supposed to follow your advice to do all this effortful research (how many hours are we talking here, anyway?), when the way to find out what I actually need is to try a bunch of versions of the product in question, at which point I immediately realize that some weird new requirement like “smartphone screen must not exceed a certain height” trumps all my other requirements.
Re: the PC build example:
First, it would be foolish to suggest that, in any given category, any more expensive thing is always worse than any less expensive thing, and indeed that is not what I claimed. (Note, again, what I said: “It is a fundamental mistake to think that spending more money necessarily gets you more of anything that you value.”—I did not emphasize ‘necessarily’ in my initial comment, but it’s there for a reason!)
Second, an obvious point, but one whose importance is easy to overlook, is that the PC builds you link to, are not single products, but collections of parts, each of which is itself a retail product. The correct question is not “is this PC build better than that PC build”; rather, the correct question is “is this PC case better than that PC case”, “is this graphics card better than that graphics card”, etc.
(Note that this does not apply to all of the parts, but only some of them! So, for example, the cheapest and the most expensive builds you link to have different amounts of RAM: 16 GB vs. 32 GB. Obviously, 32 GB of memory is going to cost you more than 16 GB of memory. On the other hand, you should ask whether DDR5-4800 RAM is actually going to give you superior performance to DDR4-3200 RAM, such that 32 GB of the former justifies a price increase of 3.5x over 16 GB of the latter, instead of only 2x. Note that there is no guarantee whatsoever that there will be any performance difference, nor even that the difference, if any, will be positive.)
And the answers to these more appropriate sub-questions may not be obvious at all! The Antec DF700 Flux ATX Mid Tower Case, from the cheapest build, costs ~$110; the Fractal Design Torrent RGB ATX Mid Tower Case, from the priciest build, costs ~$250. Is going from the former to the latter an improvement at all? Is it possible for a more expensive PC case to be worse, in all relevant ways, than a cheaper PC case? I can answer this one easily: yes, it absolutely is possible; examples abound.
On the other hand, the MSI Radeon RX 6600 XT 8 GB MECH 2X OC Video Card, from the cheapest build, costs ~$400, while the Zotac GeForce RTX 3090 24 GB GAMING AMP Core Holo Video Card, from the priciest build, costs ~$1500. Is the latter a better graphics card? Yes. (Whether the performance difference is relevant to your needs, or worth the difference in price, is another matter entirely, outside the scope of our discussion.) But this gets us into my next point…
Third, PC components such as RAM, graphics cards, CPUs, etc., are products whose performance is, compared to most consumer products, very easy to quantify and measure. (But note that the danger of Goodhart’s law lurks behind every performance benchmark! The history of the consumer/enthusiast PC market is rife with such cases…) Because these components are routinely used to produce measurable output (whether that be “Bitcoins mined per hour” or “rendered frames per second” or whatever else), there is a quantitative check on evaluated quality, which is easily measurable, more or less publicly accessible, and therefore tied closely to demand and thus price.
And it is therefore not surprising that it’s precisely such components of a build where this is not true, such as cases (see above), that are where it is most likely that you can find yourself paying more for less (or, at best, for nothing).
Fourth, the PC enthusiast community is a source of high-quality data about the market and the products therein. We build PCs, we test them, we write detailed reviews, etc. That you linked to PC Part Picker is not a coincidence. There is no Microwave Picker, Toothbrush Picker, Trash Can Picker, etc. (Amazon, you say? But Amazon sells computers, too—why didn’t you link to there? Because, of course, Amazon is an extremely low-quality signal, where the dominant optimization pressure is not toward accuracy, but toward deception. The Wirecutter? As I’ve noted before, The Wirecutter is notorious for its inaccurate and misleading claims.)
For most consumer products, there is no such high-quality data source, and consequently there is no reason to expect that the pricier product is always, or even almost always, going to be “better” in any meaningful way.
In summary: the “PC build” example is extremely non-representative, because (a) each build is a collection of individual retail product categories, in each of which relevant quality may or may not be strongly correlated with price; (b) computer components have performance that is much more easily quantifiable and measurable than is that of most consumer products; (c) there is a PC enthusiast community which generates high-quality data about the products in question, on which basis it is possible to evaluate the builds—as is not the case for the great majority of consumer products.
Here I will only note that the standard advice in such cases is “buy the cheapest version of whatever it is; use it until it breaks (or no longer satisfies your needs); at that point you will have gotten a great deal of data to inform subsequent purchases, if any”.
Doing a great deal of research first, before taking any action whatsoever, is certainly a typical (or perhaps stereotypical) “rationalist” failing, and I do not by any means endorse it. (Indeed you may note that I never said anything about “do[ing] all this effortful research”; there is no particular reason why the sort of thought process to which I alluded in my initial post should take all that long. The difficult part—apparently—is the insight, in the first place, that such an approach is correct.)