If you’re interested in this problem you should check out this preprint, written by a former colleague of mine:
https://bpb-us-e1.wpmucdn.com/sites.northwestern.edu/dist/1/2465/files/2025/02/Price-gouging.pdf
It gives a nice economic model of price gouging, elasticity, and inequality, and presents a rigorous argument for why the typical utilitarian welfare theorems do not hold in the price gouging setting.
purple fire
Karma: 73
This is incorrect. NOPAT refers to a company’s cash flows before mandatory debt repayments, recurring capex, and interest expenses. It’s used to compare companies without considering differences in capital structure; it is not a good measure of how much the company can afford to pay in dividends. Many companies that file for bankruptcy have positive NOPAT and yet strictly 0 equity value. You’re thinking of levered free cash flow.
We don’t know what the value/quantity curve looks like, or how it will change if demand changes.
This is untrue. Order book data is available for the right price (and although it doesn’t capture all capital flow, sophisticated algorithms can infer the full curves with high accuracy). For what it’s worth, many finance firms view the market price as basically noise and use order book curves as the “real” price input. Most contracts that depend on equity value will calculate it by VWAP over the last 30 or 90 trading days, which is a much better estimate of true value.
If you believe that market capitalization is meaningful, it is probably because of that conceptual error, combined with social validation. Since it is a common error, it seems correct. There are many other examples of common errors due to fallacies. People create language games that seem meaningful because people play them.
I think this post in general lacks a great deal of nuance and understanding, which would normally be fine except that you adopted such an arrogant tone that it was frustrating to read. I promise people in finance understand that the market cap isn’t literally how many dollars it would cost to buy all of the shares. Even Google’s AI summarization says that. That doesn’t make it a useless metric, and market cap remains a very important input for many processes in trading.
People think of the price metaphorically as a physical property, such as mass.
Who are these people, and where can I bet against them?
Because we don’t only care about the net effect of trade policy, we also care about other factors like the distribution of benefits and the unemployment rate. Especially during a recession or periods of high joblessness and income inequality, subsidizing exports may be necessary to stimulate economic activity among a particular demographic or in a specific region. And during economic downturns, a stronger domestic currency is not always a good thing. I’ll just quote Mankiw here, discussing export subsidies during a depression:
A weaker dollar means that our goods are cheaper relative to foreign goods. That stimulates our exports and reduces our imports. Higher net exports raise domestic production and employment. Foreign goods are more expensive, but more Americans are working. Given the desperate need for jobs, on net we are almost surely better off with a weaker dollar for a while.
This is simply incorrect, (both your and Eliezer’s explanations) and shows a misunderstanding of basic macroeconomics that would be taught at an intro-level university course.
When countries run a trade surplus—positive net exports—there mechanically must be a net inflow of capital. When I export goods and services, foreign buyers need to exchange their currency for the domestic currency to purchase those exports. This causes the domestic currency to appreciate, which puts downward pressure on domestic real interest rates relative to other countries. Businesses in the exporter’s country are thus able to invest more heavily in capital goods, raising their productivity and increasing long-term GDP growth.
For a thorough treatment, Mankiw’s textbook is a good resource: http://students.aiu.edu/submissions/profiles/resources/onlineBook/T9D9B4_Principles_of_Economics-_7th_Edition.pdf
This mechanism is covered in Chapter 31.
Are you including models that are only used by their creator firm? I work as an ML researcher in big tech (I want to keep this account anon, but it’s one of MSFT/OAI, DM, Meta, Anthropic, xAI) and have access to tooling substantially better than what’s commercially available (proof by existence?), but that’s not really what my post is about. My main model for this actually panning out is something like:
Big tech company has control over AI lab
AI lab makes cracked SWE agent
Big tech company notices that releasing that SWE agent will undermine the rest of their software development business, so instead of licensing it out they only make it available to their own staff and perhaps business allies
I’m just clarifying because it’s barely even confidential information that engineers at AI labs have better models than engineers at small or mid-size tech firms, and I want to check what you’re actually betting on.
Sorry, I can elaborate better on the situation. The big tech companies know that they can pay way more than smaller competitors, so they do. But then that group of megacorp tech (Google, Amazon, Meta, etc.) collude with each other to prevent runaway race dynamics. This is how they’re able to optimize their costs with the constraint of salaries being high enough to stifle competition. Here, I was just offering evidence for my claim that big tech is a monopsonistic cartel in the SWE labor market, it isn’t really evidence one way or another for the claims I make in the original post.
If https://outtalent.com/us50/ is to be believed, SWE engineers look pretty concentrated at the top ~5 companies and their subsidiaries. Do you think that data is incorrect?
Concretely, I would claim that >80% of the most skilled software engineers in the US work at <10 companies.Edit: I thought about it more and I think this is actually more like 65% at the 10 biggest companies, but that doesn’t change my central claims.I also disagree with your claim that they are not a cartel. I think the biggest tech companies collude to fix wages so that they are sufficiently higher than every other company’s salaries to stifle competition, while also limiting race dynamics to maintain profits. I think this is done in the form of selectively enforced non-competes, illegal non-poaching agreements, and other shady practices. This has been alleged in court and the companies just settle every time, e.g. https://www.nytimes.com/2014/03/01/technology/engineers-allege-hiring-collusion-in-silicon-valley.html?unlocked_article_code=1.uk4.A5Sn.q5fVDfF_q8Wk&smid=url-share
For those disagreeing--
1. I continue to believe that tech companies derive much of their economic power from cornering the skilled engineering labor market,2. this is highly threatened by the advent of AI capable of coding,
3. and thus many big tech companies have massive economic incentives to limit the general public’s access to models that can code well.
If I changed my mind about any of those 3 points, I would change my mind about the main post. Rather than downvoting, or in addition to it, can you please explain which part you disagree with and why? It will be more productive for everyone and I am open to changing my mind.
I also think monopolizing talent enables software companies to make sure those high fixed costs stay nice and high.
If you disagreed with this, is it because you think it is literally false or because you don’t agree with the implied argument that software companies are doing this on purpose?
Hm, this violates my model of the world.
there are too many AI companies for this deal to work on all of them
Realistically, I think there are like 3-4 labs[1] that matter, OAI, DM, Anthropic, Meta.
some of these AI companies will have strong kinda-ideological commitments to not doing this
Even if that was true, they will be at the whim of investors who are almost all big tech companies.
this is better done by selling (even at a lower revenue) to anyone who wants an AI SWE than selling just to Oracle.
This is the explicit claim I was making with the WTP argument. I think this is firmly not true, and OpenAI will make more money by selling just to Oracle. What evidence causes you to disagree?
- ^
American/Western labs.
- ^
I don’t disagree with most of what you said, maybe I should have been more explicit about some of the points related to that. In particular, I do think “the success of B2B SaaS over bespoke solutions is adequately explained by economies of scale” is true. But I think the reason there are economies of scale is that there are really high fixed costs and really low variable costs. I also think monopolizing talent enables software companies to make sure those high fixed costs stay nice and high.
With AI, engineering talent becomes cheap and plentiful. When that happens, fixed costs will plummet unless firms can control access to AI. If fixed costs plummet, economies of scale go away and the savings from the SaaS model get outweighed by the marginal benefit of bespoke solutions.
what typically happens is that Amir forks out for Slack, or some competitor, while Amir’s engineers work on software that generates revenue.
To push back a little on this, as software companies grow they do try to do this less and less. How much enterprise software do you think Microsoft or Google is outsourcing? As soon as it becomes a little bit of a dependence they usually just acquire the company.
In fairness, I don’t think this process will be rapid, nothing in B2B SaaS is. But I think tech companies see it on the horizon.
What working on AI safety taught me about B2B SaaS sales
Sure, I think social media is probably the best example of this. Suppose there are two platforms, A and B, and social media sites are worth more when more people are on it. Our “resource allocation” problem is to maximize utility, so we want to get everyone on the same site. There are two equilibria here; we can either set the price for A much higher than B and everyone will move to B, or vice versa.
If the demand functions weren’t interdependent and every agent just got some amount of utility from A and some amount of utility from B, there would be exactly one equilibrium price.
Hm, I think that’s more of a supply independence thing, what economists would call “non-excludable”. If the government funds a police force, it’s not as if they protect some citizens but not others. But that’s not a violation of the demand independence assumption because I care about living in a country with a strong police force regardless of whether you want that or not.
Goods with demand independence, from Ferraris to Facebook, generally do get provided by markets in real life, they just don’t have stable prices. It breaks some of the equilibrium models because it can cause divergences or circularity in your demand function, and then there’s no fixed point in positive price/demand space.
Luxury is a good example of this that happens in real life. Here’s an intuition-prompting setup:
Suppose I’m rich and I buy a Gucci bag
You’re poor, but you want to look rich so you also buy a Gucci handbag
Now I don’t think the bag is as exclusive, so I don’t want mine any more
Now that the rich guy isn’t wearing it anymore, you don’t want yours either
But now no one has it, so it seems exclusive again, so now I want it
Repeat
This doesn’t mean markets won’t provide Gucci bags (obviously, they do), but there isn’t a price equilibrium, it will fluctuate forever. In terms of the original point, the Gucci bag allocation problem isn’t isomorphic to a market equilibrium, because there is no such equilibrium.
purple fire’s Shortform
People should take the time to understand the ecosystem and economics of enterprise software. I expect that to be a major determinant of the incentive landscape around AGI and have found in conversations that people often don’t really get it.
I’m not sure your idea about training two different CoT processes and penalizing divergence would work...
Me either, this is something I’m researching now. But I think it’s a promising direction and one example of the type of experiment we could do to work on this.
if their models are closed, they probably want to hide the CoT so others can’t train on it / distill it
This could be a crux? I expect most of the economics of powerful AI development to be driven by enterprise use cases, not consumer products.[1] In that case, I think faithful CoT is a strong selling point and it’s almost a given that there will be data provenance/governance systems carefully restricting access of the CoT to approved use cases. I also think there’s incentive for the CoT to be relatively faithful even if there’s just a paraphrased version available to the public, like ChatGPT has now. When I give o3 a math problem, I want to see the steps to solve it, and if the chain is unfaithful, the face model can’t do that.
I also think legible CoT is useful in multi-agent systems, which I expect to become more economically valuable in the next year. Again, there’s the advantage that the space of unfaithful vocabulary is enormous. If I want a multi-agent system with, say, a chatbot, coding agent, and document retrieval agent, it might be useful for their chains to all be in the same “language” so they can make decisions based on each others’ output. If they are just blindly RL’ed separately, the whole system probably doesn’t work as well. And if they’re RL’ed together, you have to do that for every unique composition of agents, which is obviously costlier. Concretely, I would claim that “using natural language tokens that are easy for humans to understand is not the absolute most efficient way for artificial minds to think” is true, but I would say that “using natural language tokens that are easy for humans to understand is the most economically productive way for AI tools to work” is true.
PR reasons, yeah I agree that this disincentivizes CoT from being visible to consumers, not sure it has an impact on faithfulness.
This is getting a little lengthy, it may be worth a post if I have time soon :) But happy to keep chatting here as well!
- ^
My epistemically weak hot take is that ChatGPT is effectively just a very expensive recruitment tool to get talented engineers to come work on enterprise AI, lol
- ^
Yes, this is the exact setup which cause me to dramatically update my P(Alignment) a few months ago! There are also some technical tricks you can do to make this work well—for example, you can take advantage of the fact that there are many ways to be unfaithful and only one way to be faithful, train two different CoT processes at each RL step, and add a penalty for divergence.[1] Ditto for periodic paraphrasing, reasoning in multiple languages, etc.
I’m curious to hear more about why you don’t expect companies to invest much into this. I actually suspect that it has a negative alignment tax. I know faithful CoT is something a lot of customers want-it’s just as valuable to accurately see how a model solved your math problem, as opposed to just getting the answer. There’s also an element of stickiness. If your Anthropic agents work in neuralese, and then OpenAI comes out with a better model, the chains generated by your Anthropic agents can’t be passed to the better model. This also makes it harder for orgs to use agents developed by multiple different labs in a single workflow. These are just a few of the reasons I expect faithful CoT to be economically incentivized, and I’m happy to discuss more of my reasoning or hear more counterarguments if you’re interested in chatting more!- ^
To be clear, this is just one concrete example of the general class of techniques I hope people work on around this.
- ^
This is simply false. See e.g. https://en.wikipedia.org/wiki/List_of_countries_by_trade-to-GDP_ratio
The country with the highest GDP-adjusted export/import ratio is… Gabon. Then Qatar, Bermuda, Cambodia, Turkmenistan, and Libya. Norway is sandwiched between Congo and Azerbaijan. Zimbabwe ekes out a lead over the US. Luxembourg, the country with the highest GDP per capita, is right below Kuwait.
You are spreading misinformation. The impact of trade balances on GDP growth is an incredibly controversial topic among academic economists.