There’s just no way to pay for the cost of these services, though.
When someone constructs a 100 MW data center (now considered a smaller one for new construction), that’s about $2 billion in total costs to outfit the whole operation. And then once it’s on, we’re talking something like $10-20 million/month in electricity alone, and a few million in other costs. How many $20 subscriptions do you need to sell just to break even with your operating expenses? How many $100/month subscriptions do you need to sell to make a dent on your interest payments on the construction? Will there be a market for $1000/month subscriptions from millions of customers? If not, how’s this all going to be paid for?
They certainly cant make their money back writing personal emails and doing kids homework, but I dont think thats where they are aiming in the long run. We may end up with big business and military paying big money for the real frontier models and everyone else using lightweight local models on their own hardware, cheap models integrated into existing applications, and watered down frontier models on subscription.
I don’t think government funding can actually offset the crash in consumer and business demand being insufficient to cover the cost of the most expensive models on the most expensive GPUs. But if you look through my comment history I’ve made the comparison to supersonic flight, because I genuinely believe there’s a possibility that governments fund the expensive branch of this technology for their own military or surveillance or law enforcement purposes without the benefits necessarily actually spilling out into normal commercial applications.
We’ve hit the point where training a model (both pre training and post training) isn’t the expensive part, and the expensive part is actual inference, which makes it hard to scale the most expensive models to where it’s useful for a lot of people. So it might be that the companies and governments that can afford to operate an expensive model might be the only ones to do it. And they’ll be able to, without necessarily the public being able to have access to the same tech.
There’s just no way to pay for the cost of these services, though.
When someone constructs a 100 MW data center (now considered a smaller one for new construction), that’s about $2 billion in total costs to outfit the whole operation. And then once it’s on, we’re talking something like $10-20 million/month in electricity alone, and a few million in other costs. How many $20 subscriptions do you need to sell just to break even with your operating expenses? How many $100/month subscriptions do you need to sell to make a dent on your interest payments on the construction? Will there be a market for $1000/month subscriptions from millions of customers? If not, how’s this all going to be paid for?
They certainly cant make their money back writing personal emails and doing kids homework, but I dont think thats where they are aiming in the long run. We may end up with big business and military paying big money for the real frontier models and everyone else using lightweight local models on their own hardware, cheap models integrated into existing applications, and watered down frontier models on subscription.
You may be right (I hope you are). Do you think government funding would offset this?
I don’t think government funding can actually offset the crash in consumer and business demand being insufficient to cover the cost of the most expensive models on the most expensive GPUs. But if you look through my comment history I’ve made the comparison to supersonic flight, because I genuinely believe there’s a possibility that governments fund the expensive branch of this technology for their own military or surveillance or law enforcement purposes without the benefits necessarily actually spilling out into normal commercial applications.
We’ve hit the point where training a model (both pre training and post training) isn’t the expensive part, and the expensive part is actual inference, which makes it hard to scale the most expensive models to where it’s useful for a lot of people. So it might be that the companies and governments that can afford to operate an expensive model might be the only ones to do it. And they’ll be able to, without necessarily the public being able to have access to the same tech.
They would raid pensions before asking congress