Why does it matter what is replaced?

I would say if it frees up a human from having to do it that it’s good news.

“must benefit everyone, not just a handful of billionaires”

It’s hard to disagree with that statement but all this upheaval about “killing jobs” seems really pointless to me.

A manual farm has far more employees then one that uses modern machinery, is the economy in decline if the farmer replaces human labor with machines?

Good stuff, Ukraine has proven again that having a nuclear arsenal is really the only way to ensure a country’s security when faced wirh a hostile nuclear armed neighbor.

I’m selfhosting Forgejo and i don’t really see the benefit of migrating to a container, i can easily install and update it via the package manager so what benefit does containerization give?

If there is a way to combine this with something like anubis this could be very interesting, then you are also not dependent on the honesty of reading the robots.txt

Have a RSL license? here you go scrape the content.

Trying to freeload? anubis time.

Having something to talk to is a massive improvement over bottling it all up.

AI is very beneficial to people who can’t afford the cost or are otherwise unable/unwilling to speak with a professional.

How is this not a “won’t somebody please think of the children” thing?

Yes it is terrible this has happend, but there is a way to prevent children from accessing AI and its called parenting.

Kids shouldn’t be using AI if it harms them, kids can’t make this choice so it should be made for them. Same with alcohol, same with porn, same with the other things restricted to children.

But that doesn’t mean responsible adults shouldn’t be able to use it, but “won’t somebody please think of the children” litigation will make that impossible.

I don’t care if openai lose all their money but this ruling would also effect open source AI.

If somebody releases a AI why would they be liable for how people decide to use it? Its software and like any other program its the user’s choice on how to use it.

If i decide to run rm / --no-preserve-root is gnu then responsible to fix it?

AI is already very censored, and if makers become liable for what people do with their AI they will become hyper censored and performance will go down the drain.

Hitler sues Mussolini

Why do core counts and memory type matter when the table includes memory bandwith and tflop16?

The H200 has HBM and alot of tensor cores which is reflected in its high stats in the table and the amd gpus don’t have cuda cores.

I know a major deterioration is to be expected but how major? Even in extreme cases with only 10% efficiency of the total power then its still competitive against the H200 since you can get way more for the price, even if you can only use 10% of that.

Thanks! Ill go check it out.

My target model is Qwen/Qwen3-235B-A22B-FP8. Ideally its maxium context lenght of 131K but i’m willing to compromise. I find it hard to give an concrete t/s awnser, let’s put it around 50. At max load probably around 8 concurrent users, but these situations will be rare enough that oprimizing for single user is probably more worth it.

My current setup is already: Xeon w7-3465X 128gb DDR5 2x 4090

It gets nice enough peformance loading 32B models completely in vram, but i am skeptical that a simillar system can run a 671B at higher speeds then a snails space, i currently run vLLM because it has higher peformance with tensor parrelism then lama.cpp but i shall check out ik_lama.cpp.

• I know the more bandwidth the better, but i wonder how does it scale. I can only test my own setup which is less then optimal for this purpose with pcie 4.0 x16 and no p2p, but it goes as follows: a single 4090 gets 40.9 t/s while 2 get 58.5 t/s using tensor parrelism tested on Qwen/Qwen3-8B-FP8 with vLLM. I am really curious how this scales over more then 2 pcie 5.0 cards with p2p, which all cards here listed except the 5090 support.
• The theory goes that yes while the H200 has a very impressive bandwith of 4.89 TB/s, but for the same price you can get 37 TB/s spread across 58 RX 9070s, but if this actually works in practice i don’t know.
• I don’t need to build a datacenter, i’m fine with building a rack myself in my garage. And i don’t think that requires higher volumes than just purchasing at different retailers
• I intend to run at fp8 so i wanted to show that instead of fp16 but its surprisingly difficult to find the numbers for that, only the H200 datasheet, cleary displays FP8 Tensor Core, the RTX pro 6000 datasheet keeps it vague with only mentioning AI TOPS, which they define as Effective FP4 TOPS with sparsity, and they didn’t even bother writing a datasheet for he 5090 only saying 3352 AI TOPS, which i suppose is fp4 then. the AMD datasheets only list fp16 and int8 matrix, whether int8 matrix is equal to fp8 i don’t know. So FP16 was the common denominator for all the cards i could find without comparing apples with oranges.

Well a scam for selfhosters, for datacenters it’s different ofcourse.

Im looking to upgrade to my first dedicated built server coming from only SBCs so I’m not sure how much of a concern heat will be, but space and power shouldn’t be an issue. (Within reason ofcourse)

As long as Russia is fighting China gets cheap oil