If btop is to be believed my server pulls a tiny 1.5W on idle. Given everyone else’s numbers I’m wondering if that’s true of btop is confused. It’s an old optiplex 3050.
Regardless my main rig which is regularly running expensive simulations is often ~200W so my server and home labbing is a rounding error
Totally agree on the privacy sovereignty front. I suspect in the long run personal tasks will be handled by local models for this reason. I guess I’m thinking more on the scale of institutional compute. Just like every large university has their own super compute cluster I expect them to want their own AI instance (I know companies like Amazon already have such a thing) and in those instances the gap in skill really matters and these institutions are willing to pay a fair amount for the best out there.
Also I can’t speak on equivalence with frontier models 3-4 months ago because I didn’t have access to paid models back then but if that’s true and not just bench maxxing then it means the moat is widening not shrinking. Which suggests these labs are in a good position to make themselves an asset the US cannot afford to lose despite their comically costly business model.
Can you explain more what you mean by this? For a bubble like the internet I can understand the giants failing and new companies picking up the pieces because the technology was already available to everyone so there was no real moat for new companies entering the space which feels quite different from the AI situation no? I’ll admit I wasn’t alive in 2000 so I don’t know exactly how all that went down.
Yeah Qwen 3.5 122B-A10B. It’s the most capable model I’ve found that I can run on my machine. It’s useful for simple tasks but honestly it’s around the border if correcting/checking its works is comparable effort to doing it myself. So I don’t really use it in reality it’s more a novelty. I’ve also messed around with Kimi K3 since the benchmarks said it was awesome. It’s definitely a league above Qwen but must be benchmaxxed to hell cuz it’s not even close to the same league as opus (I can’t speak to OpenAI models as my team has a Claude account so those are the paid models I know)
The web app keeps a local synced copy too btw. Biometric is fair tho
I run a 122B model locally. I love local models. If you think they’re anywhere near as capable as the data center models you’ve either not tried them or are lying to yourself
I truly can’t see the bubble bursting at this point. They’ve won their race and built a technology so impressive it cannot be ignored or dropped. AI dominance is now a matter of nations (cyber) security. As soon as they’re any real chance of these labs failing they’ll be a tax payer bailout. I just hope it also results in nationalizing the leading AI labs so the models build on public knowledge can be forced into the public domain where they belong
The web interface is hosted directly by your vaultwarden interface so it’s not lockdown-able. The only things that are would be the browser extension and phone apps. Both of which are formally unnecessary because you can always just use the web interface. I’m sure if they ever locked down there’d be community versions in no time as they’re basically just web wrappers anyway.
Real af
To my admittedly limited knowledge every year russia puts forward a motion to condemn nazisim and justify their war in Ukraine (recall they claim Ukraine is run by nazis). They’ve been doing this since 2005 as they’ve always said that breakaway post soviet states are nazis. Every time since 2005 the US and others have voted against it because it’s Russia trying to justify their expansionist tendencies. Idk what happened in 2020 that only the US and Ukraine opposed but yeah that’s my vague understanding
This is exactly how I feel. I’m not even opposed to automation of these tasks on a fundamental level like some people might be. I just fear the transition won’t be thought out at all and will be devastating for the working class. The last time we had sudden automation of this scale was the Industrial Revolution which completely reshaped western society and was very bad for those who lived through it despite the eventual high modern standard of living it produced. I think this may end up going similarly.
What kinda dependency hell? I’ve been running my jellying server with it installed via pacman for over year and never had any problems. I admit I’ve also not messed with docker much in general but I assumed that would be more annoying cuz it would have to be updated separately and such
Why is docker an upgrade?
Cool project idea! Some questions:
It seems the project currently exclusively watches docker related stuff. Is there any plan to watch services more broadly? Most of my stuff is running directly on the machine as a systemd service without any container (probably bad practice sue me). I’m not sure how feasible it would be to chase down everything that’s supplying a local host endpoint tho.
What’s your AI workflow like? What model, what harness etc
Yeah that was my feeling. It seemed unrealistically low.