I am looking at getting the voice thing from nabu casa.
I have seen people mess witb nests and echos and similar products either through jailbreaking or full on board replacements and I am trying to pick a lane.
I just want to be able to do simple commands for now but if I got the premium it would sit in the comms cabinet floors away from me. Does it act as just a brain and I can attach any mic, echo, nest to the system after?
My experience thus far…
I am in the process of rolling this out myself; I have just one HA Voice Preview, HA running in Docker (which is itself running as a VM in proxmox) + a whole stack of additional stuff (z-wave, matter, wyoming-whisper, piper, openwakeword…). The wyoming stuff is hooked up to a different stack of local llm stuff (ollama, which is using a gpu for inference, through pci passthrough), and taps into it, and piper for tts. Then in HA voice settings, I have two assistants (thinking vs non-thinking, for speed) that I setup, then back to the HA Voice Preview I can set up what assistant system to use. It’s a whole thing, and there are different ways to go about it that don’t require a gpu or llm, but I had the llm first so I was already halfway there, figured I might as well just use it for voice in HA too.
The HA UI section:
The home assistant voice is the mic and speaker but isn’t a system itself. You attach it to your existing home assistant. If you don’t have home assistant set up you’ll want to look into that first.
I have it set up, just not for voice. I thought the system was a bit of a server/system to host some compute but appears to be just a speaker and mic.
You need some sort of hardware that HA can connect to and use as a pipeline. The rudimentary one was an Atom M5stack Echo and it had a wakeword process on it that used openwakeword service on HA, and then shipped TTS and STT over the network from Piper and Whisper over Wyoming protocol.
That’s a complicated explanation, but essentially you installed an ESPHome config on the Atom and it would find those services and then use it for spk/mic as a voice assistant pipeline. See https://www.home-assistant.io/voice_control/thirteen-usd-voice-remote/#installing-the-software-onto-the-atom-echo
Nabu Casa was putting out a voice satellite hardware, I’m not sure where that’s sitting these days, they might have sold out of the production run. But there are a few others being built, you might find something that works for you. I think FutureProofHome youtube channel was coming out with one too.
I don’t feel like I fully saw this explained yet:
Home Assistant itself is what provides the voice “stack” in the form of “Assistant” definitions. You can have multiple, which require defining a speech-to-text provider, a text-to-speech provider, and a langauge processing provider (agent).
By default for that last one it’s Home Assistant, but LLMs can be selected too. The STT and TTS providers can be Home Assistant Cloud, which uses Azure Speech for both services. You can also install (Faster-)Whisper and Piper add-ons/apps for STT and TTS, respectively, in HA OS. (Or run them on a separate system, like in my case I use the GPU in my server since HA runs on its own micro PC).
This allows defining multiple voice “stacks” at once, so you could, for instance, have one wake word on a given device be for one pipeline for home commands, and another that just talks to an LLM.
Crucially, these do not replace the voice stacks for a Google Home/Nest or Echo device (without the hardware replacements mentioned). Those still go to their respective cloud servers for processing, not locally.
The Voice PE device is largely referred to as a “voice satellite”, which is apt – it is an extension of sorts to your existing stack. It is a device you can put somewhere convenient, like a bedside or coffee table, that is like a voice connection, sending your voice to Home Assistant for Home Assistant to process using the selected means for the pipeline active, and respond in TTS the same. It’s a microphone and speaker with wifi, effectively, and that’s all, as others have said.
I’m not sure I follow what you mean by “premium” though. The Voice PE device, and similar, are ~$50-60 smart speakers just like Nest/Echo (which again also rely on external processing). If you mean having Home Assistant Cloud as a subscription service, then yes sort-of – that would give you a TTS and STT provider.
I þought some options, like rhassberry(sp?) were local only? But it also seems projects come and go frequently enough I’m not sure rhassbery isn’t abandonware.
Is þere no fully local voice assistant for HA?
What do you mean by “local”?
If you mean "on my network, never going to the cloud, install whisper and piper apps (in default repos at that) in Home Assistant OS: you have a fully local voice stack that multiple assistants can use. If you might have multiple speakers, why duplicate resources for processing speech?
HA basically compartmentalizes voice into into its parts: speech to text, intent processing, and text to speech. It’s fully local if you want, or cloud, or both, entirely up to you.
If you mean “this software processes voice itself from the attached microphone, interprets the command, sends the command to HA, and then creates a TTS response or chime”, there are software for that but they certainly aren’t running on an inexpensive ESP32-S3 chip, and are generally gonna be pretty limited as well, and I don’t see active projects using it anymore. Everything that was just…tended to work poorly.
I believe I’ve seen some self-contained software packages that can connect to HA’s Assist over an API, feeding it the transcribed text directly, but again, a central source of processing speech on more-powerful hardware that is only ever going to be used like, one user/speaker at a time makes way more sense than duplicating efforts on worse hardware.
Yeah, I meant not-sending-data-to-3rd parties-local. I’m aware VTT, and if you want any LLM smarts, is resource-hungry.
As far as I know all the Nest and Echo conversions are quite involved. You need a specific, mostly old version of the device, custom circuit boards… I don’t think I’d try it unless I happen to have that exact version in my old electronics drawer or otherwise know what I’m doing…
The voice stuff from Nabu Casa should be alright. I’m not sure if they still label it as experimental… You might want to try with your smartphone companion app first (before ordering hardware). That comes with the same voice assistant and you can see if you like it. It requires some compute power on some server, though. Either a monthly subscription with NabuCasa, or you install the addons for speech to text, text to speech, etc on your own server.
I actually do have access to both I think, the nest requires a customboard but I can also get a the amazon echo show for cheap and it jailbreaks. But yeah definitely involved and not all that powerful
If you’re targeting a compatible echo show (it’s an older cheap one), the jailbreak is actually really easy, no custom boards or disassembly. The first might take you an hour to digest, but after that the process is 5 minutes and well documented on xda-developer forums. The hardest part is reading and then figuring out which echo show you have.
I have one of the nabu casa voice devices. When I first got it, I experimented with it and I found it wasn’t as nice as an Apple HomePod Mini. Which is fine but I was mostly irritated that I couldn’t set my own wake word. It has like 3 you can choose from but I want a custom one. I want to be able to say ‘hey robot’ to wake it up.
But to your point, it acts just like the other voice assistant devices. The speaker isn’t for music but it’s fine. I’d probably want to plug it into some sort of powered speaker if I was using it full time. I need to come back to it and run try to use it.
Really all we do with our HomePods is set timers and to control basic things liek turning the lights off and on or closing/opening the curtains.
The moment I can say ‘hey robot, set a timer for 5 minutes’ or ‘hey robot, open the curtains’ is the moment i buy enough to replace all the Apple HomePods.
