Rebuilding my local voice assistant pipeline

(blog.platypush.tech)

64 points | by todsacerdoti 6 hours ago ago

12 comments

  • mmmooo 3 hours ago

    older amazon echo dots and shows really make great hardware for this, combined with homeassistant, and can be had for very little cost. e.g. shameless plug: https://github.com/ygelfand/echolocal

    • nl 2 hours ago

      I just commented in another thread that I built this: https://github.com/nlothian/earwing

      Beam forming microphone array, custom UI, custom case. Maybe costs more than a second hand echo dot though (hardware is probably around $30).

    • neumann 2 hours ago

      I got an echo show rooted and ROMd that I am loving - put it in the kids room as a music player in kiosk mode, but have been now wanting one for myself with wakewords. This looks like the ticket.

      • mmmooo an hour ago

        the gen 1 or 2 show 8s make great devices for this kind of stuff, 5's are a little small for me. the lenovo android things (google homes) are also really nice if you want a slightly bigger screen, but not as cheap second hand.

  • mewse-hn 3 hours ago

    Interesting write up, I've been vibe coding an android voice assistant using a similar pipeline without a wakeword - VAD > ASR > LLM > TTS

    I've found parakeet to be much, much better than whisper and I made a 4 bit quant of parakeet 110M for my crappier android test devices, it would probably fit a raspberry pi:

    https://huggingface.co/mewse/parakeet-tdt_ctc-110m-onnx-int4

    • GaggiX 2 hours ago

      There is also parakeet-redux, a 1.58bits quant of parakeet-tdt-0.6b-v3 that runs as well as the original model: https://huggingface.co/moondream/parakeet-redux

      The model is actually a distillation of a different checkpoint called parakeet-ultra, same size and architecture as parakeet-tdt-0.6b-v3 but better, that's why the 1.58bit quant works so well.

      • mewse-hn 2 hours ago

        Oh crazy I hadn't seen this yet, I might be able to unify my 0.6B vs 110M selector, thanks!

  • fragilerock 4 hours ago

    Interesting read! I have my own local voice assistant and I implemented exactly the same question mark trick just the other day.

    The overall pipeline is quite similar to my previous setup, but I found the STT -> LLM -> TTS pipeline just infuriatingly slow. TTS and STT with whisper/piper worked fine and reasonably fast for me locally, but the real bottleneck was always the fact that they're not running concurrently with the LLM call. I ended up forking an open source openai realtime API ESP32 plugin and patching it to my liking. It's a bit more expensive in terms of API cost but I only use it sparingly and it's so much nicer to use now.

  • bityard 3 hours ago

    Is this complimentary to, or an alternative to Home Assistant?

    • malfist 3 hours ago

      Home assistant doesn't really provide much out of the box, its mostly a modular/pluggable ecosystem and one of the things you can install are various voice assistants

  • dofm 3 hours ago

    Just what I want to read โ€” favourited to come back to!

  • stavros 5 hours ago

    This is really useful, thanks! The other day I bought a small screen and didn't know what to do with it so I made an assistant that talks to my bot:

    https://m.youtube.com/shorts/3FseVyAG5uM

    I used microwakeword too, and training my own wakeword was pretty simple (basically just recording a few tens of samples). Highly recommended.

    I also wanted to be able to issue some voice commands for lights and other things in my home with low latency, so I added a short-circuit plugin that runs right after recognition. If the recognized command matches one of the regexes, it's immediately executed, which has a latency of under a second. Otherwise, it gets sent to my assistant, which usually will tweak the regex (or add a new one) to improve the whole system as it fails.