Silero VAD is a voice activity detection model that runs locally on your phone, letting MyBenAI listen for speech without sending audio to the cloud. Learn how it works, why it's fundamentally different from Alexa or Google Assistant, and what trade-offs come with local voice detection.
What Is Voice Activity Detection?
Voice Activity Detection (VAD) is a machine learning task that answers one question: "Is someone speaking right now?" It listens to audio and classifies each small chunk (usually 64 milliseconds) as either speech or silence. This sounds simple, but it's harder than you'd think. A cough sounds like speech. A door slam has acoustic properties similar to consonants. A dog barking might fool a naive detector. A robust VAD has to distinguish true speech from background noise, music, mechanical sounds, and other audio that isn't human voice.
For hands-free voice assistants, VAD is essential. Without it, you'd need to press a button before every command. With VAD, you can say "Hey, what's the weather?" and the device starts listening, detects the sound wave pattern of your speech, and stops listening when you finish. All without touching the phone.
MyBenAI uses Silero VAD, an open-source, on-device voice detector trained on thousands of hours of multilingual speech. It runs entirely on your phone—no audio leaves the device while listening.
False Positives: The Central Problem
The biggest engineering challenge in VAD is false positives. A false positive happens when the detector misidentifies non-speech as speech. For example, if you're in a coffee shop and someone laughs loudly near you, the detector might think *you* are speaking and start streaming audio to the transcription engine. Or if a motorcycle passes, the detector might trigger.
Each false positive is frustrating and wasteful. Transcribing noise burns CPU and battery. If audio were sent to the cloud (as with Alexa), every false positive means unnecessary data transmission. If the AI misunderstands the noise as a command, you get annoying misactions.
Silero VAD is tuned for low false-positive rates. It uses a neural network trained on speech versus non-speech patterns, and it applies a confidence threshold: it only fires when it's reasonably certain the audio is human speech. On typical devices with normal background noise, false-positive rates are below 1%, meaning the detector rarely mistakes noise for speech. In very noisy environments (construction sites, loud parties), false positives increase slightly, but stay below 5%.
Latency: Detection Speed Matters
Latency in VAD is the delay between when someone starts speaking and when the detector recognizes it. If latency is too high (say, 500 ms), the user's first few words feel delayed—they speak, pause waiting for the phone to respond, then realize the phone is just now starting to listen. This creates a broken user experience.
Silero VAD processes audio in 64 ms chunks, which means it can detect speech with a latency of roughly 100–200 ms on mid-range phones. This is fast enough to feel natural. A user says the first word, and within 150 ms, MyBenAI has recognized speech is happening and started transcription. By the time the user is on the third word, full transcription is running.
This low latency is only possible because VAD runs locally. If you had to send audio to a remote server to decide whether speech is present, the network round-trip alone (50–100 ms each way, plus server processing) would inflate latency. This is why Alexa uses a local acoustic model for wake-word detection and only then sends audio to the cloud.
How Silero VAD Works Technically
Silero VAD is a convolutional neural network optimized for speed. It takes raw audio as input (sampled at 16 kHz) and outputs a confidence score for each 64 ms frame. The architecture is deliberately lightweight—it's less than 1 MB in size, so it loads instantly on startup. On a modern phone, it processes audio faster than real-time (one second of audio takes 200–300 ms to process), leaving plenty of headroom for other tasks.
The model was trained on multilingual data, so it recognizes speech in English, Spanish, German, French, Russian, Chinese, and others. This doesn't mean it's equally accurate in all languages—it's more accurate on English and European languages, and slightly less accurate on tonal languages—but it works as a functional detector across the board.
Crucially, Silero VAD makes decisions purely on acoustic patterns. It doesn't use deep learning to embed audio and compare it against a database of known voices. It simply asks: "Does this 64 ms chunk sound like human speech?" This is a binary, stateless classification task, which is why it's so fast and simple to run locally.
Silero vs Cloud-Based Listeners: Alexa and Google Assistant
Amazon Alexa and Google Assistant also use VAD, but with a critical difference. They use local VAD for wake-word detection (the "Alexa" or "Hey Google" part), but then send all subsequent audio to the cloud for transcription and processing. The local VAD serves as a gate: "Did someone just say the wake word? Yes? Then start streaming audio to AWS."
Why do they do this? Several reasons. First, cloud-based ASR (automatic speech recognition) is far more accurate than on-device. Cloud ASR also handles context and follow-ups better—a complex sentence like "Remind me to buy milk when I get to the grocery store" requires understanding intent, not just transcribing words. Second, Amazon and Google benefit from sending audio to the cloud. They can log commands, detect patterns, sell advertising data, and train better models on real-world usage.
MyBenAI's approach is different. It runs both VAD and transcription locally. No audio is streamed to any server. After Silero VAD detects speech, MyBenAI immediately starts transcribing using Whisper, a local speech-to-text model. Both detector and transcriber run on your device. This means your voice data never leaves your phone at any stage.
Always-Listening Without Sending Audio to the Cloud
A common misconception is that "always-listening AI" necessarily means a backend server is recording you. That's the Alexa model: the device is always listening locally, but as soon as it detects speech, it sends audio to AWS. You're trusting Amazon to handle the data ethically.
On-device VAD enables a different architecture: the phone listens, makes decisions, and transcribes, all without uploading audio. If the VAD fires and Whisper produces a transcription, the text (not the audio) is fed to the language model. The audio file is discarded. Your *voice* never leaves your device—only the final text does. And since MyBenAI works offline, even the text stays on your phone.
This is why Silero VAD is local-only by design. A VAD that uploaded raw audio to the cloud for classification would defeat the purpose. Silero was built as an open-source alternative for people who want hands-free listening without cloud involvement at any step.
MyBenAI's Hands-Free Modes
MyBenAI offers two voice input modes: push-to-talk and hands-free. Push-to-talk is simpler—you hold a button while speaking, release to transcribe. Hands-free uses Silero VAD. You can use two variations:
- Tap-to-start hands-free: You tap a button once to start the VAD listening, and the VAD runs until you finish speaking (detected via silence timeout). Useful if you want to trigger the listener explicitly but don't want to hold a button.
- True always-on hands-free: The VAD listens continuously in the background. When it detects speech, it starts transcription. When it detects silence for 2+ seconds, it stops and sends the transcript to the AI. This requires more battery and processing, but gives the most natural experience.
The trade-off is battery and CPU. Continuous VAD running in the background consumes roughly 2–4% battery per hour, depending on ambient noise. In a quiet environment, it's closer to 1%. In a loud cafe, it's higher because the neural network works harder to distinguish speech from noise.
The Accuracy Trade-Off: No Perfect VAD
Silero VAD is highly accurate for mainstream English speech, but accuracy depends on several factors. In a quiet room with clear speech, false positives are under 0.5%. In a moderately noisy environment (office, casual outdoor), they're around 1–2%. In very loud places (construction, airport, concert), false-positive rates rise to 5–10%.
False negatives (failing to detect actual speech) are generally lower. The model is calibrated to err on the side of caution: if there's any chance someone is speaking, it triggers. This minimizes missed commands, at the cost of more false positives in certain environments. You can adjust the confidence threshold in MyBenAI settings to make it more or less sensitive.
Non-English speakers will find Silero's performance varies by language. Romance and Germanic languages (Spanish, French, German) are well-supported. Tonal languages (Mandarin, Cantonese) and languages with heavy regional accents sometimes see 5–10% higher error rates. The model was trained primarily on European and American English speakers, so accent diversity affects performance.
Privacy by Design
Running VAD locally means audio is never sent anywhere. Your voice stays on your device. The transcribed text itself is stored locally by default—you only need network access if you explicitly enable optional cloud web search or custom connectors. MyBenAI does send anonymous usage analytics on by default (installation ID, device tier, model name, aggregate token counts—but never your message content or voice). This enables crash reporting and feature usage insights. Your message content is genuinely excluded and never leaves the device.
This is fundamentally different from Alexa, which sends audio streams to AWS whenever it detects the wake word. It's also different from Google Assistant on Android, which can send partial audio for on-device processing but reserves the right to log and use data for model training. MyBenAI's architecture puts the privacy boundary explicitly at the device level: your voice and message content are your own.
Real-World Performance
On modern phones (2022 or newer), Silero VAD runs with negligible impact. It adds less than 1% CPU overhead when idle and rises to 3–5% when actively listening. On older phones (2018–2021), the overhead is slightly higher, around 5–8% when listening. The memory footprint is minimal—the model is under 1 MB, and the runtime uses minimal buffers.
Battery drain during hands-free listening (always-on VAD) is typically 1–4% per hour, as mentioned. This is comparable to having the screen on medium brightness, and far less than streaming audio to the cloud would cost. A 30-minute hands-free conversation consumes roughly 1–2% of a typical smartphone battery.
Limitations and Honest Trade-Offs
Silero VAD is not as sophisticated as cloud-based speech detection. A cloud VAD can use the full audio waveform, apply acoustic models trained on billions of samples, and leverage context from previous utterances. Silero makes decisions on 64 ms windows using a lightweight model. In extremely noisy or complex acoustic environments, cloud-based detection would win.
Additionally, on-device VAD means you lose cloud benefits like automatic language detection (Silero VAD cannot reliably tell you whether someone is speaking English or Spanish; it just detects speech). You also don't get cloud-based speaker identification or emotion detection. If your use case requires those, you'd need to use a cloud service. MyBenAI assumes you'd rather trade accuracy for privacy—and we think that's the right call for personal AI.
Getting Hands-Free Working on Your Phone
MyBenAI's hands-free mode is enabled by default on iOS and Android. Open Settings, go to Voice, and toggle between push-to-talk and hands-free. If you're on an older phone with limited RAM, hands-free may be grayed out (VAD requires at least 2 GB free RAM to run smoothly). For push-to-talk, there's no minimum.
Test hands-free in a quiet environment first. Speak clearly, pause between sentences to let the VAD detect silence, and see how it feels. Adjust the confidence threshold (Sensitivity slider in settings) if you want fewer false positives (turn down sensitivity) or faster triggering (turn up sensitivity).
Hands-free voice input is one of the most natural ways to use an AI assistant. Silero VAD makes it possible without surrendering your voice to the cloud. Ready to experience truly private voice control? Get MyBenAI today and try hands-free mode. For more on how on-device speech-to-text works, check out Offline Speech-to-Text: Private Transcription With Whisper. Learn how to build a voice-first workflow in Transcribe Meetings Offline: Free Alternative to Otter.ai, or explore the full voice assistant experience in Voice Assistant Without the Cloud.