AVVF Lab
The Automated Voice Validation Framework: a customizable testbed for measuring the real-world performance of any voice-controlled device, independent of the ASR engine behind it.
Request a QuoteWhat Is the Automated Voice Validation Framework?
The Automated Voice Validation Framework (AVVF) is a Novus Labs-built testbed that measures how well any voice-controlled device understands and responds to speech, independent of the ASR engine behind it. Whether your device runs Alexa, Google Assistant, Siri, or a proprietary in-house engine, the AVVF measures what your end user actually experiences: speech-recognition accuracy, latency, and behavior under real environmental noise.
It is a robust, customizable validation environment for testing, tuning, and improving voice-controlled hardware and software. Each test campaign is tailored to the device under test and the scenarios it must perform in, from quiet living rooms to noisy retail floors, and it plugs into the rest of our audio testing program, including characterization in dedicated audio chambers.
- Cloud-service agnostic: works with any ASR engine or backend
- Customizable test scenarios for near-field and far-field applications
- Adaptive learning algorithm automatically determines correct responses
- Custom utterance generation via Speechify, Murf AI, or other TTS tools
- Endpoint and environmental-noise variable coverage
What the AVVF Delivers
Four capabilities that let the AVVF adapt to any voice-controlled device, any ASR backend, and any acoustic environment your product needs to perform in.
- Works with any cloud-based voice recognition service
- Alexa, Google, Siri, or proprietary in-house engines
- No vendor lock-in
- Testbed customized per device
- Models the operational environment the product ships into
- Scales from quiet rooms to noisy retail floors
- Automatically determines correct responses
- End-to-end test scenarios out of the box
- No manual reference-answer input required
- Audio prompts generated from text
- Integrates with Speechify, Murf AI, and other TTS tools
- Drop-in to any test workflow
Near Field and Far Field, in One Room
The AVVF supports both predefined test cases and fully customized test scenarios, covering both near-field (close-range, conversational) and far-field (across-the-room) speech recognition use cases.
Speakers positioned around the Device Under Test (DUT) at varying distances simulate the real spatial conditions your product encounters in customer environments, from a phone held up to a user's mouth to a smart speaker fielding voice commands from across the room.
Proven Against Real Devices and Real Noise
To validate the framework itself, Novus implemented a custom endpoint based on the XMOS-3500 platform and interfaced it with several popular voice assistant back-ends. The simulation system was exercised under a wide variety of adverse system tunings and environmental noise conditions to verify it performs reliably across every stage of a product's life cycle.
Endpoint System Variables
Each test campaign exercises adverse system tunings, digital signal processor (DSP) and acoustic echo cancellation (AEC) settings, so the framework's behavior is verified across the full range of device configurations a product may ship with.
Environmental Noise Conditions
Iterations run under silence, pink noise, indoor party noise, rock music, and crowd noise, verifying the framework holds up across every acoustic environment a voice product is likely to encounter in customer hands.
Structured Test Report
Each campaign delivers a structured report covering utterance, device response, score, pass/fail status, and latency metrics for every prompt evaluated.
See It Run
A typical AVVF run: speakers positioned around the Device Under Test deliver a scripted utterance sequence, the framework captures the device's responses, and a CLI overlay logs the result of each prompt. The demo below shows two iterations of the same script (the first under environmental silence, the second with added background noise), illustrating how the framework characterizes voice behavior across acoustic conditions.
Voice Validation Questions
What is the AVVF?
The Automated Voice Validation Framework (AVVF) is a Novus Labs testbed that measures how accurately and quickly a voice-controlled device recognizes and responds to speech, for any ASR engine (Alexa, Google Assistant, Siri, or a proprietary stack). It is the voice-assistant arm of our audio testing program.
Which voice assistants and ASR engines can you test?
Any of them. The AVVF is cloud-service agnostic, so it works with Alexa, Google Assistant, Siri, or a proprietary in-house engine, with no vendor lock-in, and it reports speech-recognition accuracy, latency, and pass/fail for every utterance.
Do you test near-field and far-field voice performance?
Yes. Speakers positioned around the device under test at varying distances reproduce both near-field (close-range) and far-field (across-the-room) conditions, and campaigns run under real noise such as pink noise, party, music, and crowd.
Can you test voice products in real-world conditions?
Yes. Beyond the lab, we validate voice and smart-speaker products in our residential test homes and against real device mixes for smart home and IoT programs.
Ready to Validate Your Voice Product?
Talk to a Novus engineer about ASR validation, near/far-field testing, custom utterance generation, or noise-condition characterization for your next voice-controlled product.