This guide explains how to use an ASR custom vocabulary to fix transcription errors for domain-specific terms. General-purpose speech recognition models often struggle with product names, industry jargon, and proper nouns, transcribing them as similar-sounding common words. By providing a list of these special terms, you can significantly improve transcription accuracy for your specific use case.

Key takeaways

  • Custom vocabulary improves the recognition of domain-specific terms by telling an Automatic Speech Recognition (ASR) system which words or phrases are likely to appear.
  • This technique typically works through contextual biasing, which increases the probability of specific phrases during the text-generation step without retraining the entire model.
  • While effective for rare terms, setting the "boost" value too high can cause the model to insert your custom terms where they were not spoken, creating false positives.
  • An effective vocabulary list is built from real-world data like product catalogs or support transcripts, and it accounts for spelling, casing, and pronunciation variants.
  • Success is not just a lower overall Word Error Rate; it requires measuring both the recall of your target terms and the rate of false insertions.

What Is ASR Custom Vocabulary?

ASR custom vocabulary is a list of specific words or phrases you provide to a speech recognition engine to improve its accuracy. This list gives the model context, making it more likely to recognize terms that are rare in general language but common in your specific use case, such as transcribing customer support calls or medical dictation (Source).

A general ASR model might hear a customer say "I need help with my AeroFlow X5" and transcribe it as "I need help with my arrow flow ex five." The model chooses the more common words because, based on its training, "arrow" and "flow" have a higher probability than the invented product name "AeroFlow."

Custom vocabulary solves this by telling the model to listen for "AeroFlow X5." Most systems implement this through a process called contextual biasing. This technique temporarily increases the statistical weight of your custom phrases at inference time—the moment the audio is being transcribed. This is different from fine-tuning, which involves permanently retraining the model's underlying parameters with a large, labeled dataset of audio.

How Does Contextual Biasing Work? A Step-by-Step Guide

Contextual biasing gives the ASR decoder a "hint sheet" for your specific audio. It guides the model toward your preferred terms by making them more statistically attractive options during transcription. The process generally follows these steps.

  1. Collect and Normalize Terms. First, gather the domain-specific terms you want the model to recognize. Sources can include product catalogs, user manuals, lists of acronyms, or logs of human-corrected transcripts. Normalize this list by standardizing casing, removing unnecessary punctuation, and deciding how to handle plurals and other variants.

  2. Assign Weights (Boosting). Most systems allow you to assign a "boost" value to each phrase. A boost is a number that tells the engine how much to increase the probability of that phrase (Source). A higher boost makes the phrase more likely to appear in the output. For example, the Web Speech API uses a range from 0.0 to 10.0 (Source). While positive values increase probability, some systems also support negative values for anti-biasing, which decreases the preference for a specific phrase.

  3. Provide the Vocabulary. You pass your list of phrases and their boost values to the ASR engine. This is often done with each API request for transcription. Some systems allow for dynamic biasing, where the vocabulary list can be updated in real-time as the application context changes, for example, based on the specific web page a user is on (Source).

  4. Decoding with Bias. During transcription, the ASR model's decoder generates multiple possible text hypotheses for a given piece of audio. The custom vocabulary influences this process. Methods like shallow fusion combine the probabilities from the core ASR model with scores from your external list, effectively "fusing" the general and specific knowledge (Source). Other approaches use a prefix tree (or trie) to guide the model's predictions toward valid vocabulary terms (Source).

  5. Evaluate Performance. After applying the vocabulary, test it on a representative set of audio. Check if the target terms are transcribed correctly. Just as importantly, check for new errors. A boost value that is too high can cause the model to insert your custom term even when a similar-sounding, common word was spoken.

  6. Tune and Iterate. Based on your evaluation, adjust the vocabulary list and boost values. You may need to add pronunciation variants, remove ambiguous terms, or lower the boost on phrases that cause false positives. The goal is to find the right balance between correctly identifying your terms and not corrupting the rest of the transcript.

  7. Add Downstream Validation. For applications requiring high-reliability product identification, the ASR output alone may be insufficient. Homophones or similar-sounding phrases can still be confused. A final validation step, such as checking the transcribed entity against a product database or using other contextual cues, ensures accuracy.

Worked Example: Recognizing a Product Name

A custom vocabulary can dramatically improve accuracy for specific terms, but it can also introduce new errors if not tuned carefully. The key is to find the right boost value that helps the model without making it overconfident. This balance prevents the system from inserting your terms where they do not belong.

Imagine a customer support call where a user mentions a fictional product, "SynthWave X1."

Scenario: The user says, "Hi, I'm having a problem with my SynthWave X1."

Before Custom Vocabulary

A standard ASR model, unfamiliar with this brand, might produce a transcript based on common English words:

"Hi, I'm having a problem with my synth wave exon."

The transcription is acoustically similar but factually wrong.

The Fix: Applying a Vocabulary List

To fix this, we provide a custom vocabulary list to the ASR engine before transcribing the audio. The list contains the phrase and a moderate boost value.

JSON
[
  {
    "phrase": "SynthWave X1",
    "boost": 4.0
  }
]

After Custom Vocabulary

With this hint, the engine re-evaluates the audio. The contextual bias makes "SynthWave X1" a more probable hypothesis than "synth wave exon," resulting in the correct transcript:

"Hi, I'm having a problem with my SynthWave X1."

The Pitfall: False Positives from Over-boosting

Now, consider a different part of the conversation where the customer says, "...since they've arrived, I've had nothing but trouble."

If the boost value for "SynthWave X1" is set too high (e.g., 10.0), the model can become overly aggressive. It might hear the acoustically similar "since they've" and incorrectly transcribe it as the boosted term.

"...SynthWave arrived, I've had nothing but trouble."

This is a false positive—the custom term is inserted where it was not spoken. This is why Google's documentation warns that a higher boost increases the chance of such errors (Source). Careful tuning and testing are essential to avoid this.

Custom Vocabulary vs. Fine-Tuning: Which Should You Choose?

Custom vocabulary and fine-tuning are both methods for adapting an ASR model, but they work differently and solve different problems. Contextual biasing with a vocabulary list is a lightweight, real-time adjustment, while fine-tuning is a heavier process that permanently alters the model itself.

Feature Custom Vocabulary (Contextual Biasing) Fine-Tuning
Data Required A simple list of words or phrases. A large dataset of audio paired with human-verified transcripts.
Cost & Effort Low. The list is easy to create and update. High. Requires data collection, labeling, and significant compute.
Mechanism Biases probabilities at inference time without changing model weights. Retrains the model's parameters to learn new patterns.
Primary Use Case Recognizing a dynamic or changing set of rare nouns and jargon. Adapting to a new acoustic environment (e.g., noisy factory), accent, or domain.
Flexibility High. The list can be changed for each transcription request. Low. The retrained model is static until the next training cycle.

Choose custom vocabulary when you need to quickly improve recognition of specific, known terms that change often, like product names or a meeting's agenda items.

Choose fine-tuning when the core acoustic or linguistic patterns of your audio differ significantly from the model's original training data, such as adapting a US English model for Indian English speakers.

How Do You Create an Effective Vocabulary List?

Building a high-quality vocabulary list requires a systematic approach, drawing on data and careful tuning. A well-curated list minimizes errors, while a poorly constructed one can degrade performance. Follow these best practices.

  • Source from Data, Not Guesses: Build your list from real-world sources. Use product catalogs, technical glossaries, and logs of human-corrected ASR transcripts to find the terms that are actually being misrecognized.
  • Normalize Everything: Consistency is key. Decide on a standard format for casing (e.g., all lowercase), punctuation, and hyphens. Ensure your list follows the format expected by your ASR provider.
  • Include Variants: A word is often spoken in multiple ways. For "SynthWave X1," you might add "SynthWave X one" or even common mispronunciations. For acronyms like "ASR," include the spoken form "a s r."
  • Keep Lists Short and Relevant: Don't add your entire company dictionary to every request. Larger lists can increase latency and the risk of false positives. If possible, use dynamic lists tailored to the immediate context of the conversation.
  • Beware of Homophones: Be cautious when adding terms that sound like common words. If your product is named "For Sale," boosting it will likely cause transcription errors for the common phrase "for sale."
  • Start with a Moderate Boost: Do not start with the maximum boost value. Begin with a low or medium value, test its impact, and increase it incrementally until you achieve the desired recall without introducing too many false positives.
  • Version and Maintain Your List: Products are launched, renamed, and discontinued. Keep your vocabulary list updated to reflect these changes. Version control for your list is a good practice.

What Are the Common Mistakes and Limits?

While powerful, ASR custom vocabulary is not a silver bullet. Understanding its limitations and common pitfalls is crucial for success.

Common Mistakes

  • The "Bigger is Better" Fallacy: Adding thousands of irrelevant terms to your vocabulary list rarely helps. It can slow down the ASR system and increases the chance that a random utterance is misheard as one of the many items on your list.
  • Forgetting Pronunciation: ASR systems work with sounds, not just letters. For unique names, acronyms, or foreign words, the written form may not be enough. Some systems allow you to provide phonetic spellings to guide pronunciation.
  • Poor Evaluation: Simply checking if your target word appears more often is not enough. You must also check if the overall transcript quality has decreased or if the term is being inserted incorrectly, as discussed below.

Limits of Custom Vocabulary

Custom vocabulary primarily influences the language model component of an ASR system, which deals with word sequences. It does not fix problems with the acoustic model, which translates raw audio into phonetic probabilities.

Therefore, custom vocabulary is not very effective at overcoming:

  • Heavy background noise: If the speech is drowned out by noise, the acoustic model won't produce a clear signal for the language model to work with. Improving ASR noise robustness requires different techniques.
  • Very strong or unfamiliar accents: If an accent is too different from the model's training data, the acoustic model may fail to recognize the basic sounds correctly.
  • Poor audio quality: Low-bitrate audio, microphone distortion, or dropouts will corrupt the signal before the vocabulary can be applied.

It is a tool for choosing between plausible textual interpretations of a sound, not for deciphering an unintelligible sound in the first place.

How Should You Measure Accuracy?

Evaluating the impact of a custom vocabulary requires more than looking at the overall Word Error Rate (WER). A lower WER is good, but it might hide the fact that you've introduced new, critical errors. A more nuanced approach is needed.

Instead of a single metric, you should track a pair of competing metrics:

  1. Target Term Recall: This answers the question: "Of all the times my custom terms were actually spoken, what percentage did the ASR transcribe correctly?" This measures the positive impact of your vocabulary. A high recall means you are successfully recognizing the terms you care about. In one study, a contextual biasing system retrieved 95% of previously misrecognized entities while also achieving an estimated 5% relative word-error-rate reduction (Source).

  2. False Insertion Rate: This answers the question: "How often did the ASR transcribe a custom term when it wasn't actually spoken?" This measures the negative side effect of your boost. A high false insertion rate means your boost value is too aggressive and is damaging the overall transcript.

The goal is to find a balance: maximize recall while keeping the false insertion rate acceptably low. To do this, create a dedicated test set of audio files that contains both your target terms and acoustically similar words or "negative examples" that could trigger false positives. By measuring performance on this specific test set, you can confidently tune your vocabulary and boost values.

What to learn next

Sources and further reading