Vasiliy Kudryavtsev and colleagues separate what a transcript-cleanup pass fixes from what it breaks. Average word error rate can hide both. Their study compares editing with and without the original audio, using the model’s own draft or an independent speech recognizer’s output.
Gemini 2.5 Flash is the main model. Matched samples contain 2,260 synthetic Russian clips, 270 Russian audiobook clips and 284 English medical-dictation clips. The independent recognizers are GigaAM for Russian and Whisper Large-v3-Turbo for English.
- Text-only self-correction damages correct inputs. It introduced errors into 31.9%, 16.3% and 12.8% of already-correct drafts across the three collections. Audio-backed correction reduced those rates to 2.6%, 1.0% and 2.3%.
- Fewer damaged words can accompany fewer repairs. Adding audio reduced both flows when Gemini edited its own drafts. The natural-speech samples showed no clear net word-error improvement over text-only self-correction.
- Independent transcripts behaved differently. With another recognizer’s output, audio-backed correction increased repairs and improved overall word-error scores on all three collections.
- The pattern did not generalize to Voxtral-Small-24B. That model rarely edited text-only inputs. Adding audio increased repairs, including on its own drafts, while damage moved inconsistently.
One medical example makes the risk concrete: text-only editing changed the correct phrase “loss of hair” into “loss of hearing.” The audio-backed pass preserved it. Other changes concern spelling conventions, such as joining “post operative,” rather than meaning. The reported break-rates include those differences.
The authors recommend reporting repaired words, damaged words, insertions and the share of correct transcripts broken alongside the aggregate score. These measures also need interpretation: replacing one wrong drug name with another counts as neither repair nor damage under their accounting.
Caveat: Prompts changed with audio availability, so the study cannot isolate audio’s effect; the small natural-speech samples also make break-rates descriptive rather than precise population estimates.
These are author-reported experiments on public speech collections, not clinical outcomes. For a capture pipeline, evaluate the editing pass separately and preserve the original transcript instead of treating smoother wording as proof of improved accuracy.