HoleRecord's whole premise is that you walk off a green, tap the gold record button, and just say what happened — "pushed my drive right, punched out with an 8-iron, hit the green with a 7, two-putted for bogey" — and your scorecard fills itself in. Score, putts, fairway, green, penalties, the club off the tee. No tapping through menus while your group waits on the next tee.
That magic has a hard dependency most people never think about: the speech-to-text layer has to actually hear golf. And golf, it turns out, is a linguistic minefield. This post is about how we measure that layer honestly, a mistake we caught in our own scoring, and the model upgrade it unlocked.
Why golf is hard for speech-to-text
General-purpose transcription models are trained on the language of the internet and everyday conversation. Golf is not that. It's dense with homophones and jargon that sound like ordinary words:
- "Putt" collapses into "pot," "putted" into "potted."
- "Birdie" comes out "birdy" or "birdie" depending on the day, "bogey" as "bogie" or "bogay."
- "OB" (out of bounds) becomes "oh be," "obi," or just "OB, e."
- "Par 3" versus "par three." "7-iron" versus "seven iron." "Up and down" versus "up-and-down."
A model that transcribes "I potted out for a bogie" instead of "I two-putted for a bogey" hasn't made a small stylistic error — it's handed our downstream pipeline garbage. The whole experience lives or dies on whether the specific golf terms survive the trip from your mouth to the transcript. That's the number we care about, and it's not the number most transcription benchmarks report.
How we measure it: term-level recall
Standard STT benchmarks lean on word error rate (WER) — the share of words inserted, deleted, or substituted versus a reference. WER is useful, but it treats every word as equally important. For us, "the" and "putt" are not equally important. Miss "the" and nothing breaks. Miss "putt" and the scorecard is wrong.
So we built an internal evaluation harness around the metric that actually predicts product quality: term-level recall of golf vocabulary — of the golf-specific terms a golfer actually said, how many made it into the transcript intact? We track that alongside the standard suite:
- Term-level recall — did the golf words survive? (our headline metric)
- Word error rate — the classic overall accuracy check
- Exact match — did the full phrase come back verbatim?
- Model confidence — how sure was the model, token by token?
Running these together on a fixed set of realistic golf phrases turns "the voice thing feels pretty good" into something we can actually track across model versions and regressions.
The bug was in our ruler, not our model
Here's the part we're most proud of, precisely because it started as an embarrassment.
The first version of our harness under-counted accuracy. It flagged transcriptions as wrong when they were, in fact, perfect. The culprit: naive string comparison. The golfer said "seven iron," the model wrote "7 iron," and our scorer marked it a miss. Same for "par three" versus "par 3." The model was right; our ruler was broken.
This is the classic evaluation trap — your eval is only as good as its scoring logic, and a bad scorer will happily lie to you in either direction. An over-generous scorer hides real regressions; an over-strict one, like ours, buries real gains and sends you chasing phantom problems.
We made the harness number-aware: normalize digits and number-words to a common form before comparison, so "7" and "seven," "par 3" and "par three" are treated as the equal expressions they are. The instant we fixed the ruler, the true — and higher — numbers surfaced. The lesson stuck with us: we now treat the eval harness as a first-class piece of software that deserves the same scrutiny as the thing it's measuring.
The model upgrade
With a trustworthy ruler in hand, we could evaluate an upgrade cleanly. We moved our transcription layer from OpenAI's original Whisper (whisper-1) to the newer gpt-4o-transcribe. Three things made it an easy call:
- It's more accurate on our golf vocabulary.
- It costs the same, so there's no tradeoff to agonize over.
- It returns per-token confidence, which — as you'll see — we put straight to work.
On a controlled, clean-audio baseline of 25 realistic golf phrases:
| Metric | whisper-1 (old) |
gpt-4o-transcribe (new) |
|---|---|---|
| Term-level recall | 98.9% | 100% |
| Word error rate | 2.3% | 0% |
| Exact match | 21 / 25 | 25 / 25 |
| Avg. confidence | — | ~0.95 |
Two honest caveats. First, this is a clean-audio baseline — good microphone, no wind, clear diction. It deliberately isolates vocabulary handling from acoustic difficulty, which is exactly what we wanted to measure here. Second, both models do well on clean speech; the point isn't that the old one was bad, it's that the new one is a strict improvement and hands us a signal the old one couldn't.
Confidence signals: knowing when we don't know
That per-token confidence is the quietly important upgrade. Before, when the model was unsure, it guessed — and a silent wrong guess is the worst possible failure mode, because it erodes trust in the one feature that's supposed to feel like magic.
Now, when confidence dips on a golf-critical term, HoleRecord can gently confirm — a quick "did I get that right?" — instead of silently committing a maybe-wrong stat to your scorecard. Knowing when the model doesn't know turns out to be as valuable as raising raw accuracy. It's the difference between an assistant that quietly makes mistakes and one that raises its hand when it's unsure.
What's next: getting off the driving range
Clean-audio numbers are a starting line, not a finish line. Real rounds do not happen in a recording booth. They happen with wind buffeting the mic, a foursome laughing in the background, a cart rattling by, a dozen different accents, and whatever phone happens to be in someone's pocket.
So our next phase is exactly that: recording real rounds out on the course, in genuinely noisy conditions, and running the same harness against messy real-world audio. We expect the numbers to come down — and that's the point. Real-world reliability, not benchmark hero stats, is the bar we're actually trying to clear.
We built HoleRecord so that recording a round feels like telling a buddy how it went, and getting a perfect scorecard for the trouble. Delivering on that isn't a one-time trick — it's a discipline: measure the thing that matters, distrust your own ruler, upgrade when the evidence is clean, and admit exactly where you still have work to do. Say it. We'll score it. And we'll keep sweating the details until we score it right every time.