How AI hears accents: An audible visualization of accent clusters

I'm deaf. Something close to standard Canadian English is my native language. Most native English speakers claim my speech is unmarked but I think they're being polite; it's slightly marked as unusual and some with a good ear can easily tell it's because of hearing loss.

Using the accent guesser, I have a Swedish accent. Danish and Australian English follow as a close tie.

It's not just the AI. Non-native speakers of English often think I have a foreign accent, too. Often they guess at English or Australian. Like I must have been born there and moved here when I was younger, right? I've also been asked if I was Scandinavian.

Interestingly I've noticed that native speakers never make this mistake. They sometimes recognize that I have a speech impediment but there's something about how I talk that is recognized with confidence as a native accent. That leads me to the (probably obvious) inference that whatever it is that non-native speakers use to judge accent and competency, it is different from what native speakers use. I'm guessing in my case, phrase-length tone contour. (Which I can sort of hear, and presumably reproduce well, even if I have trouble with the consonants.)

AI also really has trouble with transcribing my speech. I noticed that as early as the '90s with early speech recognition software. It was completely unusable. Even now AI transcription has much more trouble with me than with most people. Yet aside from a habit of sometimes mumbling, I'm told I speak quite clearly, by humans.

Hearing different things, as it were.

I tried the oracle and got this:

> Your accent is Dutch, my friend. I identified your accent based on subtle details in your pronunciation. Want to sound like a native English speaker?

I'm British; from Yorkshire.

When letting it know how it got it wrong there's no option more specific than "English - United Kingdom". That's kind of funny, if not absurd, to anyone who knows anything of the incredible range of accents across the UK.

I also think the question "Do you have an accent when speaking English?" is an odd one. Everyone has an accent when speaking any language.

The Australian-Vietnamese continuum is well-explained by Australia being the geographically nearest region which can supply native English language teachers to English language learners in Vietnam, rather than by any intrinsic phonetic resemblance between Vietnamese and Australian English.

> This voice standardization model is an in-house accent-preserving voice conversion model.

Not sure this model works really well. As a french/spanish native speaker, I can immediately recognize an actual French or Spanish person speaking in english, but the examples here are completly foreign to me. If I had to guess where the "french" accent was from I would have guessed something like Nigeria. For example spanish have a very distinct way of pronouncing "r" in english that is just not present here. I would have been unable to correctly guess French or Spanish for the ~10 examples present in each language (mayyybe 1 for French).

Since our own accents generally sound neutral to ourselves, I would love someone to make an accent-doubler - take the differences between two accents and expand them, so an Australian can hear what they sound like to an American, or vice-versa

Going mono-tonal to that of an expressive ebook increased my "American English" score from a 52% to 92%.

I'd suggest training a little less on audio books.

I just got a project running whereby I used python + pdfplumber to read in 1100 pdf files, most of my humble bundle collection. I extracted the text and dumped it into a 'documents' table in postgresql. Then I used sentence transformers to reduce each 1K chunk to a single 384D vector which I wrote back to the db. Then I averaged these to produce a document level embedding as a single vector.

Then I was able to apply UMAP + HDBSCAN to this dataset and it produced a 2D plot of all my books. Later I put the discovered topic back in the db and used that to compute tf-idf for my clusters from which I could pick the top 5 terms to serve as a crude cluster label.

It took about 20 to 30 hours to finish all these steps and I was very impressed with the results. I could see my cookbooks clearly separated from my programming and math books. I could drill in and see subclusters for baking, bbq, salads etc.

Currently I'm putting it into a 2 container docker compose file, base postgresql + a python container I'm working on.

This is fascinating in theory, but I'm confused in practice.

When I play the different recordings, which I understand have the accent "re-applied" to a neutral voice, it's very difficult to hear any actual differences in vowels, let alone prosody. Like if I click on "French", there's something vaguely different, but it's quite... off. It certainly doesn't sound like any native French speaker I've ever heard. And after all, a huge part of accent is prosody. So I'm not sure what vocal features they're considering as "accent"?

I'm also curious what the three dimensions are supposed to represent? Obviously there's no objective answer, but if they've listened to all the samples, surely they could explain the main constrasting features each dimension seems to encode?

Apparently Persian and Russian are close. Which is surprising to say the least. I know people keep getting confused about how Portuguese from Portugal and Russian sound close yet the Persian is new to me.

Did research on accent, pronunciation improvement, phoneme recognition, kaldi ecosystem, etc … nothing really changed in the public domain past few years. There’s no even accurate open source dataset. All self claimedccc manually labelled dataset with 10k+ hours was partly done with automation. Next issue, model models operates in different latent space often with 50ms chunks while pronunciation assessment requires much better accuracy. Just try to say B loud - silent part gathering energy in the lips, loud part, and everything what resonates after. Worst part there are too many ml papers from the last year students or junior phd folks claiming success or fake improvements, etc

The article itself is just a vector projection in 3d space … the actual reality is much complex.

Any comments on pronunciation assessment models are greatly appreciated

The source code for this is unminified and very readable if you’re one of the rare few who has interesting latent spaces to visualize.

https://accent-explorer.boldvoice.com/script.js?v=5

It would be interesting to do a wider test like this but instead of trying to clump people together into "American English" and "British English" it would be interesting if the data point was "in which city do people speak like you do?" and create a geographic map of accents.

I'm from the south of Sweden and I've had my "accent" made fun of by people from Malmö just because I grew up outside of Helsingborg, because the accent changes that much in just 60 kilometers.

I'm a native italian speaker with polish ancestry.

I've tried to do the accent oracle test few times and it catches me being Italian with a 90%+ confidence.

The interesting thing is that if I try to fake a more english accent like American...it tells me I'm polish.

Which is odd because I don't really have a polish accent and don't speak it that well. I sound Italian even in Polish.

https://start.boldvoice.com/accent-oracle

Fascinating! How did you decouple the speaker-specific vocal characteristics (timbre, pitch range) from the accent-defining phonetic and prosodic features in the latent space?

The fact that they believe there to be a single 'British' accent means this can be quickly discounted as nonsense.

Very nice viz. it reminds me of the visualizations people used to do of the mnist data set in the days when the quintessential ML project was “training a hand writing digits classifier”: https://projector.tensorflow.org/

I am curious - why UMAP not t-SNE? (See https://pair-code.github.io/understanding-umap/) When I saw the vis, there is a a collection of lines, which look as an artifact. t-SNE (typically) gives more "organic" results of blobs, provided you set perplexity high enough.

Also, while I admire examples of instances, it would be interesting to the map or original laguages - which is close to which, in terms of their English accents.

Why do the voices all sound so similar? I'm not talking about accent, I'm talking about the pitch, timbre, and other qualities of the voice themselves. For instance, all the phrases I heard sounded like they were said by a medium-set 45 year old man. Nothing from kids, the elderly, or people with lower / higher-pitch voices. I assume this expected from the dataset for some reason, but am really curious about that reason. Did they just get many people with similar vocal qualities but wide ranges of accents?

BERT still making headlines in 2025, you love to see it.

Thank you for sharing! the 3d visual was an interesting application of the UMAP technique.

Is there a way to subscribe to these blog posts for auto-notification?

All the accents sound like somebody from... somewhere in the third world...? but with a small trace of the named accent. I don't know if that's intended - maybe the different recordings are not supposed to sound like their label but like a foreigner who learned English while around people with that accent?

Try yourself here: https://start.boldvoice.com/accent-oracle

It wrongly pegged me as Swedish.

It's second choice was the place I live, and third place was where I'm from, so not too bad overall. I have been told I have a very ambiguous accent though.

really fun discovery clicking a dot and hearing the accent. neat visualization, lots to think about!

whats the dimensionality of the latent space? How were the 3 dimensions visualized selected?

It would be really cool if it could highlight the parts of the speech that gave you away your accent. It guesses mine correctly most of the time (though not the first time I tried), but also lets me know my accent is pretty light.

Irish accent appears to break it.

It got me, native English speaker with British accent.

I was hoping it might drill down into regional accents though, there is a huge variety in the UK. I have a Midlands accent which can occasionally confuse non-native speakers.

Note: this is related to https://news.ycombinator.com/item?id=42392088 from a few months ago.

Apparently my accent is "Application Error".

why is spanish so distributed?

Fascinating work — especially how geography and history influence accent clustering more than language families. Brilliant visualization!

"Audible visualization" is a visualization enhanced by auditorization. :)

it would've been nice to be able to visualize the differences between the different accents in the spanish language, really cool tho

Very interesting

Fascinated by the cluster of Australian, British and South African. As an Australian living in UK, I hear an enormous difference between these accents - even just in the British ones, the Yorkshireman and the Geordie stick out like a sore thumb to me - the narcissism of small differences perhaps. Interestingly, my partner, who is from England, often says, of various Australians we hear (either on TV or my friends), that they sound British to her. I, meanwhile, can pick an Australian from very few words. What are we hearing differently? It is a mystery to me.

this is super cool!

i love boldvoice

This is a fascinating look at how AI interprets accents! It reminds me of some recent advancements in speech recognition tech, like Google's Dialect Recognition feature, which also attempts to adapt to different accents. I wonder how these models could be improved further to not just recognize but also appreciate the nuances of regional