AI and minority languages

2026-05-18T00:00:00+00:00

Last week I attended an excellent conference in the Irish consulate in San Francisco titled “AI & Minority Languages: A Bay Area Perspective”.

Our team at thoughtbot speaks over 25 different languages, from Swedish to Bekwarra, so this conference was of particular relevance. Here are my key takeaways from the discussions.

Who gets in the ark?

AI poses both an opportunity and an existential risk for minority languages. While these are typically languages whose speakers are fewer than those of another group within a defined area, in this context it more closely relates to languages that are not one of the world’s dominant languages such as English, French, German, Spanish, Russian or Chinese (including the likes of Mandarin and Cantonese).

On the one hand, new AI-powered tools like Abair from Trinity College Dublin (for Irish/Gaeilge), Aina (for Catalan) and Project Vaani (for a whole range of Indian languages and dialects), can create new ways for people to interact with minority languages and dialects.

But the picture is not all rosy. AI is accelerating the loss of languages with 97% of the world’s languages now being categorised as “in danger”.

That is partly because language is not just about how many people speak it. It’s about usability; what you can achieve using your language. If a minority language is no longer useful in the modern world, it becomes associated with the past. And languages that are associated with the past die out. With the proliferation of AI, for the first time in history, you could have a language spoken by 20 million people that could actually be in danger because it’s about to be drowned out in the present technological wave.

The old global divide was about access; do you have a device, a connection, an account? The new divide is around quality. If a doctor can use technology to get decision support in English but a doctor looking for support in Swahili gets only noise, the language is going to be in danger.

We’re at an inflection point and the implications of not making it into the ark are profound.

What are the problems?

Biased data

The current imbalances largely occur due to gaps in the data. 93% of ChatGPT3’s training data was in English. Therefore, the language you speak determines your access to and effectiveness with the technology. If the data used to train models is only in a handful of primary languages, we will have less diversity and fewer useful living languages.

Defining success in different languages

Public success metrics for performance in general reasoning models nearly always relate to how proficient a model is in English. But if you assess a model in terms of multilingual performance, the success rate is much lower.

For example, Claude Sonnet 4.5 performs at 94% in general reasoning in English, but at only 76% in Yoruba. That is quite a disparity, and it likely increases when compared to even more marginalised languages.

Furthermore, 28% of questions that are asked of an LLM require culturally sensitive knowledge, which becomes increasingly difficult to accommodate the more marginalised a language is.

Accessibility

Language is subtle and nuanced. There can be a difference between accuracy and understandability. An AI system can be technically correct, but it still might miss out on nuances in different languages.

A simple example is that an AI system might say 25%, whereas humans might more commonly say “one in four”. It’s the same outcome, but expressed differently. Humans also use metaphors whereas AI tends to rely on literal descriptions. The question isn’t whether AI can do something, but how does AI communicate meaning.

A humorous example of this failing came during an Eleven Labs demo of an AI voice product which was tasked with creating some Irish folk tunes. While it completed the task well, the voice agent, despite speaking in the voice of an American man, decided to name itself “Aoife”. For those who don’t know, Aoife is a very popular Irish girls name. The system clearly knew enough to select an Irish name, but it clearly didn’t understand the name itself. It missed the nuance.

Computational power

Dr. Mochamed Asri brought up one of the most fascinating issues discussed during the conference. He explained that, while models might be getting better, the tokeniser is seriously lagging behind. It focuses on English, and it needs to get much better to bring parity to different languages.

Over 90% of model training data is in English. Because of this familiarity, it’s easy for the LLM to assign a single token to words. For example, the word “community” in English registers as a single token. But the word “masyarakat”, which is Japanese for the same word “community”, because it is not recognised, gets split into 4 different parts; “mas”, “ya”, “ra” and “kat”. That’s four tokens, which means four times the computational power, for the same meaning.

Energy gap

Computing power leads us on to the energy gap issue. Unfortunately, the countries with the greatest need for extra compute power because of the tokeniser imbalance, are the ones least equipped to provide the necessary energy, which prevents countries from making their own models.

For comparison, the United States has a population of ~350 million people and an electrical capacity of ~1,200 GW. Indonesia, with a population of ~288 million people, has a capacity of only ~80GW. Kenya, with a population of ~59 million people has only ~4GW of electrical capacity.

Bystander problem

AI can contribute to bystander syndrome. We might think someone else will work on adding and protecting our languages, that there’s no need for me to take any action. But the reality is that big frontier models will not do this work for us.

Improving models but more to do

The frontier models are improving, which is a step in the right direction.

During his talk, Zach Parent of OpenAI, demonstrated how ChatGPT 3.5 turbo performed much worse than ChatGPT 5.5 at Irish language tasks. The test demo was simple; he asked each version of the model a question in English but asked it to give the answer in Irish. Then he asked both to translate the Irish answer back to English. 5.5 gave a pretty accurate response, while 3.5 outputted mostly gibberish.

This demonstrated that over 3 years, it has improved a lot. But Zach highlighted that while everyone focuses on training the model, the other steps in the process are where the gaps exist.

The models improve by putting more data through them. The process needs to start with Automatic Speech Recognition, or ASR, to turn spoken word into text. The model then needs to understand the inputs before applying Text To Speech (TTS), followed by a real-time agent to make it conversational.

Each of these steps requires high quality data that is not easy to capture. In particular, there is a real gap when moving from text to voice.

How to fix it

High Quality data

We need more high quality data and representation in the models. This is not just about having more translations, but actual high quality input.

Processes like transcribing voice to text is manual, it’s labour intensive and requires native speakers. Once it’s done, that data can get ingested and the model gets trained on it.

Government support

That transcription piece won’t just happen on its own, it can’t just be automated. Governments, non-profits and universities need to step in to support this work.

Small Language Models (SLMs)

Small language models offer some promise, especially in relation to compute power and the energy gap. Again, governments will need to provide supports to build SLMs and edge deployments.

Improved tokeniser

Even as the models improve, we won’t get language parity until the tokeniser improves and becomes more diverse, reducing compute power constraints for minority languages.

Accountability

As a society, we need to hold frontier models more accountable when they publish performance metrics. Applying pressure to disclose cross language performance metrics will help highlight the gap and, hopefully, will lead to further action.

Get creative

One of the most unique, creative ways to drive a minority language and culture into the modern age came from the Iñupiat, an Alaska Native people. Gloria O’Neill of the Cook Inlet Tribal Council explained how they created a puzzle-platformer video game called Never Alone (Kisima Inŋitchuŋa) which is based on a traditional story passed down through generations.

The game was created in partnership with E-Line Media. With over 15 million players worldwide, the reception of Never Alone: Kisima Ingitchuna launched a movement of social-impact video games. It went on to win a Peabody Award for its storytelling and a BAFTA for Best Debut Game. Eight years later, Never Alone 2 is on the cusp of being released.

I was inspired and blown away by the totally outside-the-box thinking of the Iñupiat people to preserve their language and culture.

A closing thought

AI has endangered more languages than ever before. However, there are still ways that we, as minority language speakers, can preserve our languages. We can create the high quality data required to train the models, push corporations and governments to accommodate and support minority languages, or think completely differently about how to make our language relevant in the AI age.

We are at an inflection point and we need to take ownership and accountability to get our languages onto the ark.