AI Bias, or: When the Algorithm Looks at Us

When we think about artificial intelligence, or algorithms in general, it's easy to picture a cold, logical, completely objective computer. After all, a machine doesn't wake up in a bad mood or with an unjustified preference, right? Well, not quite. The AI that has been changing all our lives in recent years, whether we admit it or not, is far more human than we think, including our less attractive sides.

Let's say it loud and clear: even though it's a machine, it is anything but objective. If you like, AI models are a bit like children. They come into the world without fixed ideas, genetic wiring or thought patterns, but they learn about the world by observing what happens around them: the texts we write, the photos we take, and the information we share online. They don't receive precise instructions the way traditional software does. Instead, they learn in an open-ended way and develop according to what they "see" in the data.

And that's exactly the problem: this data, the world we've created, is anything but neutral.

Models learn from us humans, and they copy our biases and stereotypes too. If there are biases against certain groups in the real world, and there certainly are, they pass straight into the models.

For example, facial recognition systems struggle to identify people with darker skin, because they were trained mostly on images of white people. A study conducted at MIT found that facial recognition technologies are significantly less accurate for darker-skinned women than for white men. Credit scoring systems have discriminated against women in many cases, because the historical data the models were trained on reflected existing gender gaps in the labor market and the financial system.

Hiring algorithms have also been shown to be biased. Amazon, for example, discovered that its automated recruiting system was rejecting female candidates, because the model had learned to prefer male candidates based on the company's hiring history.

Even translation systems can perpetuate gender bias, for instance by automatically linking certain professions to a particular gender (like doctor with man and nurse with woman), which reinforces existing stereotypes.

This is exactly where the difference between AI and classic software becomes critical.

In traditional software, errors can be fixed fairly simply, because the code is clear and the consequences are known in advance. But with AI, bias is buried deep in the data and in the model itself, and it can be almost invisible, until the moment it harms minority or marginalized groups.

This has become a lively global conversation that can't keep up with the pace of innovation: the European Union is advancing strict regulation that will require companies to test for and reduce bias in their algorithms. In the US, major organizations like OpenAI, Google and Microsoft face public and regulatory scrutiny over questions of fairness, transparency and accountability. Public committees, researchers and activists are demanding a better understanding of how AI systems make decisions.

From the moment generative AI (GenAI) became available to everyone, discussions began about bias related to gender and ethnic groups.

But let's talk about people with disabilities: one group among many that tends to be left out of the picture when models are trained.

What happens when AI systems don't know their experience, their language, or the ways they use technology?

This very group, which especially needs technology to understand it and serve its needs, is barely represented in the model training process.

What happens when the AI we develop simply doesn't know the experience of people with disabilities, the unique language they use, or the ways they interact with the world? In that situation, the models don't just struggle to understand their needs. Sometimes they even flag them as negative or problematic.

For example, recent studies show that natural language processing (NLP) tools consistently label disability-related words and terms, such as "blind," "hearing impairment" or "autism," as negative or problematic, simply because the models were trained on texts that reflect existing prejudice and social stigma.

Instead of helping to build understanding, equality and inclusion, technology reinforces existing social discrimination, increases exclusion, and makes it harder for people with disabilities to participate equally in a digital, technological world that has become an inseparable part of our lives.

These biases create not only inequality but also a deep sense of alienation and not belonging among the very groups these systems are meant to serve. People with disabilities often face technologies built without enough understanding of their lives, their needs and their everyday experiences. Instead of supporting them and enabling them to participate, technology ends up reinforcing the feeling that they've been left out.

The solution is clear, though challenging: we need to, and must, involve people with disabilities directly in developing and training models, so that technology comes to know their world by listening to their authentic voices, rather than relying on outside, biased perspectives. Only then can we ensure that technology, evolving at a dizzying pace, truly serves everyone and contributes to a more equal and inclusive society.