Judgment & risk

Bias, briefly

Why AI can be unfair, and the checks that catch it before a customer does.

5 min readPlain English

Read it in

In one sentence

AI learns from human-made data, so it can repeat that data’s stereotypes and blind spots; the fix is to test for it and keep people in charge of decisions about people.

Think of it like…

Picture a new hire who learned the job only from decades of old office files. They’d pick up plenty of useful know-how, and the old habits too, like assuming the manager is a man. Nobody told them to be unfair; the files did.

How it works

Bias can creep in at three points. You can catch it at a fourth.

1The dataModels learn from huge amounts of human writing and images, which over-represent some people and places and repeat old stereotypes.
2The designChoices about how a tool is tuned and what it’s told to avoid reduce some biases and can miss others.
3The askPrompts about “a typical customer” or “a strong candidate” invite the model’s default assumptions.
4The checkSwap one detail (a name, gender, age or neighborhood) and compare the answers. Replies vary a little each time, so differences that keep following the changed detail point to bias.

Providers work to reduce bias, but it isn’t solved, so the last check is yours.

Good at

Flagging loaded or off-putting wording in a job ad
Rewriting copy for a wider audience
Suggesting perspectives you might have missed

Bad at

Noticing its own assumptions
Representing people and places that are thin in its data
Making fair decisions about individual people

Where you’ll see it at work

A job ad at a design studio. AI flags wording that might put some applicants off. The owner decides what to change.
Marketing images at a bakery. Image tools can default to a narrow idea of who your “customers” are. Describe the range of people who actually shop with you, and check before posting.
Customer replies at an online shop. The same complaint, run with two different customer names, should get the same tone and the same offer.

Myth vs reality

Myth: “A computer can’t be biased: it’s just math.”

Reality: The math learns from human data, so it can repeat human patterns, unfair ones included, at scale and in a confident tone. Treat its defaults as a starting point, not a neutral one.