AI Ethics, Bias, Privacy and Safety

A recruiter in Lahore rang us last year, a little shaken. The CV-ranking tool her agency had piloted for a bank was quietly pushing down every candidate with a gap in their employment history, which in practice meant women returning from maternity leave. Nobody had built it to do that. Nobody had checked that it didn't. That's the whole subject in one phone call. AI shapes decisions about loans, jobs, medical care and what millions of people read, and when it goes wrong it goes wrong at scale and out of sight. Ethics here isn't a philosophy elective. It's a short list of practical checks, ending with one you can run in ten lines of Python.

The six risks, each with its control

1. Bias and unfairness

You met this in the data lesson. Models learn historical unfairness and amplify it through feedback loops. In Pakistan the groups that get hit are usually women borrowers, rural applicants, speakers of regional languages and anyone with a thin credit file. The control is to measure accuracy and outcome rates per group, balance the training data, strip out proxy features, and keep a human on every high-stakes decision.

2. Privacy

Personal data (CNIC, phone, address, salary, health) pasted into a public AI tool may be stored and used for training. Even inside the company, data collected for one purpose shouldn't quietly feed a model for another. The control is to anonymise before use, prefer enterprise deployments with a no-training guarantee, follow the banking and telecom regulators' rules, and write down the purpose and the consent.

3. Hallucination and misinformation

LLMs produce fluent falsehoods. Invented statistics, regulations that don't exist, formulas that look right. One made-up number in a board report can undo months of credibility. The control is to ground answers in retrieved documents (RAG), demand citations, verify numbers against the source system, and keep the temperature low.

4. Security

Prompt injection hides instructions in content the AI reads (a web page, an email, a PDF) to make it leak data or take an action. Agents with tools are the most exposed. Models can also be fed adversarial inputs, and training data can be poisoned. The control is to treat all external content as data, give agents least-privilege tools, require approval for actions, and log everything.

5. Misuse

Deepfake videos, cloned voices for "CEO fraud" calls, fake product reviews, automated phishing in Urdu. The control is a verification step for any payment or credential change requested by voice or video, staff who know these scams exist, and watermark checks where available.

6. Jobs and over-reliance

Tasks change faster than job titles. The risk isn't only displacement. It's skills fading, and decisions getting rubber-stamped because "the AI said so". The control is reskilling, clear accountability (a named person owns each AI-assisted decision), and periodic checks where a human redoes the work by hand.

Principles most frameworks share

Principle	Meaning in practice
Fairness	Similar people get similar outcomes; test per group
Transparency	People know when AI is used and can get an explanation
Accountability	A human owns the decision and can be asked about it
Privacy	Minimum data, clear purpose, secure storage, retention limits
Safety and reliability	Tested under realistic and adversarial conditions; can be switched off
Human oversight	The higher the stakes, the more review before action

Regulation is catching up. The EU AI Act sorts systems by risk, many countries require impact assessments for public-sector AI, and Pakistan's data-protection framework and sector regulators (banking, telecom) increasingly expect documented controls. Even where the law is silent, buyers ask, and in our experience the European buyer asks first.

A fairness audit in ten lines

Say a hiring-screen model was tested on 200 applications. Compare the selection rate between two groups using the "four-fifths rule" that fairness reviews lean on. If one group's rate is under 80% of the other's, you investigate.

results = {
    # group: (recommended for interview, total applicants)
    "Karachi":   (42, 100),
    "Interior":  (22, 100),
}
rates = {g: sel / tot for g, (sel, tot) in results.items()}
best = max(rates.values())
for g, r in rates.items():
    ratio = r / best
    flag = "INVESTIGATE" if ratio < 0.8 else "ok"
    print(f"{g:9s} selection rate {r:.0%}  ratio to best {ratio:.2f}  {flag}")

Interior applicants get selected at 52% of the Karachi rate, well under the 80% line. That alone doesn't prove the model is unfair (qualifications might genuinely differ), but it forces the question, which is the point. Next you'd compare rates among applicants with similar experience, check whether a feature like "university city" is acting as a proxy, and read a sample of rejected Interior CVs by hand.

House rules for analysts

• Never put identifiable customer data into a public AI tool.
• Label AI-generated content in reports and keep the sources.
• Building or buying a model that affects people? Ask for per-group performance before launch, not after.
• Agents get read access by default and write access only through an approval flow.
• If something looks wrong, escalate. "The model said so" is not a defence, least of all in front of a regulator.

How the CV-ranking pilot ended. The audit confirmed what the recruiter suspected. The model favoured CVs from a handful of Lahore and Karachi universities and penalised employment gaps. The agency kept the tool but demoted it. It now extracts structured facts (degree, years of experience, certifications) into a table with no ranking, and recruiters score candidates against a written rubric. Per-group shortlist rates go to the client bank monthly, which turned out to be exactly what the bank's regulator had been asking for.

Before you move on

• The six risks are bias, privacy, hallucination, security, misuse and over-reliance, and each has a practical control.
• Fairness, transparency, accountability, privacy, safety and human oversight are the principles every framework shares.
• Audit per group with a simple check like the four-fifths rule, then go and find the cause.
• You protect your organisation by verifying, documenting and escalating.

Three things to try

1. Extend the audit to three groups and add a gender split. Which combination has too few applicants to judge?
2. Write a three-line prompt-injection attack hidden in a supplier invoice, then write the instruction you would add to an agent's prompt to defend against it.
3. Draft a one-page responsible-AI checklist for a Rawalpindi school introducing an AI tutoring app.