Including notes for the second session and the first panel.
- Tech folks have a tendency to "fly in and fix everything." That feels like a dangerous approach here. It's far better to stand on the shoulders of existing legal precedent, which has studied fairness, discrimination, and bias for decades, even if that slows down progress.
- Machine learning systems mirror and amplify bias by default. We cannot simply ignore sensitive attributes because the system averages loss over the majority. (Disparate mistreatment). Pithy corollary: this problem will only go away if we devote resources into making it go away.
- Providing explanations for decisions is the only humane way to build automatic classification systems. Why? If I can't test a result, I can't contest it. If the decisions must be testable and explainable, they will be much more reliable as a result.
Why might ML be unfair?
- Data might be encoding bias. Labels might not be "committed a crime?" but instead be "was arrested?"
- Data collection feedback loops E.g. only observe "paid back a loan?" if the loan was granted, which is an output of the system.
- Different populations might have different statistics. e.g. SAT scores might correlate with labels differently in populations that employ SAT tutors
- Less data (by definition) about minority definitions
Example parable: The best linear classifier (when I'm not allowed to take race into account) could misclassify every part of the minority population even though they could be easier to classify. Argument: To be "fair," the algorithm may need to explicitly take the minority status into account ... ... Contextual bandit problem: e.g. learn from feedback by giving loans according to different variables, repeat until you learn something Fairness is an obstacle to learning the optimal policy. If I only interview the members of the majority population, I can't tell what variables matter. I can't tell who is a high-quality or low-quality applicant. Summary: Even without a taste for discrimination "baked in", natural (optimal) learning procedures can disproportionately benefit certain populations.
- Imposing "fairness" constraints is not without cost. Most quantity tradeoffs between fairness and other desiderata
- No Pareto-dominant solution - how should we choose between different points on the tradeoff curve?
- None of these points is unique to "algorithmic" decision making. This just allows formal analysis to bring these universal issues to the fore.
Focus is on *discrimination, a specific type of unfairness.
- Measures of discrimination;
- Mechanisms to mitigate discrimination.
What is discrimination? Well-studied in social sciences: political science, moral philosophy, economics, law... But less-studied from a computational perspective. A first approximate "normative / moralized" definition:
- Discrimination is when you wrongfully impose a relative disadvantage on persons based on their membership in some socially-salient group, such as race or gender.
- What's a socially-salient group? Question for political and social scientists.
- Relative disadvantage? Question for economists and lowyers.
- What's a wrongful decision? Question for moral philosophers.
- What constitutes "based on"?
^That. That "based on" thing is a question for us computer scientists. Consider classifier with sensitive attributes and non-sensitive (unrestricted) attributes. Discrimination: Decisions should not be based on sensitive attributes?
- Naive approach: Ignore sensitive attributes (fairness through blindness / ignorance)? If you did that, you avoid disparate treatment: Get two users who are identical (but for sensitive attributes), you get the same result.
- Is this OK? Or are there problems with this?
- Two problems with the intuitive notion: Unless users of different sensitive attribute groups have similar non-sensitive feature distributions, we risk:
- Disparate mistreatment: When global risk / loss minimization for all users during learning results in different levels of risk for different sensitive attribute groups.
- Disparate impact: When labels in training data are biased due to past discrimination.
Background Learning 101:
- To learn, we define and optimize a risk/loss function over all examples in training data (e.g. hinge loss for SVM, l(wx+b) Risk function captures inaccuracy in prediction So learning is cast as an optimization problem
- For efficient learning, we typically define convex losses
Origins of disparate mistreatment:
- Minimizing Loss(all) does not guarantee Loss(black) and Loss(white) are equally minimized. Black users for example might have a different risk than white users.
- Stripping relative attributes does not help!
- To avoid disp. mistreatment, we need L(black) = L(white)
Origins of disparate impact:
- Suppose training data has biased labels.
- In this case, classifiers will learn to make biased decisions, even if you remove the stripping attributes.
- Why? NSAs will be correlated with SAs, will be given more or less weights.
- Learning tries to compensate for lost SAs.
Analogous to indirect discrimination
- Observed in human decision-making
- E.g. Vote-ID laws being passed in US states. (<-- I'm proud the speaker chooses to use this as an explicit example). Rather than directly pass a law that includes this group, get at that protected group by abusing correlations between other features.
- Notoriously hard to detect this!
- Doctirine of "disparate impact": US law applied to housing in 70s,80s
- Proportionality test over decision outcomes: see 70%/80% rule: if X% of group 1 applicants get selected, you must select Y% of group 2 applicants as well.
- Critics: There are scenarios where disproportionate outcomes are justifiable. Supporters: Provision for business necessity exists.
- Law is necessary to detect indirect discrimination.
- Back to disparate impact: If we require proportional outcomes in our classifier, we have a fundamentally difficult situation. We need
$$P(y=1|SA=1) = P(y=1|SA=0)$$ for sensitive attribute SA.
Summarize:
- Disparate treatment: Direct discrimination
- Disparate impact: Indirect discrimination when training data is biased
- Disparate (...)
Learning to avoid discrimination:
- Discrimaniton notions as a constraint on the learning process.
- Optimize for accuracy under those constraints. Problem: Additional constraints lower accuracy. Might not need all the constraints at the same time (e.g. drop dis. impact constraint when no bias in data) When avoiding disp. impact / mistreatment, you might be able to achieve higher accuracy without disp. treatment. Ie. y using sensitive attributes.
Problem: learning
About how to measure and discover instances of discrimination. (Previous talk made the point that this is hard to discover, so this talk is important.) "We should not start tackling this problem by defining our own definition of fairness. We should start by adopting the definition used within the legal community." Conclusion: Bad news: The algorithmic and big data are not just mirroring the existing bias but they are reinforcing that bias and amplifying inequality. Good news: Algorithmic discrimination: Despite its challenges, it brings a lot of opportunities for machine learning researchers to build tools for addressing different aspects of the problem.
- Growing research trend, it would be nice to see more research groups unifying for a shared agenda.
- Notions of fairness in computer science are often related to but often different than fairness in law.
- "If the issue is complex and inscruitable, a solution that is equally inscruitable is unlikely to be a good solution to the problem."
- A very serious science. Hard work; young science. Real life: everyone is trying to get their hands on Big Data. We have no idea of the tradeoffs here; it can be highly nonsensical and dangerous.
- Conceptual fidelity. Also: taking legal statements in a way that's not contextually incorrect.... Useful to do some mapping, "what are the definitions of fairness that were used in this community?" from game theory, economics; even just to map them in that meta-analysis would be helpful.
Two concepts, that have different perspectives between computer science and legal fields:
- What is fairness?
- What is a sufficient explanation?
Re sufficient explanation: A technique commonly used in private law: When common law courts ascertain what the intentions of what the parties are, there's the notion of an objective standard. We don't care whether the individual is subjectively satisfied and gets it. The typical language is "reasonableness," but those reasonableness standards are determined within a context of (another objective/moral observer?). Regulation: If there is an automated decision that has a real significance for your life, then:
- you must know that this is happening (currently egrigiously violated)
- you must gain meaningful information about the logic processes used;
- you must be told about the individual consequences of this
If the meaningful explanation is not there, IMO you should not be allowed to make that decision autonomously. E.g. "You have to pay 3x more on your health insurance / You are not given the job, you are rejected automatically"
If I can't test it, I can't contest it.
Interesting point: This will make the decisions better. If the decisions must be testable to be contestable, they will be much more reliable as a result.
Lots of EU law examples here; companies will be hit with 4% revenue fine if they cannot provide meaningful information for their decisions. If you are profiting from autonomous decisions, and if that decision affects people in material ways and if you can't even explain the basis for these decisions... Not a society we want to do.
Unless we have notions for what is a correct explanation and a wrong explanation, we might have a mess of explanations. Worry: Different groups of people have different information; everyone will extrapolate, reverse engineer, come up with completely different explanations and believe the algorithm is doing that for completely different reasons. ... We should have motions for correct and wrong explanations
Rise of populism: one reason could be growing distrust of scientific processes and distrust of explanations.