Proxy Discrimination: Algorithmic Redlining Lab |…

What Proxy Discrimination Is

Proxy discrimination occurs when an algorithm uses a seemingly neutral feature—such as zip code or credit score—as a proxy for a protected attribute, like race or gender. This can lead to discriminatory outcomes even if the model developers do not intend to discriminate.

The 'neutral' proxy feature in this lab is designed to mimic real-world scenarios where historical biases are encoded into seemingly objective data points.

How Proxy Discrimination Works

By adjusting the correlation between the proxy and the protected group, you can see how strongly the model 'learns' this bias. Even if the proxy itself is not directly correlated with the outcome, its strong association with a protected attribute can lead to discriminatory outcomes.

The four-fifths rule (also known as disparate impact) measures whether the probability of approval for one group is less than 80% of the probability for another group. This lab demonstrates how even small correlations can drive this ratio below the legal threshold.

live demo · related simulation● LIVE

Why It Matters

Understanding proxy discrimination is crucial in ensuring fairness and equity in machine learning models, especially in critical domains like lending, hiring, and criminal justice. Removing the protected attribute alone does not fix the issue because the model has learned to use the proxy as a substitute for the protected group.

This lab highlights the importance of transparency, ethical considerations, and rigorous testing to prevent unintended biases.

Real-World Examples

In housing applications, zip codes can be used as proxies for race or ethnicity. For instance, certain neighborhoods may have higher concentrations of minority populations, leading a model trained on historical data to unfairly deny loans based on the neighborhood rather than individual merit.

Credit scores are another example where a high score might correlate with being white and having more financial resources, potentially leading to discriminatory lending practices.

Frequently asked questions

How can proxy discrimination be prevented in machine learning models?

To prevent proxy discrimination, it is essential to identify and remove biased features, ensure diverse training data, and use fairness-aware algorithms that explicitly account for potential biases.

Why does deleting the protected attribute alone not fix the issue?

Deleting the protected attribute only removes explicit bias but does not address the model's learned associations. The model may still use other features as proxies, perpetuating discrimination.

Can proxy discrimination be detected during model training?

Yes, various fairness metrics and techniques can detect proxy discrimination during model training by analyzing feature correlations and impact on different demographic groups.

What are some real-world consequences of proxy discrimination in machine learning models?

Proxy discrimination can lead to systemic inequality, unfair treatment of certain groups, and legal and ethical violations. It can also damage a company's reputation and result in financial losses due to lawsuits.

Try it live

Everything above runs in your browser — open Proxy Discrimination: Algorithmic Redlining Lab and change the parameters while it is running. Nothing is installed, nothing is uploaded, the whole model lives in one tab.

▶ Open Proxy Discrimination: Algorithmic Redlining Lab simulation