AI and Society: Ethical Considerations

Duration: 45 min

AI and Society: Ethical Considerations

Duration: 45 min

Overview

As artificial intelligence becomes increasingly integrated into critical domains—healthcare, criminal justice, hiring, lending, and content moderation—the ethical implications of AI systems demand rigorous examination. This module explores the multifaceted ethical challenges posed by AI, from algorithmic bias and privacy concerns to regulatory frameworks and responsible AI development practices. Understanding these considerations is essential for building trustworthy AI systems that serve society equitably.

Algorithmic Bias: Sources and Consequences

Algorithmic bias occurs when AI systems produce systematically prejudicial outcomes for particular groups. Unlike human bias, algorithmic bias can scale to affect millions of individuals and often operates invisibly within automated decision systems.

Sources of Bias

Bias enters AI systems through multiple pathways:

1. Data Bias: Training datasets often reflect historical injustices and societal prejudices. A hiring algorithm trained on historical employment data will perpetuate discrimination if women were historically hired less frequently.

2. Representation Bias: When certain groups are underrepresented in training data, models perform worse for those groups. Facial recognition systems trained primarily on light-skinned faces show error rates up to 34% higher for dark-skinned faces.

3. Measurement Bias: Choosing inappropriate metrics or proxies can introduce bias. Using arrest records as a proxy for criminality introduces bias because arrest patterns reflect policing practices, not actual crime rates.

4. Aggregation Bias: Treating diverse populations as homogeneous ignores group-specific patterns. One-size-fits-all models may perform poorly for minority populations.

5. Evaluation Bias: Testing primarily on majority groups masks performance disparities. Equal overall accuracy can hide severe disparities across subgroups.

Real-World Consequences

The stakes are high. COMPAS, a recidivism prediction algorithm used in criminal sentencing, was found to have significantly higher false positive rates for Black defendants. In healthcare, an algorithm used to allocate resources was biased against Black patients due to using healthcare costs (which don't reflect clinical need) as a proxy for health status.

Here's how to identify and measure bias:

import numpy as np
from sklearn.metrics import confusion_matrix
Simulate predictions for two groups with different base rates

np.random.seed(42)
y_true_group_a = np.random.binomial(1, 0.3, 1000)
y_pred_group_a = y_true_group_a.copy()
y_pred_group_a[np.random.choice(1000, 50)] = 1 - y_pred_group_a[np.random.choice(1000, 50)]
y_true_group_b = np.random.binomial(1, 0.5, 1000)
y_pred_group_b = y_true_group_b.copy()
y_pred_group_b[np.random.choice(1000, 150)] = 1 - y_pred_group_b[np.random.choice(1000, 150)]
Calculate false positive rates (bias metric)

tn_a, fp_a, fn_a, tp_a = confusion_matrix(y_true_group_a, y_pred_group_a).ravel()
tn_b, fp_b, fn_b, tp_b = confusion_matrix(y_true_group_b, y_pred_group_b).ravel()
fpr_a = fp_a / (fp_a + tn_a)
fpr_b = fp_b / (fp_b + tn_b)
print(f"False Positive Rate - Group A: {fpr_a:.3f}")
print(f"False Positive Rate - Group B: {fpr_b:.3f}")
print(f"Bias Ratio (B/A): {fpr_b/fpr_a:.2f}x")

Transparency and Explainability: SHAP and LIME

Modern AI systems, particularly deep neural networks, operate as "black boxes"—their decisions are difficult for humans to understand. Explainability techniques make model predictions interpretable, enabling stakeholders to understand why a system made a specific decision.

SHAP (SHapley Additive exPlanations)

SHAP uses game theory to assign each feature a value representing its contribution to the prediction. It provides theoretically sound feature importance that satisfies desirable properties: local accuracy, missingness, and consistency.

import shap
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import make_classification
Train a simple model

X, y = make_classification(n_samples=200, n_features=10, random_state=42)
model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X, y)
Create SHAP explainer

explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X[:10])
For a specific prediction

sample_idx = 0
pred = model.predict([X[sample_idx]])[0]
print(f"Prediction: {pred}")
print(f"Top contributing features (positive contribution):")
feature_importance = sorted(
    zip(range(X.shape[1]), shap_values[pred][sample_idx]),
    key=lambda x: abs(x[1]),
    reverse=True
)
for feat_idx, importance in feature_importance[:3]:
    print(f"  Feature {feat_idx}: {importance:.4f}")

LIME (Local Interpretable Model-agnostic Explanations)

LIME explains individual predictions by fitting a simple interpretable model (like linear regression) to a locally perturbed region around the instance. It works with any model type and provides human-understandable explanations.

from lime.tabular import LimeTabularExplainer
from sklearn.ensemble import GradientBoostingClassifier
Train model

X, y = make_classification(n_samples=300, n_features=12, random_state=42)
model = GradientBoostingClassifier(random_state=42)
model.fit(X, y)
Initialize LIME explainer

feature_names = [f'Feature_{i}' for i in range(X.shape[1])]
explainer = LimeTabularExplainer(X, feature_names=feature_names, class_names=['Negative', 'Positive'])
Explain a specific prediction

instance_idx = 5
explanation = explainer.explain_instance(X[instance_idx], model.predict_proba, num_features=5)
print(f"Prediction: {model.predict([X[instance_idx]])[0]}")
print("Local explanation:")
for feature, weight in explanation.as_list():
    print(f"  {feature}: {weight:.3f}")

Privacy and Differential Privacy

Privacy in AI refers to protecting individuals' sensitive information from being exposed through model outputs or data leaks. Differential privacy provides a mathematical framework for quantifying and guaranteeing privacy protections.

Differential Privacy Concept

Differential privacy ensures that the output of a data analysis algorithm is nearly identical whether or not any single individual's data is included. It adds carefully calibrated noise to mask individual contributions.

The formal guarantee: An algorithm satisfies ε-differential privacy if for any two adjacent datasets (differing by one record), the probability of the algorithm returning any particular output differs by at most a factor of e^ε.

import numpy as np
def differentially_private_mean(data, epsilon, delta=1e-5):
    """
    Compute mean of data with differential privacy guarantee.
    epsilon: privacy budget (smaller = more privacy)
    delta: probability of privacy breach
    """
    true_mean = np.mean(data)
    
    # Sensitivity: max change in output if one data point changes
    # For mean of bounded values in [0,1], sensitivity is 1/n
    sensitivity = 1 / len(data)
    
    # Laplace mechanism: add noise proportional to sensitivity/epsilon
    noise_scale = sensitivity / epsilon
    noise = np.random.laplace(0, noise_scale)
    
    private_mean = true_mean + noise
    return np.clip(private_mean, 0, 1), noise
Example

data = np.random.uniform(0, 1, 1000)
epsilon = 0.5  # Strong privacy guarantee
private_mean, noise = differentially_private_mean(data, epsilon)
print(f"True mean: {np.mean(data):.4f}")
print(f"Private mean: {private_mean:.4f}")
print(f"Noise added: {noise:.4f}")

Regulation: EU AI Act and Governance

Regulatory frameworks are emerging to govern AI development and deployment. The EU AI Act represents the most comprehensive regulatory approach to date.

EU AI Act Overview

The EU AI Act classifies AI systems into risk categories:

  • Prohibited Risk: AI that causes unacceptable harm (e.g., real-time facial recognition for mass surveillance)
  • High Risk: Systems affecting fundamental rights (e.g., hiring systems, credit decisions, criminal sentencing). Require: impact assessments, data governance, human oversight, documentation, quality assurance
  • Limited Risk: Systems with transparency obligations (e.g., chatbots must disclose AI nature)
  • Minimal Risk: No specific requirements

High-risk systems must undergo conformity assessment before deployment. Organizations must maintain documentation of all model decisions affecting individuals' rights.

Responsible AI Frameworks

Organizations implementing responsible AI adopt structured frameworks:

1. Fairness: Ensure equitable treatment across demographic groups. Use disaggregated metrics, conduct bias audits, and test edge cases.

2. Accountability: Maintain audit trails, document decisions, and assign responsibility. Enable individuals to contest automated decisions.

3. Transparency: Disclose when AI is used and explain its role. Provide understandable information about data use and model limitations.

4. Privacy: Implement data minimization, access controls, and retention policies. Use techniques like differential privacy and federated learning.

5. Robustness: Test for adversarial examples, distribution shift, and edge cases. Establish monitoring systems for ongoing performance.

Here's a framework checklist:

class ResponsibleAIFramework:
    """Checklist for responsible AI system development"""
    
    def __init__(self, model_name):
        self.model_name = model_name
        self.checklist = {
            'bias_audit': False,
            'explainability_tested': False,
            'privacy_mechanisms': False,
            'documentation_complete': False,
            'human_oversight': False,
            'monitoring_active': False,
            'regulatory_review': False,
            'stakeholder_consultation': False
        }
    
    def add_bias_audit(self, demographic_groups, metrics):
        """Log bias audit results"""
        print(f"Bias audit for {self.model_name}:")
        for group, metric_values in zip(demographic_groups, metrics):
            print(f"  {group}: {metric_values}")
        self.checklist['bias_audit'] = True
    
    def get_compliance_status(self):
        """Check framework compliance"""
        completed = sum(1 for v in self.checklist.values() if v)
        total = len(self.checklist)
        return {
            'progress': f"{completed}/{total}",
            'ready_for_deployment': completed == total,
            'gaps': [k for k, v in self.checklist.items() if not v]
        }
Usage

framework = ResponsibleAIFramework('credit_scoring_model')
framework.add_bias_audit(['Group_A', 'Group_B', 'Group_C'], [0.92, 0.88, 0.95])
framework.checklist['explainability_tested'] = True
framework.checklist['privacy_mechanisms'] = True
status = framework.get_compliance_status()
print(f"\nCompliance Status: {status['progress']}")
print(f"Ready for deployment: {status['ready_for_deployment']}")
print(f"Gaps to address: {status['gaps']}")

Mitigation Strategies for Bias

Identifying bias is the first step; mitigating it requires systematic approaches throughout the AI pipeline:

Data-Level Mitigation

1. Data Augmentation: Deliberately oversample underrepresented groups to balance representation 2. Synthetic Data Generation: Create synthetic samples for underrepresented groups when real data is limited 3. Bias-Aware Sampling: Ensure training data proportionally represents demographic groups 4. Data Documentation: Maintain datasheets describing dataset composition, limitations, and recommended uses

Model-Level Mitigation

1. Fairness Constraints: Include fairness objectives during training, not just accuracy 2. Debiasing Algorithms: Techniques like adversarial debiasing remove bias while preserving utility 3. Regularization: Add penalties for models exhibiting disparate impact across groups 4. Threshold Optimization: Adjust decision thresholds differently for different groups to achieve fairness targets

Here's an example of threshold optimization for fairness:

from sklearn.metrics import roc_curve, confusion_matrix
import numpy as np
def optimize_thresholds_for_fairness(y_true, y_pred_proba, groups, target_fpr_ratio=1.0):
    """
    Optimize decision thresholds per group to achieve fairness.
    target_fpr_ratio: ratio of false positive rates (should be near 1.0 for fairness)
    """
    unique_groups = np.unique(groups)
    optimal_thresholds = {}
    metrics = {}
    
    for group in unique_groups:
        mask = groups == group
        y_group = y_true[mask]
        y_prob_group = y_pred_proba[mask]
        
        # Find threshold that achieves best balance
        best_threshold = 0.5
        best_diff = float('inf')
        
        for threshold in np.linspace(0, 1, 21):
            y_pred = (y_prob_group >= threshold).astype(int)
            cm = confusion_matrix(y_group, y_pred)
            tn, fp, fn, tp = cm.ravel()
            fpr = fp / (fp + tn) if (fp + tn) > 0 else 0
            fnr = fn / (fn + tp) if (fn + tp) > 0 else 0
            
            # Prefer balance between false positive and false negative rates
            current_diff = abs(fpr - fnr)
            if current_diff < best_diff:
                best_diff = current_diff
                best_threshold = threshold
        
        optimal_thresholds[group] = best_threshold
        metrics[group] = {
            'threshold': best_threshold,
            'balance_metric': best_diff
        }
    
    return optimal_thresholds, metrics
Demonstration

np.random.seed(42)
groups = np.repeat(['GroupA', 'GroupB'], 500)
y_true = np.random.binomial(1, 0.3, 1000)

GroupB has systematically different model performance

y_pred_proba = np.where(
    groups == 'GroupA',
    np.random.beta(2, 2, 500),
    np.random.beta(1.5, 2.5, 500)
)
thresholds, metrics = optimize_thresholds_for_fairness(y_true, y_pred_proba, groups)
print("Fairness-Optimized Thresholds:")
for group, threshold in thresholds.items():
    print(f"  {group}: {threshold:.3f}")

Evaluation-Level Mitigation

1. Disaggregated Metrics: Report performance separately for each demographic group 2. Fairness Metrics: Use metrics like demographic parity, equalized odds, or individual fairness 3. Intersectional Analysis: Test fairness across combinations of protected attributes 4. Continuous Monitoring: Track fairness metrics in production systems over time

Case Studies: Learning from Real-World Failures

Case Study 1: Amazon Hiring Algorithm

Amazon developed an ML recruiting tool trained on historical hiring data. The system showed bias against women because women were historically underrepresented in technical roles. The algorithm learned to penalize resumes mentioning "women" or words associated with women's colleges. Lesson: Historical data encodes historical discrimination; be explicit about protected attributes you want to prevent from influencing decisions.

Case Study 2: Facial Recognition in Law Enforcement

A facial recognition system used by law enforcement had significantly higher error rates for Black suspects, leading to false arrests. The system was trained primarily on light-skinned faces. Lesson: Ensure training data represents the full diversity of the population the system will serve. Real-world consequences demand higher accuracy standards.

Case Study 3: Healthcare Resource Allocation

A hospital system used historical healthcare spending to allocate resources for patient needs. The algorithm discovered that spending predicts outcomes but didn't consider that spending reflects healthcare access disparities, not actual patient health. Black patients, who faced systemic healthcare discrimination, had lower costs and were thus allocated fewer resources. Lesson: Question whether observed correlations reflect true causality or artifact of measurement practices.

Key Takeaways

1. Algorithmic bias stems from data, representation, measurement, and evaluation choices—each requires systematic mitigation strategies 2. Explainability techniques (SHAP, LIME) make model predictions interpretable for stakeholders and regulators 3. Differential privacy provides mathematical guarantees for individual privacy protection 4. Regulatory frameworks like the EU AI Act establish minimum standards for high-risk AI systems 5. Responsible AI frameworks integrate fairness, accountability, transparency, privacy, and robustness throughout the development lifecycle 6. Real-world AI failures provide crucial lessons about the importance of proactive bias detection and mitigation 7. Fairness requires deliberate choices about which fairness metrics matter most for your specific application

Quiz

Q1: Which of the following is NOT a primary source of algorithmic bias?

  • A) Data containing historical inequities
  • B) Underrepresentation of certain groups in training data
  • C) Using neural networks instead of decision trees
  • D) Choosing inappropriate metrics as proxies for outcomes

Answer: C ✓ — Neural network architecture doesn't inherently introduce bias; bias comes from data and measurement choices.

Q2: What does SHAP stand for, and what game-theoretic concept does it use?

  • A) SHapley Additive Predictions; uses clustering
  • B) SHapley Additive exPlanations; uses Shapley values ✓
  • C) System Hierarchical Analysis Package; uses entropy
  • D) Sensitivity Histogram Analysis Protocol; uses distribution analysis

Answer: B ✓

Q3: In differential privacy, what does a smaller epsilon (ε) value indicate?

  • A) Weaker privacy protection and less noise
  • B) Stronger privacy protection and more noise ✓
  • C) Faster computation time
  • D) Higher model accuracy

Answer: B ✓

Q4: Which AI systems are classified as "HIGH RISK" under the EU AI Act?

  • A) All machine learning models
  • B) Only deep learning neural networks
  • C) Systems affecting fundamental rights like hiring, credit decisions, and criminal sentencing ✓
  • D) Only systems used by government agencies

Answer: C ✓

Q5: What is the primary goal of a Responsible AI framework?

  • A) Minimize computational costs
  • B) Maximize model accuracy only
  • C) Integrate fairness, accountability, transparency, privacy, and robustness throughout development ✓
  • D) Reduce regulatory compliance requirements

Answer: C ✓

Q6: Why might COMPAS, a recidivism prediction algorithm, show bias against Black defendants?

  • A) It was intentionally programmed with bias
  • B) Its training data reflected historical policing patterns and arrest rates rather than actual crime rates ✓
  • C) Deep neural networks cannot be fair
  • D) It only used demographic information as features

Answer: B ✓

← Back ▶ Run in Colab