When people with disabilities interact with technologies, there is a risk of facing discriminatory impacts in several high-stakes contexts such as employment, benefits, and healthcare. Automated employment decision tools, for instance, can unfairly screen disabled applicants by flagging unusual eye movements of blind or low-vision individuals, thereby removing them from the applicant pool.
Algorithms integrated into benefits determination systems have deprived people with disabilities of their rightful benefits. This includes determining the number of hours of home-based care a disabled person can receive through Medicaid, impacting their ability to live independently. In healthcare decision-making systems, algorithms play roles in decisions like hospital discharge and post-surgical opioid prescriptions. Biased outcomes from these systems can reduce health outcomes for people with disabilities and amplify discrimination against multiply-marginalized disabled individuals.
Disability rights activists have long fought against discrimination affecting disabled people. As technology becomes more embedded in daily life, the frequency and severity of disparate effects on disabled individuals are likely to increase.
Biases often stem from problems with the data on which models are trained. Better data is essential for better results. Data collected about people with disabilities also informs advocacy efforts and supports disability-inclusive policies and civil rights laws. Addressing technology-facilitated disability discrimination requires understanding and mitigating issues endemic to disability-related data.
The report identifies ways data sets may exclude or inaccurately count disabled people and offers recommendations for improving data collection:
1. Disability data should be collected wherever other demographic data is gathered.
2. Data should be collected and stored respectfully regarding personal privacy.
3. New methods for defining disability and collecting related data must be developed.
4. Practitioners should adopt a growth mindset around disability data.
5. People with disabilities should be involved in creating, deploying, procuring, and auditing all technologies.
6. Disabled leaders should be central to creating and implementing technology policies.
7. Data collection methods must be accessible to individuals with disabilities.
Significant changes in data collection are necessary but possible to inclusively design algorithmic systems.
Read the full report.
---
