tech
Algorithmic Agency Asymmetry: When AI Decides For You, You Don't Even Have the Right to Object - 36Kr
Algorithms are not just tools, but invisible power. They shape your choices, yet you know nothing about them. Who will break this asymmetrical game?
TL;DR
- Algorithmic asymmetry exists when one party can observe and refine algorithms, while the other is largely subject to their outcomes.
- This imbalance permeates areas like hiring, loans, insurance, education, and policing.
- Three layers of 'cognitive shackles' explain this asymmetry: opacity, historical bias amplification, and recursive systems.
- Opacity refers to the lack of understanding of an algorithm's goals, thresholds, incentives, and weaknesses.
- Historical bias amplification means algorithms learn from and perpetuate past societal prejudices.
- Recursive systems involve users continually training algorithms, which then shape user behavior in a continuous feedback loop ('algorithmic drift').
- Asymmetric agency arises when organizations test and optimize influence at scale, while individuals only see surface-level information.
- People adapt to what systems reward, leading to unintended consequences in fields like hiring and education.
- AI detection tools can unfairly flag non-native speakers, and social media experiments show how content exposure alters emotional language.
- Digital systems not only classify behavior but also teach users what words to use, risks to avoid, emotions to express, and metrics to pursue.
- Policy must rebalance this relationship through meaningful notice and explanation, enforceable impact assessments, genuine human oversight, post-deployment monitoring, and prohibiting certain practices.
- Algorithmic literacy should be considered civic infrastructure, empowering citizens to understand and question algorithmic influences.
- Closing the gap requires making automated impacts visible, contestable, auditable, and governable.