Systemic erosion of digital trust: How surveillance capitalism and AI extractivism undermine privacy as a human right

Indigenous and traditional knowledge systems frame data as sacred and collective, not as a private asset to be commodified. The Māori principle of 'kaitiakitanga' (guardianship) directly conflicts with the extractive logic of AI, where data is mined without reciprocity. Western privacy frameworks, including those embedded in UX design, often disregard these epistemologies, treating indigenous communities as data sources rather than rights holders. The erasure of these perspectives reinforces colonial power structures in digital spaces.

The current discourse on 'privacy-led UX' ignores the historical continuity between 20th-century surveillance (e.g., census data used for oppression) and today’s AI-driven surveillance capitalism. Zuboff’s analysis of surveillance capitalism traces its roots to behavioral modification experiments in the 1950s, while the Cambridge Analytica scandal revealed how data extraction has been weaponized for political control. The framing also overlooks how 'trust' in tech has been systematically eroded by repeated breaches, from Facebook-Cambridge Analytica to the NSA’s PRISM program.

Cross-culturally, privacy is not universally defined as an individual right; in many societies, it is a communal or relational concept. For example, in Japan, privacy is often framed through the lens of 'wa' (harmony), where collective well-being takes precedence over individual autonomy. In contrast, Western UX design prioritizes individual consent mechanisms, which may alienate users from cultures where privacy is negotiated through social norms rather than legal contracts. These differences highlight the need for pluralistic approaches to digital rights.

Scientific literature on human-computer interaction (HCI) shows that 'privacy-led UX' often fails because users cannot meaningfully consent to data collection they don’t understand, as demonstrated by studies on dark patterns and consent fatigue. Research on algorithmic bias reveals that even 'privacy-preserving' AI systems can perpetuate harm by obscuring discriminatory training data. Meanwhile, differential privacy and federated learning offer technical pathways to reduce surveillance, but their adoption is hindered by corporate resistance to relinquishing control over data.

Artists like Trevor Paglen and Hito Steyerl critique how AI systems aestheticize surveillance, turning privacy into a commodity fetishized in UX design. Spiritual traditions, such as Buddhist mindfulness, frame data collection as a form of 'mental pollution,' where constant tracking disrupts inner peace. Indigenous artists like Jolene Rickard (Tuscarora) use digital media to reclaim data sovereignty, blending traditional storytelling with contemporary tools to resist extractive narratives. These perspectives challenge the reductive framing of privacy as a 'UX problem.'

Future scenarios suggest that without structural change, 'privacy-led UX' will remain a band-aid solution as AI systems grow more invasive, potentially leading to a dystopian 'surveillance-as-a-service' model. Scenario planning by the Open Future Foundation warns that unchecked AI could normalize real-time emotional surveillance, where users are monetized as 'data subjects' in perpetual behavioral experiments. Conversely, models like 'data cooperatives' or 'public-interest AI' offer pathways to decentralize power, but require radical shifts in governance and ownership.

Marginalized communities, including Black, Indigenous, and low-income users, face disproportionate harms from AI surveillance, from predictive policing to discriminatory loan algorithms. The framing of 'trust' as a UX issue ignores how these groups are often excluded from design processes, leading to systems that fail them by design. Activists like Malkia Devich-Cyril (MediaJustice) argue that 'privacy-led UX' is a neoliberal solution that individualizes systemic oppression, obscuring the need for collective rights frameworks like the 'Algorithmic Justice League’s' campaigns.