The human firewall: modeling AI-driven influence and human vulnerability in hybrid ecosystems - AI & SOCIETY

Artificial intelligence (AI) has rapidly transitioned from a niche technological capability to a critical socio-technical infrastructure. While delivering transformative efficiencies across domains including financial services and healthcare, AI simultaneously generates novel risks that transcend traditional cybersecurity paradigms. Unlike earlier digital threats, AI-driven risks are systemic, blending technical, organizational, and cognitive dimensions. This article advances three arguments. First, AI penetrates the cognitive domain, enabling manipulation that extends beyond discrete attacks to relationship-based grooming, agentic misalignment, and large-scale cognitive hacking (Carli et al. in Risk and exposure of XAI in persuasion and argumentation: the case of manipulation. In: International Workshop on Explainable, Transparent Autonomous Agents and Multi-Agent Systems. Cham: Springer International Publishing, pp.204–220. https://doi.org/10.1007/978-3-031-15565-9_13 , 2022; Baker in What is cognitive hacking? The cyber attack that targets your mind. EM360Tech. Available at: https://www.em360tech.com/tech-articles/what-cognitive-hacking-cyber-attack-targets-your-mind . Accessed 14 Sep 2025, 2025). Second, humans must be reconceptualised as adaptive defenders within hybrid socio-technical ecosystems rather than treated solely as points of failure. Third, effective defense requires integrating agent-based modeling (ABM), user and entity behavior analytics (UEBA), and socio-technical red-teaming into an adaptive governance ecosystem capable of anticipating, rather than merely reacting to, adversarial tactics. The article’s central contribution is to articulate the human firewall as a systemic socio-technical framework that embeds resilience at cognitive, organisational, and cultural levels. An exploratory computational model is introduced to illustrate, conceptually rather than operationally, how behavioral drift might be detected in human–AI interactions. This model is deliberately illustrative and hypothesis-generating; it does not constitute a validated detection system, and its parameters are not empirically calibrated. By reframing humans as critical nodes of resilience, the human firewall paradigm enables organisations to operationalize critical thinking, ethical judgment, and situational awareness as frontline safeguards. Resilient autonomy, achieved through digital literacy, interdisciplinary enrichment, and adaptive governance, is presented as essential to safeguarding democratic resilience, organisational trust, and systemic stability in an AI-permeated world.