AI - The Security Professional's next Maginot Line (Part 2)
In the first part of this article, I argued that many organizations are approaching AI much like military planners approached the next war using lessons from the previous one. Or to put differently: organizations approach AI through the conceptual tools they already understand.
This is not unusual. It is, in fact, how most technological transitions begin. New systems are interpreted through old mental models. New risks are evaluated using familiar categories. And new capabilities are placed into existing governance structures as if those structures define the limits of change.
The Internet followed this pattern. It was initially understood as a communication technology. A faster channel for information exchange. A more efficient way of connecting systems, people, and processes. It taught us to protect systems, networks, applications, identities and data, it also framed the security professionals to think in terms of confidentiality, integrity, availability, access control, compliance, and governance.
Those lessons were not wrong. They were necessary. But they may also be incomplete.
My argument is that AI is not primarily changing how information moves. It is changing how decisions are formed, validated, and executed. As a consequence, organizations are gradually externalizing parts of their cognition: expertise, interpretation, judgment, situational awareness, and eventually decision-making itself. This can be summarized as cognitive infrastructure - a term which is of course not new, and being used widely for similar discussions.
The risks discussed in the first part were largely internal... Loss of competence and/or independence, various degrees and types of synthetic realities, and accountability fragmentation. But those risks emerge before we even consider what happens when multiple actors begin operating inside the same cognitive environment. That is where things become more interesting, and potentially far more disruptive...
AI changes how decisions are formed. And once that shift is accepted seriously, the implications stop being operational. They become structural.
Operational risks affect what organizations do, structural risks affect what organizations are.
A data breach may disrupt operations. A failed ERP implementation may reduce efficiency. Even a major ransomware incident usually leaves the underlying organization intact.
Structural change is different. It alters the assumptions upon which decisions, authority, accountability, and competitive advantage are built.
This distinction matters because organizations have decades of experience managing operational risk. They possess frameworks, controls, governance structures, audit models, and recovery procedures. They have considerably less experience recognizing when the underlying assumptions behind those frameworks are beginning to change.
Identity Collapse
Modern organizations are built on an assumption so fundamental that most people rarely notice it: We assume somebody exists!
The same way, we assume that a customer really exists, or a supplier, an employee, or that a threat actor exists...
Not necessarily a trustworthy one, but an identifiable actor nonetheless. There is always someone behind an action, even when systems are automated, the assumption persists that responsibility can ultimately be traced back to an identifiable source
This assumption is deeply embedded. Not just in security models, but in law, governance, compliance, and organizational design itself... Entire organizational structures are built on that assumption. Contracts assume counterparties, access controls assume users, communication systems assume senders and recipients, and finally governance assumes accountable actors.
Even large parts of cybersecurity are ultimately built around answering relatively simple questions:
- Who are you?
- What are you allowed to do?
- What did you do?
For decades these questions have been difficult enough already. Cloud computing provides a useful example: One of the most misunderstood concepts introduced by cloud platforms was the idea of assumed roles, delegated permissions, service identities, and machine-to-machine authorization.
Organizations increasingly struggled to answer questions that previously appeared straightforward:
- Who actually performed this action?
- Which system initiated the request?
- Which identity was acting at the time?
- Who is accountable for the outcome?
Many security teams spent years adapting to these models. Not because authentication failed, but because attribution became harder...
The technology itself was not particularly complicated. The challenge was conceptual, and introduced something very uncomfortable: The break of one-to-one relationship between action and identity. And this challenges our deeply encoded model of how we perceive and interpret the world.
AI expands this problem even further - the future Internet may contain millions or billions of (short-lived) autonomous agents. Some will represent organizations, some will represent individuals, some will act on behalf of other agents, and some may have no direct human involvement at all.
Organizations may increasingly interact with AI-generated workforce, AI-generated demand, AI-generated supply, AI-generated advisory, AI-generated negotiators, and AI-generated threat actors.
The problem is not that these entities are artificial. The problem is that many assumptions embedded into organizational trust models begin to weaken, as a customer service conversation may no longer occur between two people. A procurement negotiation may no longer involve humans. A sales inquiry may originate from an AI purchasing agent. A phishing campaign may be conducted entirely by autonomous systems.
A single action may now be the result of chained roles, temporary credentials, automated processes, and cross-service interactions. Even experienced security teams often struggle not with authentication itself, but with attribution.
Even worse, those interactions may become indistinguishable from ordinary business activity. At that point, authentication becomes less useful than many people assume... Authentication never proved intent, it only proved identity. A valid credential tells us who performed an action: It does not tell us why, it does not reveal objectives or incentives, and it does not reveal whether the actor is operating independently, under influence, or on behalf of someone else.
Of course, that limitation existed long before AI. But, AI simply scales the problem to a level where it challenges risk matrices, or even enterprise risk management programs.
Because knowing who performed an action no longer explains why the action occurred. And organizations are built on the implicit assumption that it does. Once that link breaks, trust models begin to weaken in ways that are not immediately visible, but gradually systemic.
AI Versus AI
The second shift is subtler... Much of today's security discussion focuses on capability and their execution:
- Attackers gain AI - Defenders gain AI.
- Attackers automate reconnaissance - Defenders automate detection.
- Attackers automate exploitation - Defenders automate response.
...the conversation often turns into a comparison of capabilities. Technological shifts rarely matter because they increase capability, as it's not the most important change. The real challenge is temporal asymmetry.
Again: Even highly automated environments still rely on human decision cycles. When a threat is detected, whether using integrated security solutions, or through different means, an analyst investigates the incident, an engineer validates it, then a manager approves response measures...
For trained SecOps teams, we're talking about hours speed. For low-maturity environments about days or weeks! And almost always the bottleneck is human cognition, which is now being changed by AI.
Increasingly, systems will observe, interpret, decide, and act without waiting for human participation. Both attackers and defenders gain this capability. And the result is not necessarily that attackers become unstoppable, but that the conflict accelerates into new dimensions. And this will create entirely new dynamics.
Military strategists have historically described similar shifts as changes in decision-cycle speed. Concepts such as the OODA loop (Observe, Orient, Decide, Act) illustrate how competitive advantage often emerges from compressing the time between observation and action.
Other organizations however are accustomed to operating at human speed: Meetings happen at human speed, governance happens at human speed, escalation processes happen at human speed, as well as executive decision-making happens at human speed. AI however, does not.
To give another example... Most governance structures assume that strategic decisions happen slower than oversight processes: Boards review, committees assess, auditors validate, management approves.
Those assumptions emerged because human cognition was historically the bottleneck. When machine-speed decision systems begin interacting with other machine-speed decision systems, governance may become retrospective rather than supervisory.
The organization discovers what happened after the decision already propagated through the environment.
As decision cycles compress, organizations will realize that information is no longer the limiting factor, human reaction time is. This applies well beyond cybersecurity... Supply chains operate through decisions, financial markets operate through decisions, procurement processes operate through decisions, political institutions operate through decisions.
Competitive advantage increasingly shifts toward actors capable of operating at machine speed while maintaining coherent strategic direction.
The challenge is not merely technical, anymore, it is organizational. Most organizations were never designed for environments where critical interactions unfold faster than human cognition can reliably supervise.
Railways did not just move goods faster. They changed mobilization strategy. They changed economic geography. They changed what it meant for a city or country to be “close” or “far”. Radio did not just improve communication. It compressed decision loops across entire military structures. The same way, Aircraft did not just extend reach. They collapsed time between detection and response.
And in each case, the decisive advantage did not come from having the technology, it came from adapting decision structures to it.
AI introduces a similar but more pervasive change: It compresses the distance between observation, interpretation, decision, and action. And the deeper challenge is not be acceleration itself...
Organizations have adapted to acceleration before: Electronic trading accelerated financial markets, Continuous manufacturing accelerated production, Cloud computing accelerated software deployment. The difference is that those systems largely accelerated execution.
AI accelerates interpretation.
Historically, organizations spent significant time deciding what reality meant... Facts required analysis, Signals required interpretation, Situations required judgment.
AI increasingly participates in those activities directly. And as a result, the compression does not occur only between decision and action. It occurs between observation and understanding. This is a more profound change because organizations are not merely acting faster.
They are forming conclusions faster.
Cognitive Dominance
Which brings us to a more structural development.
The Internet promised decentralization, and whether we like all aspects or not, to some extent it delivered. But it also produced extraordinary concentration.
A small number of platforms became responsible for increasingly large portions of communication, commerce, search, software, advertising, and digital infrastructure.
And my assumption is that AI may accelerate this trend further... The common discussion focuses on compute resources, capital requirements, data access, and market concentration. Those are important concerns.
But they may not be the most important ones.
The deeper issue is cognitive dependence, as already introduced. Today, organizations produce most strategic cognition internally, and most of these expertise lives inside companies, while judgment is distributed across roles and responsibilities. Organizations could purchase software, hardware, and services while retaining the ability to think independently.
A CRM vendor affects how customer data is stored. A cloud vendor affects where workloads run. A payroll provider affects compensation processes. All are important. None fundamentally determine how executives interpret markets, evaluate risk, allocate capital, or formulate strategy.
Cognitive systems increasingly participate in those functions. Therefore the dependency shifts from execution to interpretation.
This is why distinction matters... Software performs functions and cognition shapes decisions. An organization can survive replacing a software vendor - with certain platforms, it may hurt, but it will almost always survive.
Replacing a decision architecture is considerably harder. At some point, the dependency ceases to be technological, it becomes strategic. This is why the sovereignty discussions emerging across Europe are simultaneously understandable and incomplete, as the conversation typically focuses on ownership...
The majority of RFPs, I'm seeing on a daily base, are asking the same basic questions:
- Where are the data centers?
- Which country owns the provider?
- Which jurisdiction applies?
Replacing a vendor or a software system is difficult, but conceptually straightforward. Replacing a cognitive layer embedded in decision-making processes is fundamentally different, because it is no longer a discrete system. It becomes part of how decisions are formed in the first place.
Therefore organizations will struggle to retain the ability to think independently when large portions of their cognitive infrastructure become dependent on external systems. Because dependence on cognition providers is fundamentally different from dependence on software providers. Software influences operations, cognition influences the reality itself.
Cognitive Conflict
All of this leads toward a broader question: What happens when cognition itself becomes contested?
Much of modern security still revolves around protecting assets: Networks, endpoints, applications, identities and eventually data.
Even when discussing information warfare, the underlying assumption is relatively simple: If you control information, you influence behavior.
But this model assumes a relatively stable relationship between information and interpretation. That assumption is becoming less reliable, as synthetic content becomes increasingly indistinguishable from authentic signals, the problem is no longer whether information can be manipulated. It is whether interpretation itself can remain stable.
To understand the shift, it is useful to look at intelligence history. In many strategic contexts, the most effective operations were not those that removed information, but those that altered interpretation.
A document that is stolen changes knowledge.
A narrative that is shaped changes decisions.
The second is often more strategically significant than the first. And AI increases the scale of this effect, because it allows the generation of plausible interpretive environments that are not necessarily anchored in underlying reality...
Organizations may still receive correct data.
Dashboards may still function.
Reports may still be generated.
But the interpretive layer that connects these inputs to decisions can gradually drift. Not through sudden manipulation, but through accumulation of plausible distortions. And as mentioned above that creates a form of risk that is difficult to detect using traditional security thinking.
Because nothing is visibly broken and the system continues to operate, but it may operate on an increasingly misaligned model of reality.
The same way a ransomware attack prevents access to systems, but a successful cognitive attack may leave every system operational. The organization continues functioning, employees continue working, reports continue arriving, dashboards continue updating, and even meetings continue taking place. The difference is that decisions gradually become detached from reality. And unlike a ransomware incident, there may be no obvious moment when the compromise becomes visible.
Imagine strategic assessments influenced by synthetic expertise or market intelligence distorted by AI-generated narratives, or artificial stakeholder pressure created by autonomous systems, synthetic consensus emerging across information ecosystems.
None of these scenarios require malware.
None require unauthorized access.
None require data theft.
The objective is not access.
The objective is influence.
Defense organizations increasingly describe this domain using terms such as cognitive warfare, reflecting a growing recognition that influencing interpretation may become as strategically significant as controlling information itself.
Organizations protect information because information is assumed - depending on their classification of course, to be the strategic asset.
Increasingly, the strategic asset may be the ability to maintain an accurate model of reality despite operating inside environments saturated with synthetic cognition. At least one, which is sufficiently accurate enough. This challenge again is very different from traditional security, and organizations may need new capabilities to validate reality, which is ultimately far more difficult to prove then a malware infection, a data breach, a phishing attack, etc.
Beyond Identity
Talking about risk management, organizational risk models are designed around operational failures, including systems failure, data breaches, processes breaks and human errors.
These are all discrete, observable events with recovery paths.
Even severe incidents, such as ransomware, follow this structure. They disrupt systems, require restoration, and trigger recovery procedures. The organization is challenged, but not fundamentally redefined.
Cognitive externalization does not behave in this way, as it does not fail in discrete events...
It changes underlying capability structures over time. An organization does not wake up one day and discover it has lost internal understanding of its systems, but gradually stops having to maintain that understanding.
And eventually, it no longer exists in a form that can operate without external cognitive support. This is the difference between operational failure and structural transformation.
Entire ecosystems may increasingly consist of human and synthetic cognition interacting continuously. Under those conditions, identity alone becomes an insufficient foundation. Because the critical question is no longer merely who performed an action, but why that action is occurring.
What objective does an entity even pursue or whose interests are being served or what incentives are shaping the outcome?
In short: What is the intent?
Today, we have mature frameworks for authentication, authorization, accounting. Meanwhile, we even have mature frameworks for monitoring activity.
What we do not have are frameworks for validating intent. This challenge intersects with longstanding questions in economics, intelligence analysis, and game theory, all of which attempt to infer objectives and incentives from observable behavior rather than relying solely on stated identity. And if my assumptions throughout this article are correct, then intent validation may become one of the most important strategic challenges of the next decade. Not only for security teams, but for organizations as a whole.
What Should Organizations Do Now?
You may argue, that every major technology introduces risk, and that those have became manageable over the time, without posing existential risks to broad segments. But the problem is not that AI introduces risks...
The problem is that many of the assumptions underlying modern organizations were developed for an environment in which cognition remained largely human, identifiable, and internal, which is beginning to weaken.
Unfortunately, there is no equivalent of a firewall for cognition (yet). There is no cognitive antivirus, no cognitive EDR, and no compliance framework that can fully address the structural shifts discussed throughout this two-part article. The objective therefore isn't about eliminating risk, it's about preserving cognitive resilience while the environment changes.
The following principles may help doing so...
Protect Cognitive Sovereignty
Organizations must maintain the ability to understand how critical decisions are formed, not necessarily to avoid external systems, but to avoid losing interpretive independence. Cognitive sovereignty therefore means retaining sufficient internal expertise and interpretive capability to challenge, validate, and reconstruct decisions without external systems.
Organizations routinely evaluate dependencies on suppliers, cloud providers, energy sources, and critical infrastructure. Increasingly, they should evaluate dependencies on cognition as well.
Do not get me wrong: This does neither imply rejecting AI, nor does it imply maintaining every capability internally. In the contrary, I'm advising everyone to use LLM tools aggressively. The key risk is not usage of AI. It is unexamined dependence on it.
The lesson from decades of outsourcing was never that outsourcing is inherently bad. The lesson was that organizations must retain enough understanding to remain intelligent customers of the capabilities they source externally. And the same principle applies here, actually...
An organization that no longer understands how critical decisions are formed has effectively outsourced part of its ability to think. Of course, that may increase productivity and may be efficient. Therefore, it may even be rational. But it also creates dependency.
Protecting cognitive sovereignty means ensuring that essential expertise, institutional understanding, and decision-making capability remain present inside the organization, even when external systems provide significant support.
Preserve Human Recovery Capacity
Every mature infrastructure strategy assumes failure... In the past, power grids failed, networks failed, applications failed, Cloud providers failed. And they still fail from time to time, yet we still use them, and rely on them, as we learned that when we invest in continuity planning, disaster recovery, redundancy, and fallback procedures, those failures become manageable.
Most AI discussions assume the systems continue operating, but few ask what happens when they do not. Or what happens when they become unavailable, degraded, economically inaccessible, or strategically unsuitable...
Organizations should therefore think about cognitive recovery in the same way they think about technical recovery:
- Could critical functions continue operating if major AI systems disappeared tomorrow?
- Could key decisions still be made?
- Could expertise be reconstructed?
- Could operations continue long enough to recover?
In cybersecurity, resilience often matters more than prevention, and the same principle will apply to cognition. Organizations should remain capable of operating, at least in reduced form, without synthetic cognitive support. Not because this is efficient, but because it preserves continuity of understanding under degraded conditions. A system that cannot function without AI support is not resilient, it is dependent.
Track Decision Provenance
For years, organizations focused on data provenance, including data origin, their owners, editors, etc. And sometimes, whether it can be trusted. Those questions remain important. But synthetic cognition introduces new requirements:
- How was a conclusion reached?
- Which systems contributed?
- Which assumptions influenced the result?
- Which interpretations shaped the recommendation?
As AI becomes embedded into workflows, organizations may discover that understanding the origin of information is insufficient, why they must also understand the origin of decisions. Because the strategic risk is rarely the existence of data - it is the conclusions derived from it.
As AI systems participate in reasoning chains, organizations must understand the transformations that occur between input and decision. Not only for auditability, but for interpretability of outcomes, because the highest risk is not incorrect data. It is unexamined reasoning. However, technical decision provenance is, given our current technology level, not fully possible (yet), why different concepts must be introduced, like the above-proposed intent validation.
Build for Cognitive Resilience
Traditional security architectures assume technical failures. Future organizations may also need to assume cognitive failures, as recommendations may be wrong, models may drift, synthetic realities may emerge, or external cognitive dependencies may change unexpectedly. The question is not whether such failures occur - they will.
The question is whether the organization can recognize them before they become systemic. Cognitive resilience therefore means maintaining mechanisms that preserve independent judgment, including redundancy of interpretation, diversity of perspectives, and human expertise capable of challenging automated conclusions or alternative sources of understanding. Historically, resilience was often treated as a technical property, which may become a cognitive one.
Cognitive resilience again requires redundancy of judgment, diversity of reasoning approaches, and preservation of human expertise capable of challenging synthetic outputs. Not as opposition or alternative to AI, but as a safeguard against monoculture in decision-making. Because monoculture reduces variance, and reduced variance increases systemic fragility.
Invest in Intent Awareness
The most speculative recommendation is also the one that may prove most important...
Identity tells us who acted, authorization tells us what they were allowed to do. But neither explains why an action occurs. Therefore a shift towards understanding intent may be required, answering at least the following additional questions:
- Why is the action occurring?
- Whose objectives are being pursued?
- Who benefits from the outcome?
- What incentives shape the behavior being observed?
As systems become more autonomous, intent may become the most strategically relevant missing layer. Not in the sense of perfect validation. But in the sense of recognizing directional behavior, incentive structures, and emerging patterns of influence.
Future competition will not be defined solely by control of systems. It will be defined by understanding what those systems are trying to achieve, regardless of whether that intent originates from humans or synthetic agents.
Because future conflicts may be won or lost not through superior technology, but through superior understanding of motivations, incentives, objectives, and influence.
In other words: understanding not merely what happens, but why.
Final Thoughts
The Maginot Line was not a failure because it was badly engineered... It was a failure because it was optimized for a battlefield that no longer existed.
Many of today's discussions around AI risk feel uncomfortably similar, as organizations continue focusing on systems, data, access, compliance, and governance. All of these matter. All of them remain necessary, but they may not be sufficient.
Because the strategic asset being transformed is no longer information alone, it's cognition itself. And if cognition becomes infrastructure, then it will eventually become a target. The organizations that adapt successfully may not be those that deploy the most AI, regulate the most AI, or secure the most AI. They may be the ones that preserve their ability to think independently while operating in a world increasingly shaped by synthetic cognition.
Further Reading
Readers interested in exploring related concepts may find the following works useful:
Organizational Cognition & Decision-Making
- H. A. Simon, Administrative Behavior
- H. A. Simon, The Sciences of the Artificial
- E. Hutchins, Cognition in the Wild
Knowledge, Expertise & Organizations
- I. Nonaka, H. Takeuchi, The Knowledge-Creating Company
- P. F. Drucker, Managing Oneself
Information, Influence & Cognitive Conflict
- T. Rid, Active Measures
- M. Gurri, The Revolt of the Public
- NATO cognitive warfare publications
AI Governance & Accountability
- NIST AI Risk Management Framework
- OECD AI Principles
- EU AI governance materials
Strategic Dependency & Sovereignty
- S. Zuboff, The Age of Surveillance Capitalism
- European strategic autonomy and digital sovereignty publications