The EU AI Act is often summarised as regulation of AI providers, the companies that build the models. That underplays what it means for the organisation actually running AI in production. If you deploy a high-risk system, obligations flow to you, not just to whoever built the model underneath.
There's a second point that matters just as much for procurement: the Act doesn't prescribe a specific technical implementation. It requires that you can demonstrate compliance, and that the evidence exists before an inspector asks for it. Below are the six deployer obligations that matter most operationally, and what proving each one looks like in practice.
First, know where you sit in the risk tiers
The Act classifies AI systems by risk, and your obligations depend on where each of your systems lands. Unacceptable-risk uses are banned outright, high-risk systems carry the full set of deployer obligations, limited-risk systems carry transparency duties, and minimal-risk systems have no specific obligations.
Most productive enterprise AI, such as profiling, decision support and citizen-facing services, lands in high or limited risk once it's deployed. That's why the six obligations below are worth treating as your working baseline rather than a niche concern for a few regulated sectors.
1. Human oversight
For high-risk systems, a human must be able to intervene, override or shut the AI down, and that capability has to be built in, not bolted on. You're the one who has to show it exists.
For a procedural chatbot, proving this means having routing paths to human support, being transparent about when AI is answering, and having a kill switch that actually works. "Actually works" is the operative phrase: an inspector will want to see the mechanism, not read a policy about it.
2. Fundamental rights impact assessment
Public bodies and providers of essential services must run a rights assessment before deployment. The Act doesn't treat this as a checkbox exercise. What you need to be able to produce is a documented, updatable assessment that travels with the system through its life.
The practical test is whether the document could be handed over today and still reflect the system as it currently runs, rather than as it was described at launch.
3. Logging and traceability
Every AI output must be logged so that its provenance can be reconstructed on demand. For chatbots and knowledge assistants, that means storing the retrieved passages, the model used, the prompt sent and the response given, for every interaction.
This is the obligation that most often turns out to be the gap. It's also the one that can't be added after the fact: if a system wasn't recording at the moment of the interaction, there's nothing to reconstruct later.
4. Data governance
Where you hold training data, it must be documented. Where you use a third-party model, you inherit its governance, and you must be able to prove you evaluated it.
This is where "the vendor handles that" stops being an acceptable answer. Under audit, the question isn't whether your vendor has a data governance story, it's whether you can show that you assessed it.
5. Transparency
Users must know they're interacting with AI. That's simple to state and wide in practice: chatbots must self-identify, AI-produced documents must be labelled, and search results influenced by AI must be flagged.
Proving this means being able to point to each place AI touches what a user sees, and show the disclosure at that point, not just in a terms page somewhere else.
6. Cybersecurity
AI systems must be resilient against adversarial input, and prompt injection is an in-scope attack surface. Your penetration testing has to explicitly cover it, rather than treating the model as a black box.
The evidence here is a test record: that injection scenarios were in scope, what was tried, and what was found.
The thread running through all six
Read the six together and a pattern emerges. None of them can be produced retroactively. Human-oversight paths, rights assessments, traceable logs and injection testing are architectural decisions, and they have to be designed in before deployment.
That's why these obligations belong in procurement rather than in a remediation project later. The questions you ask a vendor before signing largely determine whether you'll be able to answer an inspector afterwards. In practice, the gap we see most often is logging and traceability, the obligation that can't be added after the fact.
What to do about it
Three moves are enough to get started, and none requires a transformation programme.
- First, map your current and planned AI usage against the six obligations above and identify the gaps.
- Second, run a structured set of questions past every AI vendor in your evaluation, which gives legal and procurement a shared, defensible basis for the decision and tends to shorten the shortlist meaningfully.
- Third, ship one use case under real conditions, on your own content and against your own obligations, rather than as a sandbox proof of concept.
If you already have AI in production, our glass-box health check turns the transparency and traceability properties into an assessment you can run against an existing system.
For the vendor conversation itself, the fifteen questions in our whitepaper are designed to disqualify quickly, and the forty-question scored checklist is built to attach to a formal RFP: Download the vendor checklist.
The full reasoning, including the seven layers of AI sovereignty and the EPSO and Pidpa cases, is in the whitepaper this post is drawn from: AI sovereignty: a CIO's procurement guide for the EU AI Act.
For how data residency and audit access work at the infrastructure level, see our page on hosting and sovereignty. If your use case is citizen- or customer-facing, Trusted Answers shows what source-cited, traceable answers look like in practice, and why organisations choose Dropsolid AI sets out our own position.
Where Dropsolid AI fits in this
Dropsolid is the Drupal Diamond Partner behind the sovereign AI use cases in production at EPSO and Pidpa. Both deployments are documented end to end in our whitepaper series, including the audit traces they produce per interaction.
If you're mapping your own AI estate against these six obligations, or scoping a pilot that has to stand up to scrutiny, talk to our team or book a demo to see the architecture directly.