AI security and EU compliance: the AI Act and GDPR for businesses
Practical guide for businesses using AI in the EU: framework, transparency, GDPR, permissions, assessment and incident management.

AI security reduces the risk of a system exposing data, providing uncontrolled responses, or triggering unwanted actions. Compliance in the European Union also requires the analysis of the purpose, the role of the organization and the data processed. The AI Act and GDPR may be relevant simultaneously and the obligations differ by use. Start with a system sheet, check the legal framework and turn concrete risks into controls you can demonstrate.
A chatbot with public information and a tool used to evaluate candidates can use the same family of models. The consequences are different. The first can misdirect a visitor, the second can change a person’s access to a job. The analysis starts from the implemented function and its effects, including when employees use a product bought from someone else.
How does the AI Act apply to a company project?
Identify the stated purpose and the actual purpose. A tool described as an “administrative assistant” can only transcribe grades or order candidates based on a rating. These uses must be documented separately. Note who uses it, in which system the output is seen, which decision it can influence, and which people are affected. Read the applicable text and the Commission’s case-by-case guidance before launching. Regulation (EU) 2024/1689
Clarify the role of the company. In AI Act terminology, a provider and an implementer have distinct responsibilities. A company that uses under its authority a system of another supplier can have the role of implementer. A company that develops or commissions the development and puts a system into operation under its own name can be a supplier if the conditions of the definition are met. Substantial changes and change of purpose can change the role in certain situations. The label used in a commercial offer does not alone establish classification.
It separates legal categories from internal severity. Checks for prohibited practices, potentially high-risk cases and transparency obligations. A firm can impose strict internal controls even on a system that does not fall into a high risk category. An assistant connected to commercial documents has an access and disclosure risk that is worth addressing in the architecture regardless of the legal label.
What deadlines must be followed in the EU?
At the time of writing, the European Commission indicates August 2, 2026 for the application of the transparency obligations in Article 50, with the nuances and conditions in the text and in the guidelines. The calendar updated by Regulation (EU) 2026/1744 — Digital Omnibus on AI indicates 2 December 2027 for the rules related to certain high-risk uses in Annex III and 2 August 2028 for targeted systems integrated in regulated products. Check the deadlines and any transitional provisions before the launch decision. AI Act Calendar published by the European Commission
Transparency requires a spot check. For systems intended to interact directly with people, Article 50 includes conditions on information on interaction with AI, including an exception where this is obvious in the circumstances provided for. There are separate requirements for marking certain generated output and for disclosure in certain uses. A short message at the beginning of the chat can be useful, but it does not automatically cover all situations where material is generated or published. Commission Guidelines on Article 50
For AI literacy, check Article 4 as applicable to the organization. A practical approach is to train the team according to the task. Operators should know how to check an output and report a problem. Technical managers should be aware of access, limits and stop flow. Keep track of relevant exercises and update it when purpose or tool changes.
How do the AI Act and the GDPR overlap?
GDPR is analyzed when there is processing of personal data. Check the purposes, basis, need of each field and how long it is kept. Find out who is the operator or authorized person depending on the specific activity and what provisions are required for suppliers. Also check locations, access from other countries and conditions applicable to transfers, as applicable. An EU hosting option only solves part of the question of whether there is access or subcontracting elsewhere.
Analyze Article 22 when there are decisions based solely on automatic processing that produce legal effects or similarly and significantly affect a person. The regulation provides conditions and exceptions, and the analysis must be done for the specific process. A mechanically used acknowledge button is worth evaluating against what the person in the loop can do. For processing likely to create a high risk for the rights and freedoms of individuals, check the impact assessment requirements of Article 35. GDPR on EUR-Lex
It effectively documents the flow of data. A response can be kept in the application, in the orchestration logs, and in a quality review system. A user requesting correction of information does not see all copies. The team should know where they are, what is changing, and what information needs to be retained under an applicable rule. Tests deletion and revocation of access on an approved case, including from indexes and relevant operational copies.
What does the inventory look like before the purchase?
| What are you recording? | Useful question | Sample proof |
|---|---|---|
| Purpose and consequence | What can the system decide or change? | Flow chart and representative case |
| Inputs and outputs | Where does the data go and who sees it? | Data map and retention settings |
| Access | What instruments can he summon and on whose behalf? | Permissions and Revision Matrix |
| Suppliers | Who processes or has access to the data? | Contract documents and relevant list of subcontractors |
| Validation | How do we spot incorrect output? | Assessment set and criteria |
| Incidents | Who stops the flow and who decides the notification? | Documented procedure and exercise |
Complete inventory for multiple scenarios. A sourced search function can start internally and then be available to customers. The second stage changes the audience and can change the accessible data, information requirements and volumes. Put a review date and the condition that the scope extension be explicitly approved.
Add a responsible person who can explain each entry. When the provider changes the terms of service or adds a processor, who reads the notification? When a department requests a new index, who confirms permissions? A registry that only exists at purchase becomes obsolete as the product evolves.
What technical controls are worth implementing?
Put authorization in the app that controls data and actions. The model may suggest a search, but the service determines what documents it is allowed to return for that identity. Filtering is done before restricted documents enter the context. Also check previews, attachments and access to previously generated responses. For operations with effects, ask for proper confirmation and recheck permissions on execution.
Treats incoming text and model output as inputs that may be incorrect. Documents and conversations may contain unwanted instructions. It uses structural validation, allowed sources, and integrations with limited operations. An output containing an address, rate or identifier must be verified by the authoritative system. If the check fails, the user sees a clear status and can call the route manually. Guide to Assistant Permissions and Security
Maintain separate contexts between users and organizations. In a multi-tenant service, a shared index requires consistent checking of access limits. Monitor strange output, but limit the sensitive information you copy to the logs. An identifier and the state of the action may be sufficient for correlation. Protect history, documents and results according to their sensitivity.
Set a rating before activating each integration. Use synthetic data for situations where rights differ, results conflict or the service is not responsive. It verifies that the system can opt out, that the reviewer sees the required information, and that the data ceases to be available after the revocation. NIST Profile for Generative AI
How do you prepare the response to an incident?
Define scenarios and owners. For the disclosure of restricted information, for example, the person on duty must be able to stop the affected access, keep the necessary traces and inform the designated authorities. If the system issues a wrong operation, identify what can be undone, who can do it, and how customers receive an explanation. The procedure can have two tracks, the immediate technical limitation and the assessment of legal obligations after the situation.
It records the time, the features affected and what version you were running. It preserves evidence on restricted access channels and protects the people whose data appears in the incident. In case of a personal data security breach, the analysis of the notification is done according to Articles 33 and 34 GDPR, depending on the risks and circumstances. For systems covered by the AI Act there may also be specific obligations. Do not delay the involvement of responsible persons until technical troubleshooting is closed. GDPR on EUR-Lex
Practice an incident with dummy data and verify access to stop buttons, history and valid contacts. In a useful exercise, one person simulates the client’s reporting, one limits the function, and another reconstructs the timeline. Note how many minutes it took to identify the version and what information was missing. Apply improvements before the actual incident requires the same steps.
What documents should you be able to show at an inspection?
Maintains the system sheet, framing analysis, data map, responsibilities, supplier terms and evaluation evidence. Add relevant licenses and settings when using them may change the result. Together, the documentation should explain the difference between the evaluated and active configuration. If a parameter affects access, note who changed it and after what checks.
Check the documents by reconstructing a permitted case. It shows who submitted the request, what category of sources was accessible, and how the proposal was accepted or rejected. If the tools cannot reconstruct the case, it identifies the missing information and decides whether it can be collected proportionately. Mass collection of conversations can add risk, so choose the evidence that allows troubleshooting without remembering every detail.
How do you manage post-launch changes?
Consider a change in model, vendor or document set as an impactful change. For a new model, recheck quality, uncertainty behavior, and cost. For a new source, check who can read it and what information needs to be excluded. For a new provider, check the data flow and conditions before sending the same traffic. Ties changes to the system sheet and responsibilities established at release.
An internal assistant can be extended from answering about procedures to proposing actions. This extension adds effects even though the interface looks the same. Note what operations become possible, how they are confirmed, and who receives the alerts. When the purpose changes, it also revises the legal framework. An assessment made for reading does not automatically answer questions about writing or decision.
Ask for one piece of evidence for the essential conditions. Access revocation can be demonstrated by a test account. Restoring after unavailability can be demonstrated in the test environment. The person’s information can be checked on the actual route, in the languages offered. The proof should also be visible after the end of the pilot, with the version tested, the person responsible and the date of the next revision.
What do you check before publishing a customer chatbot?
Go through the information first from the perspective of a new visitor. He should understand who he is interacting with, what questions the service accepts, and how to reach a human when the issue cannot be resolved. Check separately whether messages used in automation reach other services and how long they are kept. If the user logs in to view an order, it compares the access from the chat with the access from the portal.
Also check the situation where there is no confirmed answer. The system must be able to explain the limit, retain the identifier, and transfer the issue when necessary. To update the terms of trade, identify the person posting the valid source and how old responses are invalidated. These product checks complement legal analysis and provide concrete examples for release documentation.
Frequently asked questions
Is a public chatbot automatically high risk?
Framing depends on purpose and use. An assistant for schedule and delivery questions is different from a system that recommends hiring decisions. It also checks the applicable reporting obligations, even if the analysis concludes that the system is not high risk.
Does a locally hosted model mean GDPR compliance?
Local hosting can reduce some transfers and dependencies. The organization must still consider the purpose, basis, access, retention and rights of individuals, where applicable. It also checks for auxiliary integrations that may send data to other services.
Who classifies the system and who tests it?
The technical and product teams describe the purpose and behavior, and the people responsible for compliance and data protection analyze the obligations. Testing brings together technical checks and operational scenarios. In high-impact projects, document who can stop the rollout when results are insufficient.
What is the first step for an already active assistant?
Inventory available resources and operations, then validate access using real test roles. It checks what the user knows about the system and if there is a manual path. Determines who handles an alert and how to stop an integration without losing requests.
What do you prepare before a technical and legal discussion?
Bring a common case, one with potential impact, list of integrations and relevant vendor terms. Also prepare an outline with the visible information for each role. These materials allow for a concrete discussion of framing and realistic control planning. For the design and implementation part you can start from John.Cloud AI solutions.