AI can make everyday business tasks faster, from drafting documents and analysing customer information to recruitment, marketing and internal decision-making. But once personal data is entered into, analysed by or generated through an AI system, data protection law may apply. The risk is not limited to businesses developing their own AI models. Using a third-party AI tool can also create compliance issues around lawfulness, transparency, security, data minimisation, accuracy and automated decision-making. The practical question is therefore not simply whether a business can use AI, but whether it understands what information the system processes, why it is being used and what safeguards are needed.
Key Takeaways
Using a third-party AI tool does not remove your organisation's responsibility for how personal data is processed.
Personal data should not be entered into AI systems simply because doing so is convenient. The business still needs an appropriate purpose and lawful basis.
Sensitive or confidential information requires particular care, including understanding how the AI provider stores, uses, transfers and retains data.
AI-generated inferences and predictions about identifiable people can themselves amount to personal data.
High-risk AI processing may require a Data Protection Impact Assessment before the processing begins.
Businesses should establish clear internal rules for approved AI tools, permitted information and human oversight rather than allowing unmanaged employee use.
When Does Using AI Become a Data Protection Issue?
Data protection law is concerned with the processing of personal data: information relating to an identified or identifiable individual. AI itself is not automatically a data protection issue.
The position changes when personal data enters the process. This might happen when an employee uploads a customer email into an AI assistant, HR uses an automated recruitment system to evaluate candidates, a business analyses customer behaviour using AI, or an AI system produces predictions or inferences about particular people.
The Information Commissioner's Office (ICO) makes clear that data protection law can apply throughout the AI lifecycle, including when personal information is used to train or test a system and when an AI tool is deployed to make predictions or decisions about individuals.
For businesses adopting existing AI products, one of the first questions should therefore be simple: what information are we actually giving this system?
What Are the Main Data Protection Risks of Using AI?
1. Uploading personal data without a clear lawful basis
AI does not create an exemption from ordinary data protection rules.
If a business processes personal data using AI, it needs to understand the purpose of that processing and identify an appropriate lawful basis. Different AI activities may involve different purposes and therefore require separate consideration.
For example, a company may already hold customer information to provide a service. That does not necessarily mean it can reuse the same information for every new AI-related purpose without further analysis.
This becomes more sensitive where the information includes special category data, such as information concerning health, racial or ethnic origin, political opinions, religious beliefs, sexual orientation or certain biometric data. Additional requirements can apply to the processing of such information.
2. Employees putting confidential or unnecessary information into AI tools
One of the most immediate business risks is also one of the easiest to overlook: employees copying information into general-purpose AI tools without considering what that information contains.
A prompt could include customer details, employee information, internal correspondence or commercially sensitive material. The business then needs to understand what happens to that information after submission.
Questions should include:
Is the information retained?
Where is it stored?
Who can access it?
Is it used to improve or train the provider's systems?
Can that use be disabled?
Is information transferred internationally?
What contractual protections apply?
The answer will depend on the particular provider, product, subscription and configuration. Businesses should therefore assess the actual service they use rather than assuming all AI products handle information in the same way.
3. Collecting more personal data than the business actually needs
AI systems can encourage a "more data is better" approach. Data protection law takes a different starting point.
The data minimisation principle requires personal data to be adequate, relevant and limited to what is necessary for the purpose for which it is processed.
If a member of staff only needs help improving the wording of a customer communication, for example, the AI tool may not need the customer's name, contact details, account information or complete correspondence history.
Removing or reducing personal data before information reaches the AI system can materially reduce risk.
4. Lack of transparency
People may reasonably want to know how their information is being used, particularly where AI plays a meaningful role in analysing them or influencing decisions.
Businesses therefore need to consider whether their privacy information accurately reflects their use of AI and the underlying processing.
Simply adding the words "we may use AI" to a privacy notice is unlikely to solve the underlying issue. Good transparency requires the business itself to understand the processing well enough to explain its purpose, the information involved, relevant sharing and retention arrangements.
5. Incorrect AI outputs becoming part of business records or decisions
AI systems can produce inaccurate information.
This creates a data protection issue where incorrect output concerns an identifiable individual and is then relied upon, stored or used to make a decision.
Consider an AI system that incorrectly categorises a customer as presenting a fraud risk or produces an inaccurate summary of an employee's performance record. The commercial problem is not merely that the technology made a mistake. The organisation may then act upon inaccurate personal data.
Human review should therefore be meaningful rather than ceremonial, particularly where AI output could materially affect an individual.
6. Bias and unfair treatment
AI systems can reproduce or amplify patterns within the information and processes on which they depend.
This matters particularly in areas such as recruitment, workforce management, credit assessment, customer segmentation and fraud detection.
Under the fairness principle, personal information should not be processed in ways that are unjustifiably detrimental, unexpected or misleading. Businesses should therefore examine not only whether an AI system functions technically, but whether its use may create unfair outcomes.
Other legal obligations, including equality and discrimination law, may also become relevant depending on the particular use case.
What About Automated Decisions Made by AI?
Automated decision-making deserves separate attention because the UK framework changed following the Data (Use and Access) Act 2025.
The legislation introduced a more permissive framework for certain significant decisions made solely through automated processing while retaining safeguards for individuals. Where the relevant provisions apply, safeguards can include informing people about significant automated decisions, allowing them to make representations or challenge the decision and enabling them to obtain human intervention.
More restrictive conditions continue to apply where significant solely automated decisions involve special category personal data.
The important distinction is whether a person genuinely participates in the decision. Having an employee simply approve whatever the system recommends may not amount to meaningful human involvement.
Businesses using AI for recruitment, eligibility assessments, access to services or similarly significant decisions should examine this area particularly carefully.
Do You Need a Data Protection Impact Assessment Before Using AI?
A Data Protection Impact Assessment, or DPIA, is required where proposed processing is likely to result in a high risk to individuals' rights and freedoms.
Not every use of AI automatically requires one. An employee using an approved tool to rewrite non-personal marketing copy presents a very different risk profile from an AI system profiling employees or making significant decisions about customers.
Where a DPIA is required, it should be carried out before the relevant processing starts. Even where it is not legally mandatory, a proportionate assessment can still help a business understand the information involved, identify risks and document why its chosen controls are appropriate.
Who Should Be Responsible for AI Data Protection Compliance?
AI governance should not sit exclusively with the IT department.
Depending on the organisation, responsibility may involve management, data protection or compliance personnel, information security, HR, legal advisers, procurement teams and the business function actually deploying the technology.
A practical governance model should identify:
which AI tools are approved;
what categories of information may be entered;
prohibited or restricted uses;
who approves higher-risk use cases;
when a DPIA or additional review is required;
how suppliers are assessed;
what human oversight is required; and
how incidents or inappropriate use are reported.
This matters because unofficial use can quickly undermine formal compliance. A business may carefully assess its enterprise AI platform while employees independently use free consumer tools for client files, HR documents or internal correspondence.
The real control environment needs to match how people actually work.
Before You Proceed, Check…
Before introducing an AI tool that may process personal data, check:
What exact business problem is the AI tool solving?
Will personal data be entered, collected, generated or inferred?
What lawful basis applies to each relevant processing purpose?
Could special category or criminal offence data be involved?
Can personal information be removed, anonymised or reduced?
How does the provider use, retain and protect submitted information?
Are international data transfers involved?
Does your privacy information accurately explain the processing?
Could the system influence a significant decision about an individual?
Is meaningful human review required?
Is a DPIA necessary?
Do employees have clear rules governing acceptable AI use?
Common Mistakes Businesses Make
Allowing AI use before setting internal rules
Employees often adopt useful technology faster than internal governance develops. That can leave management unaware of which tools are being used and what information is being uploaded.
Treating an AI provider's terms as the entire compliance exercise
Supplier terms matter, but they do not determine whether your own processing is lawful, fair, necessary and transparent. The organisation still needs to understand its purpose and responsibilities.
Uploading complete documents when only part of the information is necessary
Providing an entire employee file, customer record or email chain to obtain assistance with one narrow task can create avoidable exposure. Start by asking what information the AI genuinely needs.
Assuming human approval removes automated decision-making concerns
Human involvement needs to be meaningful. A person who routinely accepts an algorithmic recommendation without independently considering it may provide little effective protection.
Reviewing privacy after deployment
Data protection should be considered while the AI use case is being designed and procured, not after the system has already started processing personal information.
FAQs
Can employees use ChatGPT or another generative AI tool with customer information?
Potentially, but the business must first understand and control the relevant processing. The answer depends on the information involved, the purpose, the lawful basis, the provider's terms and data practices, security arrangements and the organisation's internal controls.
Is information entered into an AI tool still personal data?
Yes, if the information relates to an identified or identifiable individual. AI-generated predictions or inferences about identifiable people may also constitute personal data.
Does removing someone's name make information anonymous?
Not necessarily. Information can remain personal data where an individual can still be identified directly or indirectly from the remaining information or when combined with other information.
Does every business using AI need a DPIA?
No. A DPIA is required where the processing is likely to result in a high risk to individuals' rights and freedoms. The assessment depends on the particular AI use case and the nature and impact of the processing.
Can AI make decisions about employees or customers automatically?
UK law permits certain significant solely automated decisions subject to the applicable conditions and safeguards. Additional restrictions apply where special category personal data is involved. Businesses should assess significant automated decision-making separately rather than treating it as an ordinary AI use case.
Should our privacy notice mention AI?
Where AI changes how personal information is processed in a way that is relevant to the information businesses must provide to individuals, privacy information may need to be updated. The explanation should describe the actual processing rather than relying on a generic reference to "AI".
Is using an enterprise AI subscription automatically GDPR compliant?
No. Enterprise functionality may provide stronger contractual, security or data-control options, but the organisation still needs to assess its own purposes, lawful basis, data minimisation, transparency, security and other applicable obligations.
When to Get Support
Businesses introducing AI into customer, workforce or operational processes may benefit from reviewing the proposed use before personal information begins flowing through the system. Sterling Legal Solutions can support businesses with data protection and compliance considerations, AI governance frameworks, supplier and contractual arrangements, internal policies and the broader commercial implementation of AI within business operations.
Disclaimer
This article is provided for general information only and does not constitute legal, tax, financial, regulatory, investment or other professional advice. Specific advice should be obtained before relying on any particular course of action.
Sources
Information Commissioner's Office — Artificial intelligence and data protection guidance
Information Commissioner's Office — AI and data protection risk toolkit
Information Commissioner's Office — Guidance on lawfulness, fairness, transparency, security and data minimisation in AI
Information Commissioner's Office — Data Use and Access Act 2025 guidance for organisations
Data (Use and Access) Act 2025 — automated decision-making provisions