Is Our Data Safe If We Use AI?

The short answer

It depends on which kind of AI you’re using. “AI” covers everything from a free chatbot on someone’s phone to a model running within a cloud environment governed by your organization’s security controls. Those are very different things, and most of the confusion about AI and data comes from treating them as one.

There are four levels. Once you understand the differences, you can start asking the right questions about where your information goes, who can access it, and what happens to it along the way.

The four levels, plainly

1. Consumer AI tools

These are the free or personal versions of ChatGPT and similar apps. Depending on the product and settings, conversations may be retained and used to improve models, and there generally isn’t a business agreement governing how your information is handled.

Fine for drafting a birthday toast. Not where I’d want employees putting customer records, financials, or proprietary business information. Privacy settings help, but they aren’t a substitute for managed accounts and clear policies. If your employees are pasting work into personal AI accounts, this is the place to start looking at your current exposure.

2. Business AI services

These are the paid business and developer offerings from major AI companies. They typically come with contractual data protections, including commitments that your inputs and outputs aren’t used to train models by default, along with security controls and independent audits.

This is a meaningful step up and is appropriate for a lot of business work. Two things are worth understanding: not training on your data doesn’t mean not storing it, and retention varies by provider, product, and configuration. Your data still leaves your systems and goes to the AI provider for processing. That’s not inherently a problem. It just needs to be understood and governed.

3. AI within your existing cloud environment

Microsoft Azure and Amazon Web Services offer leading AI models through their cloud platforms. For businesses already using one of them, AI can run inside the environment you already manage, alongside your existing identity, permissions, and security controls, instead of in a separate place you have to govern from scratch.

4. Fully local AI

This means running models on hardware your organization owns and controls, potentially without an internet connection. It’s a legitimate approach for organizations with strict isolation requirements, including certain defense and classified work, and local models can be quite capable depending on the hardware.

But disconnected doesn’t mean secure. You become responsible for hardware, updates, model maintenance, access controls, backups, and physical security. For most businesses, that burden outweighs the benefit unless there’s a specific requirement for this level of isolation.

When AI inside your own cloud makes sense

For companies handling sensitive customer information, financial records, proprietary documents, or regulated data, running AI inside existing cloud infrastructure is often the practical middle ground. You get capable models while keeping sign-in, permissions, and much of the surrounding infrastructure under your organization’s control.

Here’s how that works in practice. Employees sign in with their existing work accounts. The system reads documents stored in SharePoint or other approved locations, limited to what each employee is allowed to see. When someone changes roles, their access changes with it. When IT disables an account, that person loses access to the AI system along with everything else.

That matters because security isn’t only about keeping outsiders out. It’s also about making sure employees only see what they’re supposed to see. An AI system that can search every document in the company can just as easily hand someone a file they’d never be allowed to open directly.

None of this is automatic. Using Azure or AWS doesn’t mean every AI service inherits your security settings or compliance agreements, and it doesn’t mean an AI system respects document permissions on its own. If you’re covered by a healthcare agreement, for example, the specific AI service still has to be confirmed as covered. The service, the setup, the data flows, and the contracts all need to be checked, and the permissions need to be designed and tested.

Level two can be perfectly appropriate, too. Not every company needs the strictest setup. The point is to choose the level deliberately, based on the information being handled.

Questions to ask any AI vendor

You don’t need to be technical to get a straight answer. Ask these and pay attention to who answers clearly and who changes the subject.

• Where does our data go when we use this? Whose infrastructure processes it? In which country? Is it stored somewhere else?
• Is our data ever used to train a model? You want a clear contractual commitment, not a verbal assurance from a sales rep.
• How long is our data retained, and who can access it? Ask about prompts, uploaded documents, responses, logs, and backups.
• Do our people sign in with their existing work accounts? If the answer is a new set of passwords, that’s another set of accounts IT has to manage and remember to shut off.
• Do you hold our passwords or administrative access? A well-designed system shouldn’t need them. Your IT team should control access and be able to revoke it.
• Does the AI respect existing document permissions? Can someone retrieve information through the AI that they couldn’t open directly?
• What happens when an employee leaves? Is their access to the AI and everything it connects to shut off with their account?
• Is your security audit about your company or this specific product? A vendor’s certification covers how they operate. It doesn’t establish that the system they built for you meets your requirements.

You don’t need to understand every technical answer. You do need to know someone has thought these through before your information starts flowing through the system.

The bottom line

For most mid-sized businesses, the answer looks like this: keep sensitive work off personal accounts entirely, use business AI services for everyday work, and put regulated or highly sensitive data inside your own cloud, set up and checked properly.

When I build a system for a client, I want it working within the infrastructure they already trust. Employees sign in through their existing work accounts. The system reads documents where they already live, limited to the permissions each person has. I don’t need to hold their passwords or keep a separate copy of their business files.

The goal isn’t to create another place where your company’s information lives. It’s to make the information and systems you already have more useful.

Have questions about AI and your data? Let’s talk.