Is AI safe for customer and order data? Questions to ask any vendor

The question every owner asks first

When we talk to owners about putting AI on the order desk, the first question is rarely about speed. It's this: "Is it safe to put our customer and order data into AI?" That's a fair question. Your order files hold customer names, ship-to addresses, prices, part numbers and sometimes bank details. Losing them, or having them show up somewhere they shouldn't, would hurt.

The honest answer is that AI can be as safe as the other business software you already trust, like your email, your ERP or your accounting system. It can also be a mess if people use free consumer tools with no rules. The difference comes down to which plan you use, how it's set up, and who checks the work. This guide covers what the data says, what the vendors actually promise in writing, and the questions to ask before you sign anything.

What the breach numbers say

Start with the cost of getting it wrong. IBM's Cost of a Data Breach Report 2025, a study of 600 breached organizations run by the Ponemon Institute and sponsored by IBM, put the average U.S. breach at a record $10.22 million. The global average was $4.44 million. Those are averages across all sizes of company, so a 150-person distributor would likely see a smaller bill. The point is that a breach is expensive no matter how big you are.

The same report looked at AI for the first time in detail. 13% of organizations reported a breach of an AI model or app. Of those, 97% said they had no AI access controls in place. 63% of the breached organizations had no AI governance policy or were still writing one. In plain terms: the AI breaches mostly happened where nobody had set rules about who could use what.

IBM also counted "shadow AI," meaning AI tools staff use without the company's approval. One in five organizations reported a breach tied to shadow AI. Where shadow AI use was high, breaches cost an average of $670,000 more. IBM notes that AI breaches are still a small slice of its sample, so read these as early signals rather than settled facts.

IBM has since published a 2026 edition (602 organizations). It says more than 20% of organizations reported a breach targeting AI models or apps, and the top causes were compromised APIs, apps or plug-ins and cloud misconfigurations, at 27% each. Those are setup problems, not problems with the AI itself.

Where AI breaches happened, the rules were missing

IBM Cost of a Data Breach 2025, 600 breached organizations. Each bar has its own base, noted below it.

1. AI breach, no access controlsShare of organizations with an AI model or app breach97%
2. No AI governance policyShare of breached organizations, none or still developing63%
3. Breach tied to shadow AIOne in five organizations20%
4. Breach of an AI model or appShare of all organizations studied13%

Source: IBM and Ponemon Institute, Cost of a Data Breach Report 2025 (July 30, 2025). Sponsored and analyzed by IBM.

Your AI vendor is a third party

Verizon's 2025 Data Breach Investigations Report, which analyzed more than 12,000 confirmed breaches, found that third-party involvement in breaches doubled to 30%. That number covers all kinds of suppliers, not just AI. But it's the right way to think about AI: any tool that reads your orders is a vendor with access to your data, the same as your ERP host, your EDI provider or your IT company. Treat it that way.

That means the questions you'd ask any vendor apply here too. Who are they? Where do they keep your data? Who on their side can see it? What happens if they get breached? Can you get your data back, and can you make them delete it?

What the AI companies promise in writing

This is where business and consumer plans split. Here is what the main vendors' own terms and documents say.

• Anthropic (Claude), business terms. Anthropic's Commercial Terms of Service say plainly that Anthropic "may not train models on Customer Content from Services." Its data retention page says API inputs and outputs are deleted from its back end within 30 days, with exceptions such as zero data retention agreements, usage policy enforcement and legal holds.
• Anthropic, consumer plans. Free, Pro and Max accounts are different. In August 2025, Anthropic asked consumer users to choose whether their chats could be used for training, with data kept for five years if they allow it. Anthropic says those changes don't apply to services under its commercial terms.
• OpenAI, API. OpenAI's data controls page says data sent to the API has not been used to train its models since March 1, 2023, unless you opt in. Abuse-monitoring logs are kept for up to 30 days by default. Zero data retention is available for some endpoints, subject to OpenAI's approval.
• Microsoft 365 Copilot. Microsoft's Copilot privacy page says prompts, responses and data accessed through Microsoft Graph aren't used to train its foundation models. It also says Copilot only shows a user data they already have at least view rights to. That's good, but it means a messy SharePoint with open folders becomes a messy Copilot.

The model is rarely the weak spot. The free account, the open folder and the unchecked vendor are.

So the gap that matters most isn't between AI vendors. It's between the business plan your company controls and the free personal account an employee opened on their own. Harmonic Security, which sells AI governance tools, measured 1.9 million minutes of AI use in 2026 and found that 64.5% of activity on personal AI accounts was business use. That's vendor research, so weigh it as such. It still matches what most owners suspect: people are already using these tools for work, on accounts the company can't see.

Where the real risk sits in a mid-size office

For a 40 to 400 person manufacturer or distributor, the risks that come up most are ordinary ones.

1. Free personal accounts. A buyer pastes a customer's price list into a free chatbot to reformat it. Nobody approved the tool, and the company has no contract with the vendor.
2. Too much access. An AI tool connected with an admin login can see payroll, HR files and bank records, when it only needed the order inbox and the item master.
3. No human check. Software that posts orders or changes vendor records with nobody reviewing them turns one bad read into a real shipment or a real payment.
4. Unchecked add-ons. Browser extensions and plug-ins that promise to "summarize your email" may send it to a company you've never heard of. IBM's 2026 data puts compromised apps and plug-ins among the top causes of AI breaches.
5. Payment fraud. This one isn't new, but AI makes fake emails more convincing. Business email compromise losses reported to the FBI topped $3 billion in 2025, according to the FBI's 2025 Internet Crime Report. Keep payment approvals and bank-detail changes with a person, whatever software you run.

Questions to ask any AI vendor

Bring this list to any AI vendor, including us. Good vendors answer each one in writing without stalling.

A note on SOC 2. The AICPA describes SOC 2 as an examination of a service provider's controls over security, availability, processing integrity, confidentiality or privacy. It's useful, but the AICPA itself has warned about "fast and easy" SOC promises in 2026. Ask for the actual report and who audited it. A small vendor without one isn't automatically unsafe, but they should be able to answer the other six questions clearly.

Where the other side has a point

Some owners say the safest choice is to keep AI away from customer data entirely. That's a reasonable position for some data, like health records or defense work with special rules. And no contract clause makes a breach impossible. Vendors get hacked, and terms can change, as Anthropic's consumer update shows.

But "no AI" usually doesn't mean no AI. It means AI with no rules, because staff use free tools anyway. IBM's shadow AI numbers point that way. A business plan with a contract, a short list of approved tools and a person approving each order is usually safer than a ban nobody follows.

A short checklist for this month

1. Pick approved tools. Choose one or two AI tools on business plans and tell staff which ones to use. Say plainly that customer data doesn't go into personal accounts.
2. Write one page of rules. What data can go in, what can't, and who to ask. IBM found 63% of breached organizations had no AI policy at all.
3. Limit access. Give any AI tool only the inbox, folders and ERP screens it needs. Clean up open SharePoint folders before you turn on Copilot.
4. Keep a person approving. Software can draft the order; a person approves it before it posts. Same for invoices, payments and vendor bank changes.
5. Ask the vendor questions above before any new tool touches orders or customer files, and keep the answers on file.

How Peekgentic handles your data

We never sell or share your data, and we never let it be used to train AI. We ask for the least access each program needs and set it up with your IT team, and you can take that access back at any time. A person on your team approves every order before it reaches your ERP, every action is logged, and we'll sign your NDA before we see anything. We answer vendor security questionnaires in writing. The details are on our security page. See also how an order-processing build works.