What to Ask Your IT Provider About AI

More IT and security providers now describe their services as "AI-powered." That label by itself tells you very little. What matters is where the technology is used, what it can reach, who checks its work, and what happens when it gets something wrong.

The questions below are designed to start a useful conversation with any provider, including us. A good partner should be comfortable answering them in plain language.

1. Where are you using AI, and what benefit does it provide?

A useful answer is specific. It names the tasks, such as drafting ticket summaries, sorting alerts, or searching documentation, and explains what improved: faster responses, fewer missed details, more time for senior staff. Be cautious of answers that stay at the level of "our platform uses AI throughout."

2. What business information reaches those tools?

Ask what data from your environment is sent to AI services: ticket text, emails, logs, file contents, passwords. A concrete answer explains which products are used, how retention and model training are handled under the provider's subscription and settings, and how sensitive information is kept out when it does not need to be there.

3. What can the tools access or change, and how is access restricted?

There is a big difference between a tool that reads data and one that can change accounts, reset passwords, or modify security settings. A good answer describes the permissions involved, how they are limited to what each task needs, and how access is logged and reviewed. Our article on connecting AI tools to business email explains why permission scope matters.

4. Which actions require human review, and how are results verified?

Ask which actions a person must approve before they happen, and how output is checked afterward. A thoughtful provider will be honest that review reduces mistakes rather than eliminating them, and will describe other safeguards as well, such as testing changes, limiting permissions, and being able to roll back.

5. What happens if the tool is unavailable or makes a mistake?

Every tool has outages and errors. A solid answer describes the fallback: how work continues if the service is down, how a mistake would be noticed, who fixes it, and how you would be told. If there is no clear answer, the tool may be more central to the service than it should be.

6. How do you measure whether it improves service?

Ask how the provider knows the technology is helping. Reasonable measures might include response times, resolution times, error rates, or client satisfaction, compared with how things worked before. You do not need exact figures in a first conversation. You do want a sense that someone is checking rather than assuming.

Monitoring is not the same as response

Many providers offer continuous automated monitoring. That means software watches for problems around the clock. It does not necessarily mean a person is available to act on what it finds at 2 a.m. on a Sunday.

Ask directly:

• What hours are staffed by people?
• What happens to an alert outside those hours?
• How quickly will someone respond, and is that written into the agreement?
• Who do you call in an emergency, and how is it escalated?

These answers matter more to your business than whether the tooling uses AI.

Look past the label

When you compare providers, focus on scope, access, testing, oversight, recovery, and accountability. These apply whether or not a service uses AI. Our article on the hidden risk of DIY cybersecurity explores why judgment and accountability still matter when answers are easy to find.

A balanced expectation

AI is a useful tool for some IT and security work, and less suited to others. A good technology partner should be able to explain both: where it helps, and where a simpler or more traditional approach is the better choice for your business.

If you are evaluating providers and would like to talk through these questions with someone local, you are welcome to join us for a Coffee & Cyber Chat.