Better Safe

Protecting against AI breach

It’s understandable that businesses the world over are looking for ways to incorporate AI into their operations. AI agents have helped boost productivity and improve customer service, contributing to efficiency increases and better bottom lines. Across C-suites, executives agree that businesses that don’t embrace AI will be at a significant disadvantage over the next few years. Yashin Manraj, CEO of Pvotal Technologies, is one of them. But, he warns, diving in headfirst without testing the water can lead to disaster. There’s so much data AI agents can collect – often deeply personal data – that any breach can have massive implications. The right security is paramount.

Overexposure

Leaders integrating AI agents into customer support often don’t understand the vulnerabilities.

Yashin Manraj

“One of the most common vectors we’ve seen is that people are able to basically inject some code or some prompt that allows them to retrieve other consumer data because a lot of these companies don’t understand how important it is to segregate data, how important it is to prevent their own AI agents from being able to read, for example, the entire database or entire consumer application,” Manraj told Reboot.

Too often, this information is left unprotected, accessible to bad actors who don’t even really need to resort to hacking. For example, the DeepSeek breach left more than a million critical records exposed. Chat histories, back-end data, API secrets, and other sensitive information was publicly accessible.

And whereas a lot of tracking information collected by internet search engines is anonymized, that is not usually the case with AI agents. People’s names are often connected to personal information they reveal about themselves to chatbots. Things like medical history and account information are particularly sensitive, but basically any identifying details can be leveraged by scammers.

Bad actors can use AI to their advantage, as well. They can use it to aggregate data exposed in a number of large breaches over the years and put together comprehensive personal files on individual people.

“They are able to build way better pictures of people than they used to,” Manraj said, “and so, unfortunately, that can also go into the hands of the wrong people and scammers are able to leverage the data much more effectively than they could three or four years ago.”

In the Wrong Hands

Bad actors are also using AI as a training guide to hacking businesses. While commercial LLMs have restrictions on what sort of information they will point users toward, hackers have figured out how to deploy those or similar language models locally on their computers to get around such restrictions.

“There have already been about 10 or 20 very popular repositories where people are sharing hacked LLMs that don’t have restrictions, so people are able to do everything from generating malicious code to scanning websites and finding vulnerabilities,” Manraj said.

And when companies use AI to build websites or applications, those tend to be more easily hacked.

“The biggest risk so far that is tangible is definitely that AI created code is very poorly managed and is more vulnerable than anything else,” he said. “So companies that are firing their developers or removing developers like Klarna are rehiring people now because they’ve just realized that it might not be the most judicious idea to go full scale on AI right now.”

While AI can do a lot of things, there aren’t yet any cases of hackers directly using AI to break into systems. There aren’t super hackers capable of Mission Impossible-style genius-level hacking. Rather, it’s almost always some sort of human error, including the decision to trust AI too much, that leads to breaches.

That can be good and bad, but ultimately, the buck stops with the people making the decisions anyway. Fortunately, there are best practices such as the principle of least privilege that companies can employ to keep data safe.

“Isolate the data that your AI can access from,” Manraj advises. “For example, the consumer to the internal employee or the employer’s access to your training data to your production to your staging environment. Maintaining isolation between your environments is key.”

For companies wanting to perform large-scale AI analysis to gain better insights into their operations, it can be tempting to aggregate all that data into a single use case. Avoid that temptation, Manraj warns.

“When a breach comes in, everything gets breached, and the company as a whole is vulnerable.”

DevSecOps principles like platform engineering maintain the right layers of security.

“For example, employer and employee and consumers should never be under the same access, should never be under the same level of service.”

An automated system of checks and balances in quality assurance and control processes can prevent a developer from pushing code that leaves a major security gap down the road.

Fools Rush In

For any consumer-facing enterprise with the need to evaluate data, there is no better tool than AI, Manraj says. It replaces classical dashboards, analytics, and consultants while delivering business insights. But when integrating AI tools into operations, companies need to tread lightly and test frequently.

“I would say none of the AI tools are plug-and-play,” he said. “It requires some engineering from the development team to be able to scale and fix, and address some of the issues.”

When DeepSeek came out, there was a flood of interest. It was free to use and develop. It was all over the news. But unbeknownst to so many, there was a gaping security hole.

“The major issue I think is that in all the AI consultants and the projects that people have adopted, is that you should not rush towards the shiny thing,” Manraj said. “So, I think it’s about carefully selecting which agent you have, and I always recommend choosing  more mature ones like Gemini and OpenAI. Despite controversies about the companies themselves, I think they do build solid products that allow you to be able to leverage the best of the analytics, the best of the AI components without as much security risk.”

Whatever agent they choose, businesses should speak with developers and others who have integrated them, and have a good understanding of the security, not just the AI. Proper testing to determine what data could potentially be leaked and what issues might arise helps develop a system of checks and balances to prevent those leaks. The integration scales infinitely, so just plugging and playing, and seeing what happens presents a very large risk.

Manraj recommends testing in three environments: development, staging, and production, before moving everything to production too quickly.

“Rushing often creates a lot of issues and people regret it because once your data is leaked, it’s very hard to close back Pandora’s box and your perception in the market is forever damaged.”

As with any tool, AI agents need to be used properly in the hands of skilled technicians to get the best results.