This verification software, created by researchers at the University of Surrey can end up used as part of a company’s online security protocol, helping an organization understand whether an AI has learned too much or even accessed sensitive data.
The software is also capable of identifying whether AI has identified and is capable of exploiting flaws in software code. For example, in an online gaming context, it could identify whether an AI has learned to always win in online poker by exploiting a coding fault.
“In many applications, AI systems interact with each other or with humans, such as self-driving cars in a highway or hospital robots,” said Dr. Solofomampionona Fortunat Rajaona, a research fellow in formal verification of privacy at the University of Surrey and the lead author of a paper on the subject. “Working out what an intelligent AI data system knows is an ongoing problem which we have taken years to find a working solution for.
“Our verification software can deduce how much AI can learn from their interaction, whether they have enough knowledge that enable successful cooperation, and whether they have too much knowledge that will break privacy. Through the ability to verify what AI has learned, we can give organizations the confidence to safely unleash the power of AI into secure settings,” he said.
“Over the past few months there has been a huge surge of public and industry interest in generative AI models fueled by advances in large language models such as ChatGPT,” said Professor Adrian Hilton, director of the Institute for People-Centred AI at the University of Surrey. “Creation of tools that can verify the performance of generative AI is essential to underpin their safe and responsible deployment. This research is an important step toward maintaining the privacy and integrity of datasets used in training.”
Click here to read the full paper.

