That is where a new project called VERITAS (VERified Infrastructure for Trustworthy AI in Science) comes in. Led by principal investigator Anita Nikolich, research scientist and director of research and technology innovation at the University of Illinois School of Information Sciences, the project will address this gap by establishing AI Assurance as a core function of scientific research infrastructure.
The project aims to develop practical methods for documenting, reviewing, and stress-testing AI systems before they end up used in high-impact scientific workflows.
Additional Risks
Traditional cybersecurity focuses on preventing unauthorized access, catching malware, and stopping data theft. AI-enabled research introduces additional risks that may not trigger conventional security alerts.
A poisoned dataset, for example, may appear statistically normal while causing a model to produce unreliable results. A backdoored model downloaded from a public repository may contain no recognizable malware and may operate normally until a particular input activates its hidden behavior. An autonomous AI agent may have excessive permissions that allow it to alter data, invoke laboratory tools, or manipulate a research workflow.
In each case, the infrastructure may appear secure while the scientific result ends up compromised.
“We cannot simply bolt traditional cybersecurity onto AI-driven science,” Nikolich said. “When a poisoned dataset or backdoored model produces an answer that looks plausible but is subtly wrong, no firewall or virus scanner is likely to catch it. The researchers doing our most important scientific work deserve assurance that the AI systems they rely on are documented, tested, and behaving as intended.”
University of Illinois’ Anita Nikolich is working to establish AI Assurance as a core function of scientific research infrastructure.
Source: University of Illinois
Rather than requiring scientists to become cybersecurity experts or expecting cybersecurity teams to become machine-learning specialists, VERITAS will integrate AI Assurance into the research infrastructure scientists already use, Nikolich said. The project has three connected components:
1. Model and data documentation
VERITAS will pilot standardized model cards and dataset datasheets for large scientific computing allocations. Similar to nutrition labels on packaged food, these documents describe where a model or dataset came from, how it ended up created or modified, its intended use, its known limitations, and the assumptions researchers should understand before reusing it. The goal is to improve transparency, reproducibility, and the ability to trace problems through complex AI workflows.
2. Operational AI security services
VERITAS will pilot a new AI Assurance Engineer role at the National Center for Supercomputing Applications (NCSA). The engineer will review selected technically novel AI projects before deployment, scan model files for unsafe or malicious behavior, examine software for vulnerabilities, and assess the risks around uses of autonomous agents.
3. Model and data integrity challenges
Through the National Data Platform (NDP) Education Hub, VERITAS will create hands-on challenges that train students to detect poisoned data, inspect potentially compromised models, evaluate agent permissions, and identify weaknesses in scientific AI workflows.
AI red-teaming by deliberately attacking an AI system to find its weaknesses before an adversary does is now a well-established field. Having said that it is rare you would see it used in scientific research, where a manipulated model produces a false result that can pass for legitimate science. VERITAS is among the first efforts to adapt the practice to scientific cyberinfrastructure.
“AI systems can fail in ways that are difficult to distinguish from legitimate scientific results,” Nikolich said. “Proactive red teaming allows us to identify those weaknesses before a vulnerable model or agent becomes embedded in a research pipeline. The objective is to help research teams make their systems more trustworthy and resilient.”
AI Assurance Workforce
VERITAS will also help prepare students for careers at the intersection of machine learning, cybersecurity, and scientific computing. Participants in the project’s challenges will work with realistic scientific models, datasets, and infrastructure using NDP while learning about responsible disclosure practices.
By embedding documentation, security review, adversarial assessment, and workforce development into existing scientific cyberinfrastructure, VERITAS seeks to create a model for AI Assurance that can end up adopted by supercomputing centers, research institutions, and national-scale AI infrastructure providers.
“AI is now part of the scientific workflow,” Nikolich said. “We need to protect its integrity just as seriously as we protect the networks and computing systems around it.”