Artificial intelligence (AI) is on the minds of just about everyone across all industry sectors these days, but as the technology grows, there have to be ways to ensure its security.

That is why global security agencies collaborated with the Australian Signals Directorate’s Australian Cyber Security Centre (ASD’s ACSC) to create a guidance entitled, “Engaging with Artificial Intelligence.”

AI is the theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making and translation between languages. Modern AI usually uses machine learning algorithms.

AI has several sub-fields of importance that include but not limited to:

Schneider Bold
  • Machine learning describes software components (models) that allow computers to recognize and bring context to patterns in data without the rules explicitly programmed by a human. Machine learning applications can generate predictions, recommendations, or decisions based on statistical reasoning.
  • Natural language processing analyzes and derives information from human language sources, including text, image, video and audio data. Natural language processing applications commonly end up used for language classification and interpretation. Many natural language processing applications not only process natural language but also generate content that mimics it.
  • Generative AI refers to systems that use data models to generate new examples of content such as text, images, audio, code and other data modalities. Generative AI applications typically train on large amounts of real-world data and can approximate human generated content from prompts.

AI systems are some of the fastest growing applications globally. It drives Internet searching, satellite navigation, and recommendation systems. AI also increasingly ends up used to handle activities traditionally undertaken by humans. Those tasks include  sorting large data sets, automating routine tasks, creative endeavors and augmenting business activities such as customer engagement, logistics, medical diagnosis, and cybersecurity.

Organizations from all sectors are exploring opportunities to improve their operations with AI. While AI has the potential to increase efficiency and lower costs, it can also intentionally or inadvertently cause harm.

Some common AI-related theats include:

Data poisoning of an AI model: Manipulation of training data and exploitation of model vulnerabilities (also known as adversarial AI), which can adversely affect the performance of machine learning tools and systems.

Input manipulation attacks – prompt injection and adversarial examples: Prompt injection is an input manipulation attack that attempts to insert malicious instructions or hidden commands into an AI system. Prompt injection can allow a malicious actor to hijack the AI model’s output and jailbreak the AI system.

Generative AI hallucinations: Outputs generated by an AI system may not always be accurate or factually correct. Generative AI systems can hallucinate information that is not factually correct. Organizational functions that rely on the accuracy of generative AI outputs could end up negatively impacted by hallucinations, unless appropriate mitigations are implemented.

Privacy and intellectual property concerns: AI systems may also present a challenge to ensuring the security of sensitive data an organization holds, including customers’ personal data and intellectual property.

Model stealing attack: A model stealing attack involves a malicious actor providing inputs to an AI system and using the outputs to create an approximate replica of it. AI models can require a significant investment to create, and the prospect of model stealing is a serious intellectual property concern.

The following organizations collaborated with ACSC on the guidance:

  • Cybersecurity and Infrastructure Security Agency (CISA)
  • Federal Bureau of Investigation (FBI)
  • National Security Agency (NSA)
  • United Kingdom (UK) National Cyber Security Centre (NCSC-UK)
  • Canadian Centre for Cyber Security (CCCS)
  • New Zealand National Cyber Security Centre (NCSC-NZ) and CERT NZ
  • Germany Federal Office for Information Security (BSI)
  • Israel National Cyber Directorate (INCD)
  • Japan National Center of Incident Readiness and Strategy for Cybersecurity (NISC) and the Secretariat of Science,
  • Technology and Innovation Policy, Cabinet Office
  • Norway National Cyber Security Centre (NCSC-NO)
  • Singapore Cyber Security Agency (CSA)
  • Sweden National Cybersecurity Center

Click here for more on mitigations explained in the guidance.

Pin It on Pinterest

Share This