Generative artificial intelligence (AI) and large language models (LLMs) are valuable prizes for hackers as well as vector for security breaches. Semiconductor technology that is enabling the GenAI boom must be hardened to protect against emerging threats.
The GenAI Security Readiness Report 2024 recently released by Lakera found that the surge in adoption of GenAI and LLMs is accompanied by a low level of confidence in current security measures. The survey of more than 1,000 security professionals combined with real-world findings from Lakera’s AI hacking game, Gandalf, found that GenAI-related security risks may be underestimated, with only 5% of organizations expressing high confidence in their GenAI security frameworks.
The Lakera report found that although 38% of respondents are highly concerned about the risks from GenAI/LLM-related vulnerabilities, 62% have only moderate to low concern, and may not believe models are not accessing confidential data, even as use cases indicate otherwise.
The Lakera report concluded that there’s an urgent need for robust, AI-specific security strategies to address the unique challenges posed by GenAI, but there is uncertainty about the effectiveness of existing security approaches in protecting against sophisticated AI threats, with 86% of survey respondents holding moderate or low confidence levels in their current measures.
The rapid adoption of GenAI can learn from the deployment of edge AI and machine learning at the edge.
Scott Best, technical director at Rambus, said what makes GenAI different is that it is massive cloud deployment of LLMs using distributed compute systems. “There's only four or five platforms in the world capable of actually executing at scale right now,” he said. “They have vastly different security concerns.”
Most Rambus customers are looking at edge-based AI for use cases such as sensor fusion or self-driving AI solutions in automotive platforms, where the threat is that an adversary might extract something of value inside an FPGA and reverse-engineer it. “It's like giving them the source code to your software product,” he said. “Adversaries are again looking to borrow their resources of their competitors by reverse engineering what they're doing in their training model and then deploying it into their own systems.”
Training data must be tamper-proof
The data being collected at edge is often used for training, Best said, so authenticity is chief concern, as is privacy. When the data is at rest, sitting in flash memory, it’s critical that it’s kept private from prying eyes and authentic before it’s loaded into a chip, he said. “There's a lot of very interesting security trade-offs of turning data at rest into data for use very quickly and very securely.”
An LLM in a data center environment is both valuable intellectual property and a threat vector that can be exploited, Best said. “Can your adversary get malware loaded onto the same compute platform that your LLM is running on? And can they get access to your data?”
The data being used for the training model could be corrupted into misinformation through an injection attack of false data, false positives, false negatives and be manipulated so that system will fail in the field, Best said.
“There are a lot of side channels that can be used to exfiltrate data,” he added, while infiltrating data involves delivering malware through a successful phishing attack. “You usually need layers and layers of authorization to push data into a system, whereas pulling data out of a system is an order of magnitude easier if the system itself is not secured.”
If an LLM is plain text, Best said, and being stored in DRAM that doesn’t have adequate security features, it’s not possible to backport memory security into a 10,000-rack system. “It becomes one of those compromises you make very early on that becomes a permanent compromise.”
Memory makers like Micron Technology are taking a “security by design” approach with its product designs, which includes implementing secure boot processes, establishing a hardware root of trust, encrypting data where applicable, and implementing standardized security features. Its DRAM products are ASIL D certified, the highest level of ISO 26262 functional safety requirements.
GenAI is both a threat vector and a target
Steve Hanna, distinguished engineer at Infineon Technologies, told Fierce Electronics that the net benefit of AI is that it can be used to defend systems, but also be used as a tool by bad actors. GenAI has become a popular threat vector – it’s targeted for theft and for corruption.
“GenAI is itself an active agent,” he said. “We need to keep it safe as we would impressionable child if it's still in training mode.”
Hanna said protecting GenAI from being poisoned means being able to authenticate training data to make sure it’s coming from reliable sources. “Hardware can play a role in that.”
If a massive source of data is coming from IoT, such as images from a doorbell camera to train an AI model for image recognition, all the cameras must be authenticated, Hanna said, and once the model is trained, it must be kept under lock and key because it’s value intellectual property. “You can encrypt that model as it's transferred out of the memory and as it's in transit among different servers within a cloud or server farm.”
Classic cloud security techniques can keep that model secure as it’s being executed, while also doing secure booting, firmware updates and securing communications, Hanna said. “The model execution is going to be depending on other systems. One of the tricks that attackers often use is they'll look for the weakest link.”
Even if the model has been built, stored and transferred securely, the system will be a viable target if it’s not adequately secured, Hanna said. “If they can intercept the communications to the model at any point and modify it in transit, then the results you're getting from that model are going to be unreliable and potentially dangerously tampered with.”
Rambus’ Best said the problem GenAI presents to security is that the market is moving so quickly, and cybersecurity considerations are not being given their due. “Security is very hard to do well at high speed.”
Many companies are moving quickly to achieve GenAI at scale without backporting security. “It could become too late by the time you've achieved scale.”