What Zero Day Attacks Are In Generative AI Models

 

If you are in Cybersecurity, one of the new pieces of techno jargon that you will often hear about is a “Zero Day Attack”.  I have heard about it numerous times, especially when I did the auto upgrades to my Windows machines.  But to be honest, this is the first time I have written about it.  So, if you are like me when I was a few months ago, wondering what it is was all about, here is a technical definition of it:

“A zero-day (or 0-day) vulnerability is a security risk in a piece of software that is not known about, and the vendor is not aware of. A zero- ay exploit is the method an attacker uses to access the vulnerable system. These are severe security threats with high success rates as businesses do not have defenses in place to detect or prevent them.

A zero-day attack is so-called because it occurs before the target is aware that the vulnerability exists. The attacker releases malware before the developer or vendor has had the opportunity to create a patch to fix the vulnerability.”

(SOURCE:  What is a Zero Day Attack? | Fortinet)

Let us break this definition down into its components:

Vulnerability:  A gap, or weakness that exists in a software application.

Exploitation:  The Cyberattacker discovers this weakness and takes advantage of it by deploying it into a malicious payload.

Attack:  This is where the Cyberattacker attempts to do some damage, such as Data Exfiltration.

As it relates to Zero Day, it is a hole that exists that nobody, not even the vendor knows about. The Cyberattacker discovers this just by pure chance, or through some covert intel.  Because it is not known, they can then exploit this weakness without anybody noticing, and from there, launch the attack. 

The key point here is that though this process, a Zero Day Attack can be very devastating, because it takes everybody by surprise.  When the damage is done, it is then too late to fully recover it.  But now, with Generative AI exploding on the scene and its subsets, especially that of Machine Learning, Zero Day Attacks are now becoming much more pronounced.

One of the primary reasons for this is that the models are constantly evolving and becoming more dynamic by nature.  Even if the CISO and the IT Security team were to discover any gaps or weaknesses and remediate them, the chances of new ones coming out the next day are very high.  Add to this the fact that these models also increase the attack surface, which makes it even more complex to get a true gauge of the Cyber Threat Landscape.

Here are some examples of Zero Day attacks as it relates to the models of Generative AI:

1)     Prompt Injection:

This can be technically defined as:

“Prompt injection is the use of specially crafted input to bypass security controls within a Large Language Model (LLM), the type of algorithm that powers most modern generative AI tools and services.”

(SOURCE:  What Is Prompt Injection, and How Can You Stop It? - Aqua)

               To make this definition clearer, let us backtrack a little bit.  Suppose you use ChatGPT for daily               job         tasks, and one day you have been asked to visit a customer on site.  True, you could use         Google Maps for this, but you want noticeably clear and concise directions on how to get there.  You simply enter your query into ChatGPT, and it gives you various routes you can choose    from.  But in order to get the specific answer you are looking for; you must create the query   with specific keywords.  These are also technically called “Prompts”.  In fact, this has given              birth to an entirely  new field called “Prompt Engineering”.  But as it relates to a Zero Day         Attack with a Generative AI model, a Cyberattacker can very easily hijack your ChatGPT session,            and insert their own prompts.  The end result is that you are given a set of directions, which   although will get you to   the client site, will take you in a far more convoluted manner than what you had intended.  The   consequences of this kind of Zero Day Attack is far more dangerous if       you ask ChatGPT to automatically log into your financial portals (such as          your credit card or bank      account), and ask, or “prompt” it to give you advice on how you should manage your money.

2)     Training Data:

As I have analogized before, a Generative AI model is like a car.  Like this needs fuel to drive, the model needs data (and lots and lots of it) to propel the queries or the “prompts” into giving you the right answers (also known as the “Outputs”).  But you simply cannot dump all kinds of data into the model.  First, you need to make sure that whatever you feed into it is relevant.  For example, if you have developed a model to predict prices for certain stocks, you need to pump in those datasets that belong to them.  Not those of other stocks.  Second, you need to make sure that the data you feed into the model are as optimized and cleansed as much as possible.  This simply means that there are no outliers that exist in the dataset.  If you do not do this, your results will be highly skewed, in the negative direction.  In this regard, it is quite possible that the Cyberattacker can find a hole in the model as it is being developed.  From there, they can then exploit by inserting fake datasets (also known as “Synthetic Data”), into it.  Thus,  once the model is formally launched into the production environment, it can wreak havoc to your business like nobody has seen before.

My Thoughts on This:

Apart from the dynamic nature of Generative AI models as mentioned before, it is very often typically the case, that the time to market of them takes more precedence than developing the secure design of them.  Also, the AI scientists who create these models have security far from their mindset, because they are simply not trained in this area. 

Thus, to help mitigate the risks of Zero Day Attacks from happening, there is now a new movement that is happening now in Corporate America.  This is the adoption of what is known as an “MLSecOps” team.  This is where the AI scientists work in tandem with the IT Security Team and Operations Team to ensure that security model design starts from the very beginning and receives top priority after the model has been launched and deployed for public use.

An important concept here is also the “MLBOM”, which is an acronym that stands for the “Machine Learning Bill Of Materials”. This will be examined in closer detail in a future blog.