AI Security Hands-On: Understanding and Red Teaming the LLM as a Black Box

Chatbots and Agents may seem like magic, but they become far less intimidating when you realize the Large Language Model at its core is basically a stateless, turn-based black box which receives input and produces an output with nothing remembered in between. In this next real training for free session, I will demonstrate that to you by stripping away the chatbot interface in front and show you the actual APIs used by chatbots and agents to communicate with the LLM.  Hands-on and live.

Understanding raw LLM APIs is easier than you might expect, and no other way approaches the insight you’ll gain from seeing the actual, stateless interaction with an LLM.  Insight into the fundamental components of AI is as crucial to cyber security pros as understanding packets, ports and protocols in network security.

First step in the webinar will be to give you a model for conceptualizing how LLMs work.  If you are like me, you don’t have the higher math and data science or neural networks creds to get deep into the innerworkings of LLMs.  Instead, most of the time it’s better to treat the LLM itself as a black box and focus on technologies evolving around LLMs that actually provide the higher level behavior you experience with chatbots and agents like:

• Message pipeline
• guard rails
• harnesses
• retrieval augmented generation
• model-context-protocol
• skills

Things we’ll dive into:

• System prompts vs User prompts. I’ll show you via the actual API how system prompts are specified by the application developer vs user prompts from the end-user and we’ll demonstrate how effective higher authority system prompts are at overriding what end-users prompt.
• Context window: How LLMs seem to “remember” things you said several messages ago in a long conversation
• Longer term memory: how are chatbots and agents beginning to remember stuff you talked about in different conversations from even months ago?

One area where we will peak inside the LLM black box is with regard to tokens and tokenization since tokens are so fundamental to AI built on LLM both from understanding and dealing with the economic aspects of AI technology. 

Finally, we’ll build on this foundation and get into security by looking at one of the most effective ways to find security risks in AI deployments – AI Security Red Teaming.

My sponsor is A10 Networks who recently acquired TrojAI, a leader in AI security testing and red teaming.  Stan Petley, who is the founding engineer and AI security specialist from TrojAI, will show you the types of vulnerabilities organizations are finding in AI production environments and how security teams can identify and mitigate these risks before they become incidents.

Please join us for this real training for free session.