In the past, we’ve created a list of our favorite talks and workshops at the event. This year we’re changing it up a bit. It was such a packed event with so many amazing sessions that it'd be a disservice to even attempt to create a Top 5. Instead, we decided to focus on our most memorable experiences of DEF CON 33, focusing on three themes:

#1. AI security: LLMs cannot be fully secured yet

No surprise, AI in security once again had a strong line-up of sessions. Some showed how to use AI for red teams and blue teams, while others focused on attacking LLMs. 

One such talk was: "Illuminating the Dark Corners of AI: Extracting Private Data from AI Models and Vector Embedding". The title describes it well. The speaker, Patrick Walsh, reminded us that LLMs are built around statistical models, which means persistence matters when attacking GenAI. It was shocking to see how effectively model inversion attacks extracted data out of the hidden layers of fine-tuned models.

A few hours later, in the Red Team Village, Itsik Mantin expanded on these ideas as he dove into automation for red teams that want to attack LLMs. Once again this is a fairly unexplored area that showed how easy it still is to convince these novel systems to behave in unintended ways. Itsik put a strong emphasis on LLM application security, which is covered by 6 of the OWASP Top 10 for LLM apps. Particularly interesting was the optimization of restrictions on models, which can both prevent attacks, but can also cause leaks when these guardrails grow too much for small models.

All of these talks had two things in common: LLMs aren't predictable - which is half the fun and rewards persistence - and they require a lot of automation to be secured well. It is becoming more important to chain components together effectively, especially with the rise of agentic solutions.

#2. Network security: policy violations are still a problem

This year's networking talks were once again some of the most fun deep dives of the event, especially if you’ve worked in this area for a while. 

You would be right to assume that most of these technologies have existed for long enough that we’ve discovered everything we need to. That is until you step into "HTTP/1.1 Must Die!" by James "albinowax" Kettle and realize how much there’s still to be discovered and how many surprising places still use this protocol. People mistakenly think HTTP/1 is simple. Simple means secure. It's been around for a long time. That should be secure. All of these are faulty assumptions, which the speaker demonstrated with bounties as proof. To learn more, check out [http1mustdie.com](https://http1mustdie.com/).

Another fun networking talk was "From Spoofing to Tunneling: New Red Team's Networking Techniques for Initial Access and Evasion". If you ever see a public IP trying to connect to a system in your private network, you may want to check out this talk. Many of the techniques may appear well-known at first, like spoofing your MAC and your IP addresses. But the way the speaker Shu-Hao Tung "123ojp" combined them with components like VXLANs and GREs led to some very creative ways to infiltrate a target.

Ultimately these talks all had something in common: They were based on misconfigurations and ultimately policy violations in these environments. They showed how even well-known components can accidentally be set up in a way that exposes you to attacks. It's not just about creating a secure surface, but about configuring every layer in the chain and hardening it.

#3. Supply chain attacks: remediation in focus

The last group of talks was focused on the supply chain. During "Breaking the Chain: Advanced Offensive Strategies in the Software Supply Chain" the speakers Roni "lupin" Carta and Adnan Khan demonstrated how attacks against well-known software components were successfully executed via dependency confusion and npx confusion. The latter is an attack where you claim the unused package on npm and have a victim run your code with the use of npx. 

In "Edge of Tomorrow: Foiling Large Supply Chain Attacks By Taking 5k Abandoned S3 Buckets from Malware and Benign Software" we were once again reminded that buckets can be re-allocated once they expire and can be used for nefarious purposes. It happens so quickly: You copy a link into an install script, which points to a static S3 bucket, and don't think about it for a few years, until someone takes it over.

Finally, we take a wider perspective and include the amazing "Patching Critical Infrastructure, Announcing the Winners of DARPA’s AI Cyber Challenge" talk in this list as well. The contest announced its winners this year, which had demonstrated how to use AI to not only find vulnerabilities in critical open-source projects, but to patch them as well. In fact, patches were worth more than findings. We congratulate all 7 finalists and the winners of this contest!

This aligns so well with our own perspective on the security industry as a whole. We believe over the years a lot of focus has been put on detection, but too little attention was put on prioritization and fixing findings. This is finally changing. Remediations are worth more than just findings on their own and if you can effectively automate this process you will be in a much better position to defend yourself against the new reality of highly automated and autonomous attacks.

Honorable mentions

There were simply too many talks to give all of them the attention they truly deserve. Once again, DEF CON was stacked. However, here are a few more that stood out to us:

• "EntraGoat - A Deliberately Vulnerable Entra ID Environment" by Tomer Nahum and Jonathan Elkabas demonstrated an attack environment in EntraID that everyone can use now, with varying difficulties for beginners and advanced users.
• "Shaking Out Shells with SSHamble" by HD Moore once again shook SSH and the various exotic implementations we can find in the wild.
• "Safe Harbor or Hostile Waters: Unveiling the Hidden Perils of the TorchScript Engine in PyTorch" by Ji'an "azraelxuemo" Zhou and Lishuo "ret2ddme" Song was an eye-opening dive into command execution in this widely used Python ML library.

There are so many more we didn't get to see and we can't wait to see the videos when they come online.

Conclusion

This year's DEF CON continued to demonstrate hacks and jailbreaks against LLMs while also encouraging us to use these solutions for new ways to automate the discovery and patching of vulnerabilities. As we see every year, there were countless examples of attacks that could have been prevented with better system configuration and hardening.

At Mondoo we’re focused on delivering agentic vulnerability management that not only tells you where your vulnerabilities are, but actually helps you fix them as fast as possible. This decreases the remediation gap, improves your security posture, and reduces manual work - and should hopefully keep you from ending up on one of the DEF CON slides!