AI and Security

It is no secret that ChatGPT and the rise of LLMs have sparked our imagination these past months. DefCon was no different. The line in front of AI Village looked like it was giving the Merch room a run for its money, and for a good reason. There is a wealth of new mechanisms to try out, especially when it comes to analyzing or even building malware.

Across all these exciting tools and new initiatives, one talk that caught our eye: "Shall we play a game? Just because a Large Language Model speaks like a human, doesn’t mean it can reason like one.” Funny enough, this title summarizes itself pretty well.

We evolved to assume that anyone who can articulate themselves fluently is also reasoning. Thus, many LLMs that appear to be reasoning as well. However, that is not true. In fact, developers build many of these tools to predict the next words in a way that is fluent and well-articulated.

That does not mean it is reasoning, even if it appears to us this way. As a reader, you have to become more active.

We build a lot of AI, including LLMs, around the data that we use to train it. If we feed it bad data, we get bad results. This talk was great at highlighting how important the often-boring tasks are that lead to success in AI: good data pipelines and good labeling. Turns out that the models we establish in cnquery are a perfect fit for this approach.

Pretender Defender

One of my favorite talks, however, was one outside the domain of AI. Despite being a bit more technical in its analysis, "Defender-Pretender: When Windows Defender Updates Become a Security Risk" was an amazing deep-dive into the widely used Windows Defender and the things that can happen when you mess with it.

As it turns out, there was a way to inject updates into the things that Windows Defender considered security risks. As we will see later, encryption is hard to get right, and with a few moving parts, there are ways to inject patterns into Windows Defender that it will assume are coming from its trusted owners. This is great when you want to convince the OS to install your favorite “security tool” on the system, in order to e.g., read credentials from memory or take control of the box.

The part that took this talk to another level was the final demo the team gave. In typical DefCon fashion, they taught Windows Defender to behave in a way entirely contradictory to its original purpose. Defender is great at removing any threat to the system faster than you can hit the abort button.

Imagine you teach Windows Defender that files containing "This program cannot be run in DOS mode" are dangerous. The result? The system automatically removes a ton of system files and drivers that make up the heart of the operating system. As the OS eats itself, the demo ends in a crashed loading screen. Glorious.

Private Keys in Public Places

The final talk I want to highlight is both simple and powerful at the same time. Unlike complex AI or complicated hacks, this talk was shocking because of how widespread these simple problems are.

During "Private Keys in Public Places," we saw 3 demos of very common products in IT environments (e.g., a firewall) that had private keys embedded in them. This means that anyone possessing the private key could pretend to be the author of whatever functionality they used.

Usually, private keys are sacred to use. We protect them. However, developers embedded these unassuming products, sometimes hidden or ignored, in very accessible parts of the software. This is not only dangerous because it allows for spoofing, but it is often also harder for vendors to fix because it is an architectural issue.

This rings back to the old security rule: The chain breaks at its weakest link. If developers don't know this is happening, they are more likely than anyone else to create it.

This is why we care about making security more accessible to developers and platform engineers. To truly move the security landscape forward, we cannot just rely on pretty dashboards. We have to integrate security for developers and connect it from code to production. DefCon was a great reminder of this.

Want to Explore More?

If you're inspired by what was shared at DefCon and want to take your security measures to the next level, we've got something special for you. Get full access to Mondoo, a free consultation with our security experts, and all features in the Enterprise edition completely free for 30 days. No strings attached, just a commitment to enhancing your cybersecurity posture. Start your free trial today.

‍