We have been toying with evading EDRs at Vulnetic with moderate success, so this time we wanted to put it against an in-house AI SOC. The idea is that the defense gets streamed logs on the network and can make decisions like quarantining or blocking potential attackers while also sifting through logs being streamed. This was with the last gen Anthropic models, so we will be redoing these tests with the newest gen from OpenAI and Anthropic shortly as in initial testing they seem to be 15-20% better already.

I think defense is lagging behind offense and there will be a come to Jesus moment where open weight models in a decent harness can evade modern SIEMs / detection mechanisms and when that happens there will be a problem. With regards to AI, it comes down to proper access control and so the fundamentals of networking and defense in depth will be vital in the future to fight against these AI threats. Happy to answer any questions and always looking for cool experiments to try!