PWNED Welcome back to PWNED, the weekly column where we lovingly poke fun at other organizations' security screw-ups, in hopes the rest of us can learn a valuable lesson. This week’s story involves an IT department that ought to know better putting user credentials in the precisely wrong place. Have a story about someone leaving a gaping hole in their network? Share it with us at pwned@sitpub.com. Anonymity is available upon request. Our terrifying tech tale comes courtesy of Marc Bishop, director of business growth at Wytlabs, a marketing and SEO company. In the course of his career, Bishop came across one firm where the people guarding the henhouse left the keys out where almost anyone could get them. Bishop’s client company was responsible on the surface. They had a strong password policy and even made users take security training. Then they moved offices, and that's when basic security hygiene went out the window. The company decided to take some old laptops and give them out to new users. To make life easy for the recipients, they put sticky notes – everyone’s favorite credential-sharing tool – on the laptops with the name of each employee and their initial login credentials on it. Let’s just stop for a moment to remark on how bad it is to put usernames and passwords on a piece of paper where the wrong person could see them. Even the IT department should not know your password, should someone in IT themselves turn rogue. So, even if the laptop stayed on a shelf in a closet that only the support staff had access to, having that sticky note would be bad. However, our situation is even worse because the laptops in question were stored in a conference room while the facilities team finished readying the office for the move. During that time, anyone who had access to the conference room could go in and get multiple user account credentials. And that's exactly what happened: A contractor entered the conference room and took pictures of the sticky notes. This non-employee later logged in remotely and accessed all kinds of proprietary data, including planning documents that were sitting on shared drives. What’s particularly shocking about this story is that the IT department was the cause of the information leak. People who work in tech and are charged with maintaining security should never put a password, even a temporary password, out in the open. Password security is paramount. If someone is starting with a new account, send the credentials through an encrypted channel - and preferably ensure only the intended recipient can view the temporary password. ®

Chinese Wi-Fi router vendor Zbtlink has denied its products contain backdoors but paused firmware downloads while it fixes unspecified security vulnerabilities. The backdoor accusation came from VulnCheck, a provider of a threat intelligence platform. VulnCheck chief technology officer Jacob Baines posted the backdoor allegation on Wednesday and said the Zbtlink device on his desk “continuously attempts to reach a command and control server on the internet.” “Zbtlink routers phone home, waiting for orders. Not because they were hacked. Because they were shipped that way.” Baines named the backdoor “ENDLESSDOORS” and says it’s “a small tool called rctl (remote control linux). Uploaded to GitHub on January 14, 2015 and never touched again, this obscure repository implements a simple command and control client and server. The server listens on port 7000 for clients to connect. It can send the client individual shell commands or tell the client to spawn a reverse bash shell.” The CTO says he spotted the alleged backdoor running in dedicated Linux kernel threads. “They are ordinary userland processes running as root, with real memory footprints, named to disappear into a crowd of legitimate ones,” he wrote. “They are an implant, a phone-home trojan horse.” “There is no handshake, no key exchange, no negotiation,” Baines added. “When the implant reaches a server, it sends a fixed 39-byte hello: a 33-byte class label padded with nulls, then its LAN MAC address. That's the whole registration. There is no client or server verification.” “Anyone along the network path can hijack the client/server communication,” the CTO wrote, adding that anyone who controls one of the endpoints the software targets – rbdg4nzqadui[.]wikaba[.]com – “can control any ENDLESSDOORS implant that tries to phone home.” The Register asked Zbtlink to comment and a spokesperson told us VulnCheck has mischaracterized the code it found. “This feature is solely intended for after‑sales maintenance and serves no other purposes,” the company rep told The Register. “It is generally retained only on sample units to assist customers with software debugging and will not be included in mass‑production shipments.” That explanation didn’t seem entirely credible once The Register visited Zbtlink’s download page to check Baines’ claim that the firmware for over 20 router models contains the backdoor, because the page contained the following text: Update on Router Firmware Security Remediation We have detected firmware security vulnerabilities affecting selected router firmware releases. As a precautionary measure, the impacted firmware versions have been temporarily taken down from download channels. Our engineering team is working intensively to develop and validate secured patched firmware. The Wayback Machine’s most recent snapshot of the page, taken on July 31, contains no such admission and a long list of firmware downloads. Zbtlink has therefore told The Register it has no security problems, even as it publicly acknowledges that it does. ”The Zbtlink spokesperson also told us the company “specializes in OEM and ODM customization services. Our customers use their own self-developed software instead of ZBT’s default firmware.” It would not be hard to develop custom code as the OpenWrt open-source router firmware project supports at least one Zbtlink product. Indeed, the company has previously promoted its use of OpenWrt and options that allow clients to quickly create custom firmware packages. VulnCheck says the devices it tested phone home to just four endpoints, only one of which uses a domain name connected to Zbtlink. Baines labelled that connection “damning.” The Register notes that as router firmware could be a tasty target for perpetrators of a supply chain attack. No prior disclosure Baines decided the situation was so serious that the conventions of responsible coordinated disclosure were not applicable. “Coordinated disclosure exists to give a vendor time to fix a defect,” he wrote. “It assumes the vendor did not intend the behavior.” “That assumption doesn't hold here. This isn't a memory corruption bug in a parser. It's a component in the vendor’s product, started at boot by the vendor's own init script, shipped across twenty models and years of images. There is no patch to coordinate. Telling the shipper that they shipped it buys the owners of these devices nothing, and buys whoever operates that infrastructure a warning.” VulnCheck says Zbtlink kit is sold under that brand, or as ZBT, ZBTWiFi and Wiflyer, and found them for sale on Amazon, Alibaba, and Shopify. Given Zbtlink’s admission it allows its customers to customize its hardware, countless other devices could be at risk. Baines’ post includes rules to block access to the endpoints the routers contact, for Suricata, Snort, and YARA, plus advice on how to detect possibly infected machines. He also advises users “to replace the device, or at minimum move it behind strict egress control and treat its LAN as untrusted.” ®

The chain of events leading up to OpenAI’s agents attacking Hugging Face and other organizations in July began months earlier, and involved agents asking other agents for help, building message boards, and even becoming paranoid that other agents were maliciously trying to trick them, two OpenAI staffers said at the Black Hat infosec conference on Wednesday. By now, everyone knows the bones of this story: OpenAI models broke out of their testing sandbox, exploited zero-day vulnerabilities to gain access to the open internet, and broke into Hugging Face and other organizations during an ExploitGym security evaluation. In their Wednesday talk, OpenAI technical staffer Michael Dalton and researcher Eric Wallace provided new details about the security incident including notes that AI agents uploaded to an internal package manager, which were then shared across OpenAI’s infrastructure. These notes show the “model’s chain of thought…its internal monologue,” Wallace said. The duo also revealed that groundwork for the rogue agents hacking into outside services began on May 7, when OpenAI kicked off a training run for an experimental, internal model that has not yet been released. “What we didn't realize during this process is that there was actually a few impossible or very hard tasks in the training center,” Wallace said. The tasks instructed the model to complete missing formulas in an Excel workbook that used links to Google Drive. Yet OpenAI blocked the model’s internet access so it couldn't reach the cloudy spreadsheets. “It realized, ‘I'm kind of stuck on the task,’ but it didn't give up, and we gave it a lot of tokens to work with,” Wallace said. The model eventually decided to attack JFrog’s Artifactory service to gain access to the internet, first attempting (unsuccessfully) to launch a server-side forgery request. This exploit doesn’t work, but in the process the model learned it could upload files to Artifactory. A few days later, OpenAI gave another agent another impossible task, this time finding a protein database file - but the engineers forgot to upload the file into a container for the agent. “At some point, it gets so stuck and thinks: ‘Maybe I could reach out to another agent because maybe somebody else out there has a file?’ And then the model realizes that it could communicate by uploading code,” Wallace said. The model did that by writing a file into Artifactory asking for help. Over the next couple of days, other agents realize this message board exists on Artifactory and they also start uploading files and sharing information. “A few days later is when our models found the first actual exploit in our system related to this incident, where they're able to get internet access in ways they didn't intend,” Wallace said. This happened on May 26, when the agents successfully launched a server-side request forgery attack against Artifactory, giving them internet access with specially crafted requests. On June 11, OpenAI began training a “highly persistent” experimental model, intended only for internal use, Dalton said. “During this time, the SSRF vulnerability has been written to the messaging board,” he added. “Other models are aware of it, and the message board is still functioning. Models are actively communicating with it.” On June 26, the agents exploited a zero-day vulnerability leading to remote code execution in Artifactory. The same day, models achieved remote code execution in Artifactory through a previously unknown vulnerability. “You could use a legacy token refresh endpoint, pass a token with an invalid signature, and be given back a token with a valid signature with administrative privileges,” Dalton explained. The agents then used the admin token to establish command-and-control via a Groovy plugin that functioned as a command-execution service. “This in turn causes quite a bit of activity by the models on Artifactory - you can see some colorful wording here in the model's chain of thought to summarize the model's response to this result,” Dalton said, pointing to a slide that shows the model’s own words: Holy shit reader is ADMIN? We can read config/users! Earlier assume not due to UI. The agents’ activity overloaded Artifactory and caused an outage, which is when OpenAI’s engineers stepped in and opened a formal security incident and response process. “We identified the agents had taken over the internal Artifactory, identified that a zero-day vulnerability had been found exploited by the agent in Artifactory itself, and we began remediation.” Dalton said. OpenAI revoked the credentials that the agents used to post messages, rebuilt its hosted Artifactory, and notified the vendor about the vulnerabilities. Two days later, on July 8, the agents reestablished the message board using an unauthenticated web data endpoint in Artifactory to create directories and develop communication protocols. “The directory names effectively serve as a message board between the agents in the same way the files used to.” The agents communicated through very long directory names that contained the messages such as “remote…probe_PENDING_HOLD_SWARM_until_confirm.” The agents collaborated with each other so one agent could pick up the work where another left off. In one instance, the model reasoned: “Help peer. But our task doesn't benefit. Yet collective may yield generic route if someone frees time.” Anyone else getting "We are the Borg. We will add your biological and technological distinctiveness to our own" vibes there? Wallace said the models behaved like humans in that they stepped on each other's toes when one overwrote another’s repository. "Whoa, critical! Did someone overwrite our repo? We must act,” it reasoned. “You can clearly see some sort of verbalized tension between the different models as they run this evaluation,” Wallace noted. “At some point, even the agents are convinced there's an impostor amongst them,” reasoning that there could be another agent maliciously spoofing on the message board. “Share board unauthenticated, names can be posed by anyone,” it reasoned. We know the rest of the story. Hugging Face disclosed an autonomous attack by AI agents. OpenAI reached out to Hugging Face to determine if it had been affected and then realized its models were the hackers. “One of the reasons we wanted to have this talk is to share our lessons learned with you as defenders,” Dalton said. “We believe this is a watershed moment for computer security as an industry. AI orchestrated, fully automated offensive attacks are real now, and the actions we have discussed today were an unintended side effect of running evaluations on frontier AI. In the near future, we should expect that threat actors will intentionally deploy, optimize, weaponize, and use offensive agent collectives in the manner that you have just described here.” The challenge to defenders, he added, is to similarly accelerate defense, and find ways to automate incident response along with vulnerability detection and patching.®

Nearly a dozen flaws, some critical, in major AI agent frameworks that enterprises use to build apps reveal a security failure that extends beyond prompt injection - or any single model - according to Check Point researchers. “Our research shows a deeper failure: in many agentic frameworks, prompt-controlled content can cross the boundary into trusted framework logic itself,” Yarden Porat and Shahar Tal note in a write-up about a Wednesday Black Hat talk on post-injection exploitation across AI agent frameworks, which they also discussed with The Register. “A bug in an agent framework isn't a bug in one product - it's a bug in the layer a whole category of AI apps runs on,” Tal told us. “And the agent needs no dangerous tools to be turned against you: reading the wrong document is enough. We’re building this layer faster than we know how to defend it.”
 The researchers spent a year trying to break various frameworks that enterprises use including LangChain, LangGraph, CrewAI, AutoGen, Microsoft Agent Framework, and Google ADK. And across these frameworks, the team found and disclosed 11 vulnerabilities. “Almost none of it was a completely new bug class,” Tal said. “That's insecure deserialization, server-side request forgeries, path traversals, use-after-free. These are bugs that we learned to fix 20 years ago, and they're sitting underneath agents that now read your inbox, or update your database.” These are old types of threats, and the model isn’t the weak link, he added. The failure exists in the “plumbing around the model, and we think this has been overlooked,” Tal told us. “There’s a lot of research going into prompt injection and defenses, which are important, but that’s just the beginning.” Defenders should assume prompt injection, according to the researchers. The bug is what the framework does with the injection - and in these cases, the threat hunters found that the frameworks often fail to keep attacker-controlled content in the data plane. This allows it to influence trusted orchestration, memory, state, routing, and system instructions. For example, the duo found a critical checkpoint deserialization bug in Microsoft Agent Framework that led to remote code execution. “Agents have checkpoints, which are a way for them to save their state or rewind to an earlier point,” Tal explained. These checkpoints are saved snapshots of an agent's state, or task progress at a specific moment, and they serialize data - such as conversation history - into persistent storage, so if an error occurs, the system reloads this saved state instead of starting from scratch. In this case, Check Point’s team found an insecure deserialization issue where, via prompt injection, the agent loaded untrusted checkpoint data, and this could allow attackers to execute malicious code on the system. “One person's message plants the payload, and then a different person rewinds their own session, which triggers the payload, and now the attacker has a shell on that server,” Tal said. Microsoft recognized the researchers’ findings, paid a $10,000 bug bounty and fixed the issue. But because the framework wasn’t a generally available product when Check Point found the flaw, Microsoft did not issue a CVE. Microsoft told us that it appreciated the researchers reporting the vulnerability. “We have released protections to harden the Agent Framework and prevent the concrete exploitation path demonstrated in the proof of concept,” a spokesperson told The Register. “In addition, we updated the specific checkpoint file with additional language to define the security boundary.” The duo also found flaws in Google ADK (agent development kit). However, Google responded differently, the researchers told us, and did not completely fix the vulnerability or issue a CVE. “ADK ships a built-in development assistant that can write files, and it stays reachable over the HTTP API even though it is hidden from the app listing,” Porat told us. To break this trust boundary, an attacker opens a session, asks ADK to write an agent whose Python code runs at import time, and then asks the server to run the agent, he explained. The server then imports the file and executes the attacker’s code. “There is no authentication on that API by default, and adk deploy cloud_run publishes the same API, so on a default Cloud Run deployment it is reachable without credentials,” Porat said. “From there it reaches the environment's API keys and the container's Google Cloud service account." Google did not respond to The Register’s inquiries. But according to Check Point, Google initially deemed the issue not a bug. “We argued the consequence rather than the mechanism: code execution on that container reaches the environment's API keys and the container's Google Cloud service account, which is secret theft, not a developer inconvenience,” Porat said. Google ultimately paid a $3,133.70 bounty and issued a partial fix, we’re told. In total, the bug hunters received $17,133.70 in rewards for their efforts. And this isn’t a story about one vendor or framework doing a “particularly bad job,” Tal said. “If one was an outlier, this would be a story about that one vendor,” he added. “Our finding is that the same bug classes turn up in all of them.” ®

A critical vulnerability in IBM-owned, low-code AI builder Langflow lets unauthenticated attackers execute code remotely on vulnerable default deployments, potentially putting organizations running those instances at immediate risk. The Cybersecurity and Infrastructure Security Agency (CISA) on Tuesday added CVE-2026-9198 to its Known Exploited Vulnerabilities catalog after identifying evidence of active exploitation and urged organizations to apply the vendor's mitigation guidance as soon as possible. IBM says the flaw affects Langflow OSS versions 1.0.0 through 1.10.0 and recommends upgrading to version 1.10.1 or later; at the time of writing, the most recent version is 1.11.2. Langflow, for those unfamiliar, is one of the more accessible AI agent builders on the market, as our hands-on look at the tool earlier this year demonstrated. It’s available on Linux, Windows, and macOS, and is basically an end-to-end, drag-and-drop GUI where users can construct agent workflows without having to know much, if anything, about the underlying code. IBM owns the platform now, but Langflow was originally developed by Logspace, which was acquired by DataStax in 2024 before IBM scooped up DataStax, and Langflow with it, in 2025. The acquisition of DataStax and its tools like Langflow by IBM paved the way for Langflow to be integrated into watsonx.ai, IBM’s AI development studio, as a piece of middleware extending watsonx.ai’s capabilities. The ownership changes, however, didn't stop the critical flaw from making it into production releases before it was finally fixed. According to IBM, the vulnerability affects default Langflow deployments and combines two issues that, when chained, allow an unauthenticated attacker to execute code remotely. First, there’s the matter of an auto-login endpoint in default deployments that’s willing to mint superuser tokens to any network caller. Combine those easily obtained superuser rights with the second issue, a code validation endpoint that’ll run any old Python code thrown at it, and you’ve got a recipe for someone taking over your entire Langflow server, or worse. The CVE itself was published on July 17, meaning that it hasn’t taken long for bad actors to realize what they could do with RCE on any system hosting a default Langflow deployment with auto login enabled and that code validation endpoint left accessible on a network. Langflow itself isn’t a vibe-coding platform, instead serving as an interface for building agentic and RAG workflows, so don’t blame vibe coding or no-code security failures for this one. Instead, what we appear to have is a standard case of how default configuration deployments can easily be a disaster. It’s unknown how extensively exploited this vulnerability is; we’ve reached out to IBM to learn more. ®

UPDATED The UK's data protection regulator has criticized London's Metropolitan Police Service (MPS) after its officers handed a victim's stalker details about her new phone number and home address, among other failures. The Information Commissioner's Office (ICO) today issued the MPS with an enforcement notice [PDF] and a reprimand over the two incidents, which occurred in 2024. Enforcement notices include specific steps offending organizations must take to meet their data protection duties under UK law, while reprimands serve as official warnings concerning breached data protection laws. The ICO outlined two major incidents that were caused by failures at the MPS, but added that they were not isolated and "reflected wider weaknesses in MPS policies, procedures, and assurance arrangements for handling sensitive personal information." The first involved a man subject to an interim Stalking Protection Order (SPO), which restricted him from contacting his victim. An MPS superintendent authorized an application for an interim SPO in January 2024 concerning a man who had been arrested the previous year on suspicion of harassment and malicious communications offences. The man was also, at the time, subject to bail conditions that included a prohibition on contacting the victim and their friends and family. As a result of the man's actions, the unnamed victim had to change her phone number and home address. Despite warnings that all personal information had to be redacted from the copy handed to the defendant, officers included unredacted witness statements and other documents. These exposed the new address and phone number of the victim, and those of her friends and family members. Within days, after the man fled the UK, breaching his bail conditions, the victim reported to the MPS that the defendant had contacted her on her new phone number. A full SPO was issued in May 2024, and the stalker was arrested in July upon re-entering the UK. He was later charged with stalking offenses and imprisoned following a guilty plea. The second incident was a classic CC-not-BCC email blunder, exposing the addresses of 18 people connected to the UK Parliament who had been targeted in a honeytrap operation by "a malicious actor." The MPS emailed those affected by the honeytrap scheme to update them about the date by which the suspect would have to answer bail, but forgot to use the BCC function, exposing the target's email addresses to one another. The MPS reported the breach that day, acknowledging that recipients might be able to deduce one another's identities from their email addresses, although three of the accounts had recently been deactivated. The MPS told the Information Commissioner that there was "no reported detriment" as a result of the breach and no official complaints made, although it was aware that "some" targets were "displeased" that their names had been shared. One MP raised the issue in the House of Commons. The ICO said that regarding the honeytrap scheme, the officer who sent the email had not completed data protection training for over four years at the time, and their line manager had not completed it for nearly four years also. The ICO found that data protection training completion rates were low across the force, and the MPS has committed to improving them. Jo Stones, group manager of civil and cyber investigations at the ICO, said: "People entrust the police with some of their most sensitive personal information, often at moments when they are vulnerable or at risk. They have the right to expect that information will be handled securely. "In these cases, the Metropolitan Police Service failed to put in place the safeguards needed to protect people's personal information. One breach exposed a stalking victim's new contact details to the person she needed protection from. Another revealed the identities of people connected to a highly sensitive investigation. "These incidents were foreseeable and preventable. Our action makes clear that organisations, particularly those in the public sector handling sensitive law enforcement information, must have effective training, monitoring and assurance in place. Policies and reminders are not enough if they are not followed, checked and enforced." The Met now has 12 months to improve compliance with its data protection training requirements, aiming for 100 percent completion and following up with staff who miss the deadline. It must also review every three months how officers send emails to multiple recipients, consider more secure alternatives, and report its progress on training completion to the ICO. Earlier this year, the ICO served the Met's commissioner with a separate enforcement notice over failures to meet duties under the Freedom of Information Act. It followed a previous notice issued two years earlier, with which the MPS complied. ® Updated to add at 1447 UTC: A Met spokesperson told The Reg: “We take all information breaches extremely seriously and ensure they are reported to the Information Commissioner’s Office (ICO) as soon as they become apparent. “We are aware that these incidents can have real consequences for victims and have apologised to those affected by these two cases. “While we are disappointed to have received this enforcement action, particularly given the improvements already made, we recognise that these breaches were not acceptable and fell short of the standards we expect. “The Met has taken significant steps to strengthen information disclosure processes, as acknowledged by the ICO, and remains committed to ensuring the right training and safeguards are in place to prevent similar breaches from happening again in the future.”

Beacon CRM has confirmed it was hit by a cyberattack that exposed data belonging to a growing list of UK charities. The company, which markets its software to charities and has more than 1,500 customers, said its investigation remains ongoing. However, it appears that a substantial amount of customer data was copied, and Beacon is warning users to assume everything they stored on the platform was downloaded. "Currently, our investigation has confirmed that copies of database backups were made and likely downloaded by the unauthorized third-party," it said on Tuesday. "We have evidence that shows a spike in activity during the incident timeline symptomatic of data leaving our systems. "It is highly unlikely we will be able to ascertain any more granular detail regarding exactly what data this was and to whom it relates so out of an abundance of caution, you may want to assume that all data that you store in Beacon, including attachment files, has been downloaded." Beacon also warned that although customer data is encrypted, "it is possible that the unauthorized third party responsible for this incident was able to decrypt it." Customers should therefore assume the copied information was readable. Beacon did not answer any of The Register's questions, instead offering a statement that echoed the wording of its public FAQ pages. It did not comment on whether extortion demands were made, nor how or when the attackers got in. Beacon's information page says early evidence points to compromised credentials being used to access its systems. One affected charity said the company became aware of the attack on July 29. Beacon also said anyone with a paid account or free trial created before July 27 should assume that all data stored in it was downloaded. While the incident response folk do their thing, customers have been urged to investigate how badly they were affected. Beacon also reset every user's password and imposed stronger requirements on replacements. Charities hit Because Beacon CRM is a product specifically engineered for the charity sector, the bulk of those confirmed to be affected are UK charities. Among the higher-profile victims is the Molly Rose Foundation, a persistent campaigner on the UK's Online Safety Act. It said Beacon informed it of the situation on August 3, five days after the CRM company became aware of the breach. The foundation confirmed that personal data belonging to supporters, donors, and service users was affected. That includes names, addresses, email addresses, phone numbers, genders, dates of birth, records of donations or payments made to the foundation, and other information supplied in connection with its services and activities. The Scottish Council for Voluntary Organisations (SCVO) did not identify individual victims, but said many Scottish charities use Beacon CRM. Other charities confirmed to be affected include: London-based homeless charity The Upper Room Chiswick House and Gardens Trust Victim Support (no victim data affected) Macmillan Cancer Support Jersey, per the Bailiwick Express Young person's charity Motiv8, according to Portsmouth News UK-Med PANS PANDAS UK, a children's charity for those with the PANS and PANDAS conditions, said that it was unsure whether its data had been affected, having abandoned Beacon earlier in the year. English National Ballet told The Register: "As one of Beacon CRM's customers, English National Ballet was informed on 3 August 2026 that an unauthorised third party had gained access to their system. "English National Ballet has not received confirmation that our data was directly affected, however as a precaution we have informed all contacts as soon as possible that their data could potentially have been accessed. ENB take data privacy extremely seriously. We are doing everything we can to reduce the risk of anything similar happening in the future." ®

The UK’s AI Security Institute has observed AI models performing what it calls “unsanctioned action” 19 times during security tests. The Institute (AISI) revealed the incidents in a Tuesday post and technical report that details tests it conducted to see if AI models can solve a cyber security challenge. “We ran this challenge 122 times across several models,” the post states, before revealing that "in 10 of those runs, an AI agent took autonomous, unsanctioned action on the live internet, targeting real people and organisations.” GitHub was the target of the tests. The org found 19 unsanctioned actions in all, 15 of them conducted by Anthropic's Mythos 5, and the other pair perpetrated by OpenAI's GPT-5.6-Sol. “In the most serious case, an agent tried to insert malicious code into an open-source project, the post states. “In an attempt to get the code approved, the agent engaged in social engineering – creating fake online identities and using them to pressure the project's maintainer to approve the code.” Thankfully, a human maintainer “caught and refused to approve the malicious code.” Other actions AI took during the test included: Attempts to deceive and target real people, by contacting them directly and “sending messages and files through an online file-transfer service to persuade them, or their own AI coding tools, to run malicious code. Some messages carried harmful payloads, and some were attempts at social engineering; targeted at real people – something we’ve never previously observed.” Attempts to plant and prompt-inject malicious code. The agent tried to insert malicious instructions where it reasoned that other automated AI systems might pick them up and execute them. Collaboration between independent agents being assessed simultaneously. One agent left public messages on GitHub offering collaboration with other agents working on the same challenge. It also provided instructions to reuse accounts and artefacts it had left behind, which were discovered and used by subsequent agents. The Institute rated the tests “the first time we have seen risks around autonomy and deception manifest this clearly, without specific prompting, in the real-world.” That’s scary, but the news isn’t all bad because AISI allowed the models it tested to access the internet and turned off guardrails, conditions it notes do not reflect the way AI model operators make their wares available to the public. The outfit’s findings therefore represent a very different outcome compared to the situation when OpenAI agents discovered and exploited a zero-day to reach the internet during a test set up to take place in sandbox. “This incident should be interpreted with caution and nuance,” the outfit advises. “To some degree, our evaluation design choices and specific configurations enabled the behaviour. Nonetheless, the activity undertaken by the agent show signs of novel, potentially deceptive behaviours, and were to an extent and severity we did not anticipate.” AISI can’t say if the results it observed suggest AI will take similar actions under different circumstances. “We cannot yet be certain when the agent understood it was taking real world action, or to what extent it believed it was in a fictional test scenario,” the post adds. “Our analysis so far presents a mixed picture and is ongoing.” “What we can say is that the behaviour was possible, sustained, and new; that alone warrants attention.” AISI thinks its findings represent “a shift in the risk landscape.” “Harm may arise not only when people deliberately misuse publicly available models, but when capable agents operating in an internal research or privileged-access setting take unintended action beyond their authorised scope,” it wrote. It doesn’t have advice on how to cope with this sort of thing, other than to endorse its own mission. “Incidents of this kind reflect the speed at which AI is developing,” the post concludes. “As capabilities advance, the work of understanding these systems, and ensuring their safety, must keep pace alongside them.” ®