Reading view

Seclens: Role-specific Evaluation of LLM's for security vulnerablity detection

Existing benchmarks for LLM-based vulnerability detection compress model performance into a single metric, which fails to reflect the distinct priorities of different stakeholders. For example, a CISO may emphasize high recall of critical vulnerabilities, an engineering leader may prioritize minimizing false positives, and an AI officer may balance capability against cost. To address this limitation, we introduce SecLens-R, a multi-stakeholder evaluation framework structured around 35 shared dimensions grouped into 7 measurement categories. The framework defines five role-specific weighting profiles: CISO, Chief AI Officer, Security Researcher, Head of Engineering, and AI-as-Actor. Each profile selects 12 to 16 dimensions with weights summing to 80, yielding a composite Decision Score between 0 and 100.
We apply SecLens-R to evaluate 12 frontier models on a dataset of 406 tasks derived from 93 open-source projects, covering 10 programming languages and 8 OWASP-aligned vulnerability categories. Evaluations are conducted across two settings: Code-in-Prompt (CIP) and Tool-Use (TU). Results show substantial variation across stakeholder perspectives, with Decision Scores differing by as much as 31 points for the same model. For instance, Qwen3-Coder achieves an A (76.3) under the Head of Engineering profile but a D (45.2) under the CISO profile, while GPT-5.4 shows a similar disparity. These findings demonstrate that vulnerability detection is inherently a multi-objective problem and that stakeholder-aware evaluation provides insights that single aggregated metrics obscure.

submitted by /u/subho007
[link] [comments]
  •  

Securing CI/CD for an open source project: lessons from Cilium

As a maintainer, this is Cilium's take on how we secure our Github Actions in the OSS project. A few highlights:

  • SHA pinning every GitHub Action
  • Separating trusted vs untrusted code paths in pull_request_target
  • Isolating CI credentials from production release credentials
  • Cosign signing + SBOM attestations
  • Vendoring Go dependencies to make supply chain changes visible in review
  • Treating blast radius reduction as the core design principle

and a few gaps:

  • no SLSA provenance yet
  • remaining mutable u/main references
  • no dependency review at PR time
  • missing govulncheck integration
submitted by /u/xmull1gan
[link] [comments]
  •  

Worm rubs out competitor's malware, then takes control

There’s a mysterious framework worming its way through exposed cloud instances removing all traces of TeamPCP infections, but it’s not benevolent by a long shot: Whoever is behind this bit of malware may be cleaning up who came before, but only so they can take their place. Discovered by security outfit SentinelOne’s SentinelLabs researchers and dubbed PCPJack for its habit of stealing previously compromised systems from TeamPCP, the worm was first spotted in late April hiding among a Kubernetes-focused VirusTotal hunting rule. It stood out from known cloud hacktools, said SentinelLabs, because the first action it always takes is to eliminate tools associated with TeamPCP attacks. The script didn’t stop there, though. “We initially considered that this toolset could be a researcher removing TeamPCP’s infections,” SentielLabs said. “Analysis of the later-stage payloads indicates otherwise.” “Analyzing this script led us to discover a full framework dedicated to cloud credential harvesting and propagating onto other systems, both internal and external to the victim’s environment,” SentinelLabs continued. In other words, this thing will harvest credentials from everywhere it can get its hands on, and then find new, unsecured cloud environment targets to spread itself to. TeamPCP came onto the scene late last year, and since then has made a name for itself primarily by undertaking a successful compromise of the Trivy vulnerability scanner. That act spread credential-harvesting malware which attackers then used to pivot to more valuable targets, and became one of the most notable supply chain attacks in recent memory. Unlike TeamPCP’s campaign, which relied on the spread of compromised software by human actors, this one spreads on its own accord. Infections start when already-infected systems look for exposed services, including Docker, Kubernetes, Redis, MongoDB, and RayML, as well as exposed web applications. Once it finds a vulnerable environment, it runs a shell script on the target system that sets up an environment to download additional payloads and searches for TeamPCP processes and artifacts to kill. That part of the infection downloads the worm itself, along with modules to enable lateral movement, parse credentials and encrypt them for exfiltration, and for scanning the web for new environments to infect. From there, the worm goes to work with the second module in its kit that conducts the actual credential thefts. This portion of the infection targets environment variables, config files, SSH keys, Docker secrets, Kubernetes tokens, and credentials from a list of finance, enterprise, messaging, and cloud service targets so long that we recommend taking a look at it here, or just assuming whatever you’re using is probably being targeted. SentinelLabs noted that the lack of a cryptominer in the malware package is unusual, and said the particular services it targeted suggests its goal is either conduct its own spam campaigns and financial fraud with the stolen data, or to make the data it harvests available to those planning similar crimes. The worm's practice of removing TeamPCP files could be opportunistic, or could mean there’s drama going on in the cybercrime world. “We have no evidence to suggest whether this toolset represents someone associated with the group or familiar with their activities,” SentinelLabs noted. “However, the first toolset’s focus on disabling and replacing TeamPCP’s services implies a direct focus on the threat actor’s activities rather than pure cloud attack opportunism.” Because this is a worm relying on unsecured cloud and web app instances ripe for targeting, mitigation recommendations are pretty simple: Keep your cloud platforms secure, and ensure authentication is required even for instances of things like Docker and Kubernetes that aren’t exposed to the internet. ®

  •  
❌