The U.S. Cybersecurity and Infrastructure Security Agency (CISA) said today it is investigating a breach at business intelligence company Sisense, whose products are designed to allow companies to view the status of multiple third-party online services in a single dashboard. CISA urged all Sisense customers to reset any credentials and secrets that may have been shared with the company, which is the same advice Sisense gave to its customers Wednesday evening.
New York City based Sisense has more than a thousand customers across a range of industry verticals, including financial services, telecommunications, healthcare and higher education. On April 10, Sisense Chief Information Security Officer Sangram Dash told customers the company had been made aware of reports that “certain Sisense company information may have been made available on what we have been advised is a restricted access server (not generally available on the internet.)”
“We are taking this matter seriously and promptly commenced an investigation,” Dash continued. “We engaged industry-leading experts to assist us with the investigation. This matter has not resulted in an interruption to our business operations. Out of an abundance of caution, and while we continue to investigate, we urge you to promptly rotate any credentials that you use within your Sisense application.”
In its alert, CISA said it was working with private industry partners to respond to a recent compromise discovered by independent security researchers involving Sisense.
“CISA is taking an active role in collaborating with private industry partners to respond to this incident, especially as it relates to impacted critical infrastructure sector organizations,” the sparse alert reads. “We will provide updates as more information becomes available.”
Sisense declined to comment when asked about the veracity of information shared by two trusted sources with close knowledge of the breach investigation. Those sources said the breach appears to have started when the attackers somehow gained access to the company’s Gitlab code repository, and in that repository was a token or credential that gave the bad guys access to Sisense’s Amazon S3 buckets in the cloud.
Customers can use Gitlab either as a solution that is hosted in the cloud at Gitlab.com, or as a self-managed deployment. KrebsOnSecurity understands that Sisense was using the self-managed version of Gitlab.
Both sources said the attackers used the S3 access to copy and exfiltrate several terabytes worth of Sisense customer data, which apparently included millions of access tokens, email account passwords, and even SSL certificates.
The incident raises questions about whether Sisense was doing enough to protect sensitive data entrusted to it by customers, such as whether the massive volume of stolen customer data was ever encrypted while at rest in these Amazon cloud servers.
It is clear, however, that unknown attackers now have all of the credentials that Sisense customers used in their dashboards.
The breach also makes clear that Sisense is somewhat limited in the clean-up actions that it can take on behalf of customers, because access tokens are essentially text files on your computer that allow you to stay logged in for extended periods of time — sometimes indefinitely. And depending on which service we’re talking about, it may be possible for attackers to re-use those access tokens to authenticate as the victim without ever having to present valid credentials.
Beyond that, it is largely up to Sisense customers to decide if and when they change passwords to the various third-party services that they’ve previously entrusted to Sisense.
Earlier today, a public relations firm working with Sisense reached out to learn if KrebsOnSecurity planned to publish any further updates on their breach (KrebsOnSecurity posted a screenshot of the CISO’s customer email to both LinkedIn and Mastodon on Wednesday evening). The PR rep said Sisense wanted to make sure they had an opportunity to comment before the story ran.
But when confronted with the details shared by my sources, Sisense apparently changed its mind.
“After consulting with Sisense, they have told me that they don’t wish to respond,” the PR rep said in an emailed reply.
Update, 6:49 p.m., ET: Added clarification that Sisense is using a self-hosted version of Gitlab, not the cloud version managed by Gitlab.com.
Also, Sisense’s CISO Dash just sent an update to customers directly. The latest advice from the company is far more detailed, and involves resetting a potentially large number of access tokens across multiple technologies, including Microsoft Active Directory credentials, GIT credentials, web access tokens, and any single sign-on (SSO) secrets or tokens.
The full message from Dash to customers is below:
“Good Afternoon,
We are following up on our prior communication of April 10, 2024, regarding reports that certain Sisense company information may have been made available on a restricted access server. As noted, we are taking this matter seriously and our investigation remains ongoing.
Our customers must reset any keys, tokens, or other credentials in their environment used within the Sisense application.
Specifically, you should:
– Change Your Password: Change all Sisense-related passwords on http://my.sisense.com
– Non-SSO:
– Replace the Secret in the Base Configuration Security section with your GUID/UUID.
– Reset passwords for all users in the Sisense application.
– Logout all users by running GET /api/v1/authentication/logout_all under Admin user.
– Single Sign-On (SSO):
– If you use SSO JWT for the user’s authentication in Sisense, you will need to update sso.shared_secret in Sisense and then use the newly generated value on the side of the SSO handler.
– We strongly recommend rotating the x.509 certificate for your SSO SAML identity provider.
– If you utilize OpenID, it’s imperative to rotate the client secret as well.
– Following these adjustments, update the SSO settings in Sisense with the revised values.
– Logout all users by running GET /api/v1/authentication/logout_all under Admin user.
– Customer Database Credentials: Reset credentials in your database that were used in the Sisense application to ensure continuity of connection between the systems.
– Data Models: Change all usernames and passwords in the database connection string in the data models.
– User Params: If you are using the User Params feature, reset them.
– Active Directory/LDAP: Change the username and user password of users whose authorization is used for AD synchronization.
– HTTP Authentication for GIT: Rotate the credentials in every GIT project.
– B2D Customers: Use the following API PATCH api/v2/b2d-connection in the admin section to update the B2D connection.
– Infusion Apps: Rotate the associated keys.
– Web Access Token: Rotate all tokens.
– Custom Email Server: Rotate associated credentials.
– Custom Code: Reset any secrets that appear in custom code Notebooks.
If you need any assistance, please submit a customer support ticket at https://community.sisense.com/t5/support-portal/bd-p/SupportPortal and mark it as critical. We have a dedicated response team on standby to assist with your requests.
At Sisense, we give paramount importance to security and are committed to our customers’ success. Thank you for your partnership and commitment to our mutual security.
Regards,
Sangram Dash
Chief Information Security Officer”
The U.S. government agency in charge of improving the nation’s cybersecurity posture is ordering all federal agencies to take new measures to restrict access to Internet-exposed networking equipment. The directive comes amid a surge in attacks targeting previously unknown vulnerabilities in widely used security and networking appliances.
Under a new order from the Cybersecurity and Infrastructure Security Agency (CISA), federal agencies will have 14 days to respond to any reports from CISA about misconfigured or Internet-exposed networking equipment. The directive applies to any networking devices — such as firewalls, routers and load balancers — that allow remote authentication or administration.
The order requires federal departments to limit access so that only authorized users on an agency’s local or internal network can reach the management interfaces of these devices. CISA’s mandate follows a slew of recent incidents wherein attackers exploited zero-day flaws in popular networking products to conduct ransomware and cyber espionage attacks on victim organizations.
Earlier today, incident response firm Mandiant revealed that since at least October 2022, Chinese cyber spies have been exploiting a zero-day vulnerability in many email security gateway (ESG) appliances sold by California-based Barracuda Networks to hoover up email from organizations using these devices.
Barracuda was alerted to the exploitation of a zero-day in its products in mid-May, and two days later the company pushed a security update to address the flaw in all affected devices. But last week, Barracuda took the highly unusual step of offering to replace compromised ESGs, evidently in response to malware that altered the systems in such a fundamental way that they could no longer be secured remotely with software updates.
According to Mandiant, a previously unidentified Chinese hacking group was responsible for exploiting the Barracuda flaw, and appeared to be searching through victim organization email records for accounts “belonging to individuals working for a government with political or strategic interest to [China] while this victim government was participating in high-level, diplomatic meetings with other countries.”
When security experts began raising the alarm about a possible zero-day in Barracuda’s products, the Chinese hacking group altered their tactics, techniques and procedures (TTPs) in response to Barracuda’s efforts to contain and remediate the incident, Mandiant found.
Mandiant said the attackers will continue to change their tactics and malware, “especially as network defenders continue to take action against this adversary and their activity is further exposed by the infosec community.”
Meanwhile, this week we learned more details about the ongoing exploitation of a zero-day flaw in a broad range of virtual private networking (VPN) products made by Fortinet — devices many organizations rely on to facilitate remote network access for employees.
On June 11, Fortinet released a half-dozen security updates for its FortiOS firmware, including a weakness that researchers said allows an attacker to run malware on virtually any Fortinet SSL VPN appliance. The researchers found that just being able to reach the management interface for a vulnerable Fortinet SSL VPN appliance was enough to completely compromise the devices.
“This is reachable pre-authentication, on every SSL VPN appliance,” French vulnerability researcher Charles Fol tweeted. “Patch your #Fortigate.”
In details published on June 12, Fortinet confirmed that one of the vulnerabilities (CVE-2023-27997) is being actively exploited. The company said it discovered the weakness in an internal code audit that began in January 2023 — when it learned that Chinese hackers were exploiting a different zero-day flaw in its products.
Shodan.io, the search engine made for finding Internet of Things devices, reports that there are currently more than a half-million vulnerable Fortinet devices reachable via the public Internet.
The new cybersecurity directive from CISA orders agencies to remove any networking device management interfaces from the internet by making them only accessible from an internal enterprise network (CISA recommends an isolated management network). CISA also says agencies should “deploy capabilities, as part of a Zero Trust Architecture, that enforce access control to the interface through a policy enforcement point separate from the interface itself (preferred action).”
Security experts say CISA’s directive highlights the reality that cyberspies and ransomware gangs are making it increasingly risky for organizations to expose any devices to the public Internet, because these groups have strong incentives to probe such devices for previously unknown security vulnerabilities.
The most glaring example of this dynamic can be seen in the frequency with which ransomware groups have discovered and pounced on zero-day flaws in widely-used file transfer applications. One ransomware gang in particular — Cl0p — has repeatedly exploited zero day bugs in various file transfer appliances to extort tens of millions of dollars from hundreds of ransomware victims.
On February 2, KrebsOnSecurity broke the news that attackers were exploiting a zero-day vulnerability in the GoAnywhere file transfer appliance by Fortra. By the time security updates were available to fix the vulnerability, Cl0p had already used it to steal data from more than a hundred organizations running Fortra’s appliance.
According to CISA, on May 27, Cl0p began exploiting a previously unknown flaw in MOVEit Transfer, a popular Internet-facing file transfer application. MOVEit parent Progress Software has since released security updates to address the weakness, but Cl0p claims to have already used it to compromise hundreds of victim organizations. TechCrunch has been tracking the fallout from victim organizations, which range from banks and insurance providers to universities and healthcare entities.
The always on-point weekly security news podcast Risky Business has recently been urging organizations to jettison any and all FTP appliances, noting that Cl0p (or another crime gang) is likely to visit the same treatment on other FTP appliance vendors.
But that sound advice doesn’t exactly scale for mid-tier networking devices like Barracuda ESGs or Fortinet SSL VPNs, which are particularly prominent in small to mid-sized organizations.
“It’s not like FTP services, you can’t tell an enterprise [to] turn off the VPN [because] the productivity hit of disconnecting the VPN is terminal, it’s a non-starter,” Risky Business co-host Adam Boileau said on this week’s show. “So how to mitigate the impact of having to use a domain-joined network appliance at the edge of your network that is going to get zero-day in it? There’s no good answer.”
Risky Business founder Patrick Gray said the COVID-19 pandemic breathed new life into entire classes of networking appliances that rely on code which was never designed with today’s threat models in mind.
“In the years leading up to the pandemic, the push towards identity-aware proxies and zero trust everything and moving away from this type of equipment was gradual, but it was happening,” Gray said. “And then COVID-19 hit and everybody had to go work from home, and there really was one option to get going quickly — which was to deploy VPN concentrators with enterprise features.”
Gray said the security industry had been focused on building the next generation of remote access tools that are more security-hardened, but when the pandemic hit organizations scrambled to cobble together whatever they could.
“The only stuff available in the market was all this old crap that is not QA’d properly, and every time you shake them CVEs fall out,” Gray remarked, calling the pandemic, “a shot in the arm” to companies like Fortinet and Barracuda.
“They sold so many VPNs through the pandemic and this is the hangover,” Gray said. “COVID-19 extended the life of these companies and technologies, and that’s unfortunate.”