It can be difficult for security teams to continuously monitor all on-premises servers due to budget and resource constraints. Signature-based antivirus alone is insufficient as modern malware uses various obfuscation techniques. Server admins may lack visibility into security events across all servers historically. Determining compromised systems and safe backups to restore from during incidents is challenging without centralized monitoring and alerting. It is onerous for server admins to setup and maintain additional security tools for advanced threat detection. The rapid mean time to detect and remediate infections is critical but difficult to achieve without the right automated solution.
Determining which backup image is safe to restore from during incidents without comprehensive threat intelligence is another hard problem. Even if backups are available, without knowing when exactly a system got compromised, it is risky to blindly restore from backups. This increases the chance of restoring malware and losing even more valuable data and systems during incident response. There is a need for an automated solution that can pinpoint the timeline of infiltration and recommend safe backups for restoration.
The solution leverages AWS Elastic Disaster Recovery (AWS DRS), Amazon GuardDuty and AWS Security Hub to address the challenges of malware detection for on-premises servers.
This combo of services provides a cost-effective way to continuously monitor on-premises servers for malware without impacting performance. It also helps determine safe recovery point in time backups for restoration by identifying timeline of compromises through centralized threat analytics.
AWS Elastic Disaster Recovery (AWS DRS) minimizes downtime and data loss with fast, reliable recovery of on-premises and cloud-based applications using affordable storage, minimal compute, and point-in-time recovery.
Amazon GuardDuty is a threat detection service that continuously monitors your AWS accounts and workloads for malicious activity and delivers detailed security findings for visibility and remediation.
AWS Security Hub is a cloud security posture management (CSPM) service that performs security best practice checks, aggregates alerts, and enables automated remediation.
The Malware Scan solution assumes on-premises servers are already being replicated with AWS DRS, and Amazon GuardDuty & AWS Security Hub are enabled. The cdk stack in this repository will only deploy the boxes labelled as DRS Malware Scan in the architecture diagram.
Amazon Security Hub enabled. If not, please check this documentation
Warning
Currently, Amazon GuardDuty Malware scan does not support EBS volumes encrypted with EBS-managed keys. If you want to use this solution to scan your on-prem (or other-cloud) servers replicated with DRS, you need to setup DRS replication with your own encryption key in KMS. If you are currently using EBS-managed keys with your replicating servers, you can change encryption settings to use your own KMS key in the DRS console.
Create a Cloud9 environment with Ubuntu image (at least t3.small for better performance) in your AWS account. Open your Cloud9 environment and clone the code in this repository. Note: Amazon Linux 2 has node v16 which is not longer supported since 2023-09-11 git clone https://github.com/aws-samples/drs-malware-scan
cd drs-malware-scan
sh check_loggroup.sh
Deploy the CDK stack by running the following command in the Cloud9 terminal and confirm the deployment
npm install
cdk bootstrap
cdk deploy --all
Note
The solution is made of 2 stacks: * DrsMalwareScanStack: it deploys all resources needed for malware scanning feature. This stack is mandatory. If you want to deploy only this stack you can run cdk deploy DrsMalwareScanStack
* ScanReportStack: it deploys the resources needed for reporting (Amazon Lambda and Amazon S3). This stack is optional. If you want to deploy only this stack you can run cdk deploy ScanReportStack
If you want to deploy both stacks you can run cdk deploy --all
All lambda functions route logs to Amazon CloudWatch. You can verify the execution of each function by inspecting the proper CloudWatch log groups for each function, look for the /aws/lambda/DrsMalwareScanStack-*
pattern.
The duration of the malware scan operation will depend on the number of servers/volumes to scan (and their size). When Amazon GuardDuty finds malware, it generates a SecurityHub finding: the solution intercepts this event and runs the $StackName-SecurityHubAnnotations
lambda to augment the SecurityHub finding with a note containing the name(s) of the DRS source server(s) with malware.
The SQS FIFO queues can be monitored using the Messages available and Message in flight metrics from the AWS SQS console
The DRS Volume Annotations DynamoDB tables keeps track of the status of each Malware scan operation.
Amazon GuardDuty has documented reasons to skip scan operations. For further information please check Reasons for skipping resource during malware scan
In order to analize logs from Amazon GuardDuty Malware scan operations, you can check /aws/guardduty/malware-scan-events
Amazon Cloudwatch LogGroup. The default log retention period for this log group is 90 days, after which the log events are deleted automatically.
Run the following commands in your terminal:
cdk destroy --all
(Optional) Delete the CloudWatch log groups associated with Lambda Functions.
For the purpose of this analysis, we have assumed a fictitious scenario to take as an example. The following cost estimates are based on services located in the North Virginia (us-east-1) region.
Monthly Cost | Total Cost for 12 Months |
---|---|
171.22 USD | 2,054.74 USD |
Service Name | Description | Monthly Cost (USD) |
---|---|---|
AWS Elastic Disaster Recovery | 2 Source Servers / 1 Replication Server / 4 disks / 100GB / 30 days of EBS Snapshot Retention Period | 71.41 |
Amazon GuardDuty | 3 TB Malware Scanned/Month | 94.56 |
Amazon DynamoDB | 100MB 1 Read/Second 1 Writes/Second | 3.65 |
AWS Security Hub | 1 Account / 100 Security Checks / 1000 Finding Ingested | 0.10 |
AWS EventBridge | 1M custom events | 1.00 |
Amazon Cloudwatch | 1GB ingested/month | 0.50 |
AWS Lambda | 5 ARM Lambda Functions - 128MB / 10secs | 0.00 |
Amazon SQS | 2 SQS Fifo | 0.00 |
Total | 171.22 |
Note The figures presented here are estimates based on the assumptions described above, derived from the AWS Pricing Calculator. For further details please check this pricing calculator as a reference. You can adjust the services configuration in the referenced calculator to make your own estimation. This estimation does not include potential taxes or additional charges that might be applicable. It's crucial to remember that actual fees can vary based on usage and any additional services not covered in this analysis. For critical environments is advisable to include Business Support Plan (not considered in the estimation)
See CONTRIBUTING for more information.
The purpose of the project is to create rate limit in AWS WaF based on HTTP headers.
Golang is a dependencie to build the binary. See the documentation to install: https://go.dev/doc/install
make
sudo make install
The rules configuration is very simple, for example, the threshold is the limited of the requests in X time. It's possible to monitoring multiples headers, but, the header needs to be in HTTP Request header log.
rules:
header:
x-api-id: # The header name in HTTP Request header
threshold: 100
token:
threshold: 1000
It's possible send notifications to Slack and Telegram. To configure slack notifications, you needs create a webhook configuration, see the slack documentation: https://api.slack.com/messaging/webhooks
Telegram bot father: https://t.me/botfather
notifications:
slack:
webhook-url: https://hooks.slack.com/services/DA2DA13QS/LW5DALDSMFDT5/qazqqd4f5Qph7LgXdZaHesXs
telegram:
bot-token: "123456789:NNDa2tbpq97izQx_invU6cox6uarhrlZDfa"
chat-id: "-4128833322"
To set up AWS credentials, it's advisable to export them as environment variables. Here's a recommended approach:
export AWS_ACCESS_KEY_ID=".."
export AWS_SECRET_ACCESS_KEY=".."
export AWS_REGION="us-east-1"
retrive-logs-minutes-ago is the time range you want to fetch the logs, in this example, logs from 1 hour ago.
aws:
waf-log-group-name: aws-waf-logs-cloudwatch-cloudfront
region: us-east-1
retrive-logs-minutes-ago: 60
Cybersecurity remains a very important topic and point of concern for many CIOs, CISOs, and their customers. To meet these important concerns, AWS has developed a primary set of services customers should use to aid in protecting their accounts. Amazon GuardDuty, AWS Security Hub, AWS Config, and AWS Well-Architected reviews help customers maintain a strong security posture over their AWS accounts. As more organizations deploy to the cloud, especially if they are doing so quickly, and they have not yet implemented the recommended AWS Services, there may be a need to conduct a rapid security assessment of the cloud environment.
With that in mind, we have worked to develop an inexpensive, easy to deploy, secure, and fast solution to provide our customers two (2) security assessment reports. These security assessments are from the open source projects βProwlerβ and βScoutSuite.β Each of these projects conduct an assessment based on AWS best practices and can help quickly identify any potential risk areas in a customerβs deployed environment. If you are interested in conducting these assessments on a continuous basis, AWS recommends enabling Security Hubβs Foundational Security Best P ractices standard. If you are interested in integrating your Prowler assessment results with Security Hub, you can also do that from Prowler natively following instructions here.
In addition, we have developed custom modules that speak to customer concerns around threats and misconfigurations of those issues, currently this includes checks for ransomware specific findings.
The architecture we deploy is a very simple VPC with two (2) subnets, one (1) NAT Gateway, one (1) EC2 instance, and one (1) S3 Bucket. The EC2 instance is using Amazon Linux 2 (the latest published AMI), that is patched on boot, pulls down the two projects (Prowler and ScoutSuite), runs the assessments and then delivers the reports to the S3 Bucket. The EC2 instances does not deploy with any EC2 Key Pair, does not have any open ingress rules on its Security Group, and is placed in the Private Subnet so it does not have direct internet access. After completion of the assessment and the delivery of the reports the system can be terminated.
The deployment is accomplished through the use of CloudFormation. A single CloudFormation template is used to launch a few other templates (in a modular approach). No parameters (user input) is required and the automated build out of the environment will take on average less than 10 minutes to complete. These templates are provided for review in this Github repository.
Once the EC2 Instance has been created and begins, the two assessments it will take somewhere around 40 minutes to complete. At the end of the assessments and after the two reports are delivered to the S3 Bucket the Instance will automatically shutdown, You may at this time safely terminate the Instance.
Here is a diagram of the architecture.
A VPC
An EIP
A NAT Gateway
A Security Group
A single m5a.large instance with a 10 GB gp2 EBS volume
An Instance Role
An IAM Policy
An S3 Bucket
These security assessments are from the open source projects βProwlerβ and βScoutSuite.β Each of these projects conduct an assessment based on AWS best practices and can help quickly identify any potential risk areas in a customerβs deployed environment.
The first assessment is from Prowler.
The second assessment is from ScoutSuite
When enabled, this module will deploy a lambda function that checks for common security mistakes highlighted in https://www.youtube.com/watch?v=tmuClE3nWlk.
A Lambda function that will perform the checks. Some of the checks include:
When enabled, this module will deploy separate functions that can help customers with evaluating their environment for ransomware infection and susceptibility to ransomware damage.
When enabled, this module will deploy separate functions that can help customers with evaluating their environment for SolarWinds vulnerability. The checks are based on CISA Alert AA20-352A from Appendix A & B.
Note
: Prior to enablement of this module, please read the module documentation which reviews the steps that need to be completed prior to using this module.
Note
: This module MUST be run separately as its own stack, select the S3 URL SelfServiceSecSolar.yml to deploy
See CONTRIBUTING for more information.
This project is licensed under the Apache-2.0 License.
Matano is an open source security lake platform for AWS. It lets you ingest petabytes of security and log data from various sources, store and query them in an open Apache Iceberg data lake, and create Python detections as code for realtime alerting. Matano is fully serverless and designed specifically for AWS and focuses on enabling high scale, low cost, and zero-ops. Matano deploys fully into your AWS account.
Matano lets you collect log data from sources using S3 or SQS based ingestion.
Matano normalizes and transforms your data using Vector Remap Language (VRL). Matano works with the Elastic Common Schema (ECS) by default and you can define your own schema.
Log data is always stored in S3 object storage, for cost effective, long term, durable storage.
All data is ingested into an Apache Iceberg based data lake, allowing you to perform ACID transactions, time travel, and more on all your log data. Apache Iceberg is an open table format, so you always own your own data, with no vendor lock-in.
Matano is a fully serverless platform, designed for zero-ops and unlimited elastic horizontal scaling.
Write Python detections to implement realtime alerting on your log data.
View the complete installation instructions.
You can install the matano CLI to deploy Matano into your AWS account, and manage your Matano deployment.
Matano provides a nightly release with the latest prebuilt files to install the Matano CLI on GitHub. You can download and execute these files to install Matano.
For example, to install the Matano CLI for Linux, run:
curl -OL https://github.com/matanolabs/matano/releases/download/nightly/matano-linux-x64.sh
chmod +x matano-linux-x64.sh
sudo ./matano-linux-x64.sh
Read the complete docs on getting started.
To get started with Matano, run the matano init
command. Make sure you have AWS credentials in your environment (or in an AWS CLI profile).
The interactive CLI wizard will walk you through getting started by generating an initial Matano directory for you, initializing your AWS account, and deploying Matano into your AWS account.
Initial deployment takes a few minutes.
View our complete documentation.