Drs-Malware-Scan - Perform File-Based Malware Scan On Your On-Prem Servers With AWS

Perform malware scan analysis of on-prem servers using AWS services

Challenges with on-premises malware detection

It can be difficult for security teams to continuously monitor all on-premises servers due to budget and resource constraints. Signature-based antivirus alone is insufficient as modern malware uses various obfuscation techniques. Server admins may lack visibility into security events across all servers historically. Determining compromised systems and safe backups to restore from during incidents is challenging without centralized monitoring and alerting. It is onerous for server admins to setup and maintain additional security tools for advanced threat detection. The rapid mean time to detect and remediate infections is critical but difficult to achieve without the right automated solution.

Determining which backup image is safe to restore from during incidents without comprehensive threat intelligence is another hard problem. Even if backups are available, without knowing when exactly a system got compromised, it is risky to blindly restore from backups. This increases the chance of restoring malware and losing even more valuable data and systems during incident response. There is a need for an automated solution that can pinpoint the timeline of infiltration and recommend safe backups for restoration.

How to use AWS services to address these challenges

The solution leverages AWS Elastic Disaster Recovery (AWS DRS), Amazon GuardDuty and AWS Security Hub to address the challenges of malware detection for on-premises servers.

This combo of services provides a cost-effective way to continuously monitor on-premises servers for malware without impacting performance. It also helps determine safe recovery point in time backups for restoration by identifying timeline of compromises through centralized threat analytics.

• AWS Elastic Disaster Recovery (AWS DRS) minimizes downtime and data loss with fast, reliable recovery of on-premises and cloud-based applications using affordable storage, minimal compute, and point-in-time recovery.

• Amazon GuardDuty is a threat detection service that continuously monitors your AWS accounts and workloads for malicious activity and delivers detailed security findings for visibility and remediation.

• AWS Security Hub is a cloud security posture management (CSPM) service that performs security best practice checks, aggregates alerts, and enables automated remediation.

Architecture

Solution description

The Malware Scan solution assumes on-premises servers are already being replicated with AWS DRS, and Amazon GuardDuty & AWS Security Hub are enabled. The cdk stack in this repository will only deploy the boxes labelled as DRS Malware Scan in the architecture diagram.

1. AWS DRS is replicating source servers from the on-premises environment to AWS (or from any cloud provider for that matter). For further details about setting up AWS DRS please follow the Quick Start Guide.
2. Amazon GuardDuty is already enabled.
3. AWS Security Hub is already enabled.
4. The Malware Scan solution is triggered by a Schedule Rule in Amazon EventBridge (with prefix DrsMalwareScanStack-ScheduleScanRule). You can adjust the scan frequency as needed (i.e. once a day, a week, etc).
5. The Schedule Rule in Amazon EventBridge triggers the Submit Orders lambda function (with prefix DrsMalwareScanStack-SubmitOrders) which gathers the source servers to scan from the Source Servers DynamoDB table.
6. Orders are placed on the SQS FIFO queue named Scan Orders (with prefix DrsMalwareScanStack-ScanOrdersfifo). The queue is used to serialize scan requests mapped to the same DRS instance, preventing a race condition.
7. The Process Order lambda picks a malware scan order from the queue and enriches it, preparing the upcoming malware scan operation. For instance, it inserts the id of the replicating DRS instance associated to the DRS source server provided in the order. The output of Process Order are malware scan commands containing all the necessary information to invoke GuardDuty malware scan.
8. Malware scan operations are tracked using the DRSVolumeAnnotationsDDBTable at the volume-level, providing reporting capabilities.
9. Malware scan commands are inserted in the Scan Commands SQS FIFO queue (with prefix DrsMalwareScanStack-ScanCommandsfifo) to increase resiliency.
10. The Process Commands function submits queued scan commands at a maximum rate of 1 command per second to avoid API throttling. It triggers the on-demand malware scan function provided by Amazon GuardDuty.
11. The execution of the on-demand Amazon GuardDuty Malware job can be monitored from the Amazon GuardDuty service.
12. The outcome of malware scan job is routed to Amazon Cloudwath Logs.
13. The Subscription Filter lambda function receives the outcome of the scan and tracks the result using DynamoDB (step #14).
14. The DRS Instance Annotations DynamoDB Table tracks the status of the malware scan job at the instance level.
15. The CDK stack named ScanReportStack deploys the Scan Report lambda function (with prefix ScanReportStack-ScanReport) to populate the Amazon S3 bucket with prefix scanreportstack-scanreportbucket.
16. AWS Security Hub aggregates and correlates findings from Amazon GuardDuty.
17. The Security Hub finding event is caught by an EventBridge Rule (with prefix DrsMalwareScanStack-SecurityHubAnnotationsRule)
18. The Security Hub Annotations lambda function (with prefix DrsMalwareScanStack-SecurityHubAnnotation) generates additional Notes (Annotations) to the Finding with contextualized information about the source server being affected. This additional information can be seen in the Notes section within the Security Hub Finding.
19. The follow-up activities will depend on the incident response process being adopted. For example based on the date of the infection, AWS DRS can be used to perform a point in time recovery using a snapshot previous to the date of the malware infection.
20. In a Multi-Account scenario, this solution can be deployed directly on the AWS account hosting the AWS DRS solution. The Amazon GuardDuty findings will be automatically sent to the centralized Security Account.

Usage

Pre-requisites

• An AWS Account.
• Amazon Elastic Disaster Recovery (DRS) configured, with at least 1 server source in sync. If not, please check this documentation. The Replication Configuration must consider EBS encryption using Custom Managed Key (CMK) from AWS Key Management Service (AWS KMS). Amazon GuardDuty Malware Protection does not support default AWS managed key for EBS.
• IAM Privileges to deploy the components of this solution.
• Amazon GuardDuty enabled. If not, please check this documentation

• Amazon Security Hub enabled. If not, please check this documentation

  Warning
  Currently, Amazon GuardDuty Malware scan does not support EBS volumes encrypted with EBS-managed keys. If you want to use this solution to scan your on-prem (or other-cloud) servers replicated with DRS, you need to setup DRS replication with your own encryption key in KMS. If you are currently using EBS-managed keys with your replicating servers, you can change encryption settings to use your own KMS key in the DRS console.

Deploy

1. Create a Cloud9 environment with Ubuntu image (at least t3.small for better performance) in your AWS account. Open your Cloud9 environment and clone the code in this repository. Note: Amazon Linux 2 has node v16 which is not longer supported since 2023-09-11 git clone https://github.com/aws-samples/drs-malware-scan

   cd drs-malware-scan

   sh check_loggroup.sh

2. Deploy the CDK stack by running the following command in the Cloud9 terminal and confirm the deployment

   npm install cdk bootstrap cdk deploy --all Note
   The solution is made of 2 stacks: * DrsMalwareScanStack: it deploys all resources needed for malware scanning feature. This stack is mandatory. If you want to deploy only this stack you can run cdk deploy DrsMalwareScanStack
   * ScanReportStack: it deploys the resources needed for reporting (Amazon Lambda and Amazon S3). This stack is optional. If you want to deploy only this stack you can run cdk deploy ScanReportStack

   If you want to deploy both stacks you can run cdk deploy --all

Troubleshooting

All lambda functions route logs to Amazon CloudWatch. You can verify the execution of each function by inspecting the proper CloudWatch log groups for each function, look for the /aws/lambda/DrsMalwareScanStack-* pattern.

The duration of the malware scan operation will depend on the number of servers/volumes to scan (and their size). When Amazon GuardDuty finds malware, it generates a SecurityHub finding: the solution intercepts this event and runs the $StackName-SecurityHubAnnotations lambda to augment the SecurityHub finding with a note containing the name(s) of the DRS source server(s) with malware.

The SQS FIFO queues can be monitored using the Messages available and Message in flight metrics from the AWS SQS console

The DRS Volume Annotations DynamoDB tables keeps track of the status of each Malware scan operation.

Amazon GuardDuty has documented reasons to skip scan operations. For further information please check Reasons for skipping resource during malware scan

In order to analize logs from Amazon GuardDuty Malware scan operations, you can check /aws/guardduty/malware-scan-events Amazon Cloudwatch LogGroup. The default log retention period for this log group is 90 days, after which the log events are deleted automatically.

Cleanup

1. Run the following commands in your terminal:

   cdk destroy --all

2. (Optional) Delete the CloudWatch log groups associated with Lambda Functions.

AWS Cost Estimation Analysis

For the purpose of this analysis, we have assumed a fictitious scenario to take as an example. The following cost estimates are based on services located in the North Virginia (us-east-1) region.

Estimated scenario:

• 2 Source Servers to replicate (DR) (Total Storage: 100GB - 4 disks)
• 3 TB Malware Scanned/Month
• 30 days of EBS snapshot Retention period
• Daily Malware scans

Service Breakdown:

Note The figures presented here are estimates based on the assumptions described above, derived from the AWS Pricing Calculator. For further details please check this pricing calculator as a reference. You can adjust the services configuration in the referenced calculator to make your own estimation. This estimation does not include potential taxes or additional charges that might be applicable. It's crucial to remember that actual fees can vary based on usage and any additional services not covered in this analysis. For critical environments is advisable to include Business Support Plan (not considered in the estimation)

Security

See CONTRIBUTING for more information.

Authors

• Rodrigo Monge
• Thierry Francois
• Diego Pérez Holguín
• Leandro Santi

Aws-Waf-Header-Analyzer - The Purpose Of The Project Is To Create Rate Limit In AWS WaF Based On HTTP Headers

The purpose of the project is to create rate limit in AWS WaF based on HTTP headers.

Golang is a dependencie to build the binary. See the documentation to install: https://go.dev/doc/install

make
sudo make install

The rules configuration is very simple, for example, the threshold is the limited of the requests in X time. It's possible to monitoring multiples headers, but, the header needs to be in HTTP Request header log.

rules:
  header:
    x-api-id: # The header name in HTTP Request header
      threshold: 100

    token:
      threshold: 1000

It's possible send notifications to Slack and Telegram. To configure slack notifications, you needs create a webhook configuration, see the slack documentation: https://api.slack.com/messaging/webhooks

Telegram bot father: https://t.me/botfather

notifications:
  slack:
    webhook-url: https://hooks.slack.com/services/DA2DA13QS/LW5DALDSMFDT5/qazqqd4f5Qph7LgXdZaHesXs

  telegram:
    bot-token: "123456789:NNDa2tbpq97izQx_invU6cox6uarhrlZDfa"
    chat-id:  "-4128833322"

To set up AWS credentials, it's advisable to export them as environment variables. Here's a recommended approach:

export AWS_ACCESS_KEY_ID=".."
export AWS_SECRET_ACCESS_KEY=".."
export AWS_REGION="us-east-1"

retrive-logs-minutes-ago is the time range you want to fetch the logs, in this example, logs from 1 hour ago.

aws:
  waf-log-group-name: aws-waf-logs-cloudwatch-cloudfront
  region: us-east-1
  retrive-logs-minutes-ago: 60

Aws-Security-Assessment-Solution - An AWS Tool To Help You Create A Point In Time Assessment Of Your AWS Account Using Prowler And Scout As Well As Optional AWS Developed Ransomware Checks

Self-Service Security Assessment too l

Cybersecurity remains a very important topic and point of concern for many CIOs, CISOs, and their customers. To meet these important concerns, AWS has developed a primary set of services customers should use to aid in protecting their accounts. Amazon GuardDuty, AWS Security Hub, AWS Config, and AWS Well-Architected reviews help customers maintain a strong security posture over their AWS accounts. As more organizations deploy to the cloud, especially if they are doing so quickly, and they have not yet implemented the recommended AWS Services, there may be a need to conduct a rapid security assessment of the cloud environment.

With that in mind, we have worked to develop an inexpensive, easy to deploy, secure, and fast solution to provide our customers two (2) security assessment reports. These security assessments are from the open source projects “Prowler” and “ScoutSuite.” Each of these projects conduct an assessment based on AWS best practices and can help quickly identify any potential risk areas in a customer’s deployed environment. If you are interested in conducting these assessments on a continuous basis, AWS recommends enabling Security Hub’s Foundational Security Best P ractices standard. If you are interested in integrating your Prowler assessment results with Security Hub, you can also do that from Prowler natively following instructions here.

In addition, we have developed custom modules that speak to customer concerns around threats and misconfigurations of those issues, currently this includes checks for ransomware specific findings.

ARCHITECTURE OVERVIEW

Overview - Open Source project checks

The architecture we deploy is a very simple VPC with two (2) subnets, one (1) NAT Gateway, one (1) EC2 instance, and one (1) S3 Bucket. The EC2 instance is using Amazon Linux 2 (the latest published AMI), that is patched on boot, pulls down the two projects (Prowler and ScoutSuite), runs the assessments and then delivers the reports to the S3 Bucket. The EC2 instances does not deploy with any EC2 Key Pair, does not have any open ingress rules on its Security Group, and is placed in the Private Subnet so it does not have direct internet access. After completion of the assessment and the delivery of the reports the system can be terminated.

The deployment is accomplished through the use of CloudFormation. A single CloudFormation template is used to launch a few other templates (in a modular approach). No parameters (user input) is required and the automated build out of the environment will take on average less than 10 minutes to complete. These templates are provided for review in this Github repository.

Once the EC2 Instance has been created and begins, the two assessments it will take somewhere around 40 minutes to complete. At the end of the assessments and after the two reports are delivered to the S3 Bucket the Instance will automatically shutdown, You may at this time safely terminate the Instance.

How to deploy this tool

• See the following guide on installation steps

How do I read the reports?

• See the following guide on accessing and reading the report outputs

Diagram

Here is a diagram of the architecture.

What will be created

• A VPC

  • This will be a /26 for the VPC
  • This will include 2 subnets both in the same Availability Zone, one Public and one Private
  • This will include the required Route Tables and ACLs

• An EIP

  • For use by the NAT Gateway

• A NAT Gateway

  • This is required for the instance to download both Prowler and ScoutSuite as well as to make the API Calls

• A Security Group

  • For the Instance

• A single m5a.large instance with a 10 GB gp2 EBS volume

  • This is the instance in which Prowler and ScoutSuite will run
  • It will be in a Private Subnet
  • It will not have an EC2 Key Pair
  • It will not allow ingress traffic on the Security Group

• An Instance Role

  • This Role is required so that Prowler and ScoutSuite can run the API calls from the EC2 Instance

• An IAM Policy

  • Some IAM permissions are required for Prowler and ScoutSuite
    • Prowler info here
    • ScoutSuite info here

• An S3 Bucket

  • This is the location where the reports will be delivered
  • It will take about 40 minutes for the reports to show up

Open source security Assessments

These security assessments are from the open source projects “Prowler” and “ScoutSuite.” Each of these projects conduct an assessment based on AWS best practices and can help quickly identify any potential risk areas in a customer’s deployed environment.

1. Prowler

The first assessment is from Prowler.

• Prowler follows guidelines of the CIS Amazon Web Services Foundations Benchmark (49 checks) and has 40 additional checks including related to GDPR and HIPAA, in total Prowler offers over 160 checks.

2. ScoutSuite

The second assessment is from ScoutSuite

• ScoutSuite has been around since 2012, originally a Scout, then Scout2, and now ScoutSuite. This will provide a set of files that can be viewed in your browser and conducts a wide range of checks

Overiew of optional modules

► Check for Common Security Mistakes module

When enabled, this module will deploy a lambda function that checks for common security mistakes highlighted in https://www.youtube.com/watch?v=tmuClE3nWlk.

What will be created

A Lambda function that will perform the checks. Some of the checks include:

• GuardDuty set to alert on findings
• GuardDuty enabled across all regions
• Prevent accidental key deletion
• Existence of a Multi-region CloudTrail
• CloudTrail validation enabled
• No local IAM users
• Roles tuned for least privilege in last 90 days
• Alerting for root account use
• Alerting for local IAM user create/delete
• Use of Managed Prefix Lists in Security Groups
• Public S3 Buckets

► Ransomware modules

When enabled, this module will deploy separate functions that can help customers with evaluating their environment for ransomware infection and susceptibility to ransomware damage.

What will be created

• AWS Core security services enabled
  • Checks for AWS security service enablement in all regions where applicable (GuardDuty, SecurityHub)
• Data protection checks
  • Checks for EBS volumes with no snapshot
  • Checks for outdated OS running
  • Checks for S3 bucket replication JobStatus
  • Checks for EC2 instances that can not be managed with SSM
  • Checks for Stale IAM roles that have been granted S3 access but have not used them in the last 60 days
  • Checks for S3 deny public access enablement
  • Checks to see if DNSSEC is enabled for public hosted zones in Amazon Route 53
  • Checks to see if logging is enabled for services relevant to ransomware (i.e. CloudFront, Lambda, Route53 Query Logging, and Route 53 Resolver Logging).
  • Checks to see if Route 53 Resolver DNS Firewall is enabled across all relevant regions
  • Checks to see if there are any Access Keys that have not been used in last 90 days

► SolarWinds module

When enabled, this module will deploy separate functions that can help customers with evaluating their environment for SolarWinds vulnerability. The checks are based on CISA Alert AA20-352A from Appendix A & B.

Note: Prior to enablement of this module, please read the module documentation which reviews the steps that need to be completed prior to using this module.

Note: This module MUST be run separately as its own stack, select the S3 URL SelfServiceSecSolar.yml to deploy

What will be created

• Athena query - AA20352A IP IOC
  • This Athena query will scan your VPC flow logs for IP addresses from the CISA AA20-352A.
• SSM Automation document - SolorWindsAA20-352AAutomatedScanner
  • This is a systems manager automation document that will scan Windows EC2 instances for impacted .dll files from CISA AA20-352A.
• Route53 DNS resolver query - AA20352A DNS IOC
  • This Athena query will scan your DNS logs for customers that have enabled DNS query logging

Frequently Asked Questions (FAQ)

1. Is there a cost?
   • Yes. This should normally cost less than $1 for an hour of use.
2. Is this a continuous monitoring and reporting tool?
   • No. This is a one-time assessment, we urge customers to leverage tooling like AWS SecurityHub for Ongoing assessments.
3. Why does the CloudFormation service error when deleting the stack?
   • You must remove the objects (reports) out of the S3 bucket first
4. Does this integrate with GuardDuty, Security Hub, CloudWatch, etc.?
   • Not at this time. In a future sprint we plan to incorporate integration with AWS services like Security Hub and GuardDuty. However, you can follow the instructions in this blog to integrate Prowler and Security Hub.
5. How do I remediate the issues in the reports?
   • Generally, the issues should be described in the report with readily identifiable corrections. Please follow up with the public documentation for each tool (Prowler and ScoutSuite) as well. If this is insufficient, please reach out to your AWS Account team and we will be more than happy to help you understand the reports and work towards remediating issues.

Security

See CONTRIBUTING for more information.

License

This project is licensed under the Apache-2.0 License.

Matano - The Open-Source Security Lake Platform For AWS

Matano is an open source security lake platform for AWS. It lets you ingest petabytes of security and log data from various sources, store and query them in an open Apache Iceberg data lake, and create Python detections as code for realtime alerting. Matano is fully serverless and designed specifically for AWS and focuses on enabling high scale, low cost, and zero-ops. Matano deploys fully into your AWS account.

Features

Collect data from all your sources

Matano lets you collect log data from sources using S3 or SQS based ingestion.

Ingest, transform, normalize log data

Matano normalizes and transforms your data using Vector Remap Language (VRL). Matano works with the Elastic Common Schema (ECS) by default and you can define your own schema.

Store data in S3 object storage

Log data is always stored in S3 object storage, for cost effective, long term, durable storage.

Apache Iceberg Data lake

All data is ingested into an Apache Iceberg based data lake, allowing you to perform ACID transactions, time travel, and more on all your log data. Apache Iceberg is an open table format, so you always own your own data, with no vendor lock-in.

Serverless

Matano is a fully serverless platform, designed for zero-ops and unlimited elastic horizontal scaling.

Detections as code

Write Python detections to implement realtime alerting on your log data.

Installing

View the complete installation instructions.

You can install the matano CLI to deploy Matano into your AWS account, and manage your Matano deployment.

Requirements

• Docker

Installation

Matano provides a nightly release with the latest prebuilt files to install the Matano CLI on GitHub. You can download and execute these files to install Matano.

For example, to install the Matano CLI for Linux, run:

curl -OL https://github.com/matanolabs/matano/releases/download/nightly/matano-linux-x64.sh
chmod +x matano-linux-x64.sh
sudo ./matano-linux-x64.sh

Getting started

Read the complete docs on getting started.

Deployment

To get started with Matano, run the matano init command. Make sure you have AWS credentials in your environment (or in an AWS CLI profile).

The interactive CLI wizard will walk you through getting started by generating an initial Matano directory for you, initializing your AWS account, and deploying Matano into your AWS account.

Initial deployment takes a few minutes.

Documentation

View our complete documentation.

License

• Apache-2.0 License