StackZone Feature: S3 AntiVirus

ClamAV is an open source (GPLv2) anti-virus toolkit, designed especially for e-mail scanning on mail gateways. It provides a number of utilities including a flexible and scalable multi-threaded daemon, a command line scanner and advanced tool for automatic database updates. The core of the package is an anti-virus engine available in a form of shared library.

ClamAV is a fast scanning piece of open-source software capable of detecting many viruses, worms, trojans and other forms of malware. ClamAV can also scan within archives and compressed files which means it is a suitable took for S3 object level scanning. Often we upload compressed files such as .zip or .rar and many other files to Amazon S3 Buckets. 

For a full list of the compatible file types, we can review the ClamAV documentation page.

The S3 AntiVirus Feature can be enabled in StackZone from the StackZone Console. To do this, let's head on over to Provisioning > Core Accounts > Shared Services

Once here, we want to select the resource Amazon S3 AntiVirus.

Whilst the bulk of this feature is built within the Shared Services AWS Account, we are deploying Events Rules, IAM Roles and SQS Queue's in each other StackZone managed account. This is how we can maintain at scale a solution which is deployed within one single AWS Account which has the visibility and reach into all of your other AWS Accounts and can provide antivirus scanning - without setting up multiple Autoscaling Groups in every single account you have.

There is a lot to configure here as it's quite a technical feature. Take a look at the example image below to see a setup example, below we will dive into each of the settings and what they mean.

Amazon S3 Bucket Restriction:

We can lock this to a single bucket, by adding the arn of a valid S3 Bucket here, to only scan one bucket, or we can leave it as a * wildcard

Amazon S3 Object Restriction:

We can restrict access to specific S3 Objects within specific or all S3 Buckets, or we can leave it as * to allow access to all S3 Objects

Amazon S3 Tag Name:

Define a Tag Name which can trigger an Events Rule to scan viruses using this S3 AntiVirus solution. By default this is set to "S3Antivirus"

Amazon S3 Tag Value:

Define a Tag Value which can trigger an Events Rule to scan viruses using this S3 AntiVirus solution. By default this is set to "True"

Delete Infected Objects:

Yes / No - We can define here if we want the feature to automatically delete infected files or not.

Report Clean Objects:

Yes / No - We can define here if we want to receive a report of successful scan of clean as well as infected files to the SNS Topic

Tag Objects:

Yes / No - Tag S3 Object upon successful scan accordingly with values of "clean" or "infected" using key specified by below Tag-Key. 

Note: "infected" Tag only works if Delete Infected Objects is set to disabled.

Tag-Key:

S3 Object Tag Key used to specify the values of "clean" or "infected" tag values as explained above.

Autoscaling Min Size:

Minimum amount of EC2 Instances to use for scanning process. Must be 0 or above. By default this is 0

Autoscaling Desired Size:

Desired amount of EC2 Instances to use for scanning process. Must be 1 or above. By default this is 1

Autoscaling Max Size:

Maximum amount of EC2 Instances to use for scanning process. Must be 1 or above and not below the desired size. By default this is 5

Install SSM Agent: 

Yes / No - Enables AWS Systems Manager agent and authorization

Instance Type:

Specifies the instance type used for the scanning process and allocated to the Autoscaling Group.

Spot Price:

Optional spot price for EC2 Instances used for scanning process. This can be left as empty to use on-demand EC2 Instances. By default this is 0.0155

Log Retention In Days:

Defines the number of days you want to retain log events for this feature in CloudWatch Logs, by default this is set to 90 days

Volume Size:

Defines the size of the volume in GB. You can only scan files that are smaller than half of this value. By default this is set to 16GB

Swap Size:

Defines the size of the SWAP area, which must be able to fit inside the Volume Size above. By default this is set to 2

In order to ensure your S3 Bucket is sending data to the Scanning Auto Scaling Group (ASG) in the Shared Services Account - you must first tag your S3 Bucket in your member account with the Tag Key and Tag Value defined in the configuration above.

Amazon S3 Tag Key: S3Antivirus

Amazon S3 Tag Value: Scan

You will need to ensure that there are no Event Notifications on the S3 Bucket prior to tagging. This will add an S3 Event Notification to your bucket which will send to a local SNS Topic in your member account.

This SNS Topic will then forward onto the SQS Scan Queue in the Shared Services Account. From here, the scanning automation will pick up these messages and begin the next process.

For this example, we are going to upload a tiny .txt file into the Amazon S3 Bucket in our account called "scan-this-test-bucket"

This simulates how an average AWS Account would behave when this feature is active and we are adding objects to either a single, or perhaps multiple Amazon S3 Buckets. (Remember we can set the Amazon S3 Bucket Restriction to "*" so we cover all AWS Buckets we own)

Once we have uploaded this object to our Bucket, the EC2 AutoScaling Group will spin up some EC2 Spot Instances which installs the required software packages and runs the worker which will perform a scan of the newly uploaded S3 Object. Once this is complete, your object will be tagged appropriately if you have allowed for this in the feature settings.

Here is an example of an outcome in which our object was considered clean and tagged by the S3 Antivirus Feature accordingly

I am seeing the error "Failed to receive 1 resource signal(s) for the current batch. Each resource signal timeout is counted as a FAILURE" - What does this mean?

You may see this error in the CloudFormation Stack which is used to build all S3 AntiVirus resources within the Shared Services Account. This error message is linked to the EC2 Instances and Auto Scaling Groups. Cloudformation has not received back a signal from the EC2 Instance in the Auto Scaling Group to signify that all is ok and well with the box.

Revise your network configuration. Your Shared Services VPC chosen to build these resources inside must be able to reach the internet. This means the chosen VPC must have a NAT Gateway and the route tables must be configured appropriately.

The S3 AntiVirus Feature is not scanning one particular bucket only

You may see this problem if the feature cannot add an Event Notification to your S3 Bucket. This is required in order to receive notifications of events such as objects arriving in the designated S3 Bucket. If your Amazon S3 Bucket already has an Event Notification configured then the StackZone S3 AntiVirus feature will not be able to apply it's own.

I don't have an S3 Event Notification on my S3 Bucket after I tagged it

Double check the Tag Key and Tag Value are correct (case-sensitive) in comparison to your StackZone Configuration. You may be able to see the PutBucketNotification call in CloudTrail which may help discover why the api call was unsuccessful.

My SNS & SQS Subscriptions is stuck in Pending Confirmation

This can sometimes happen when we attempt to subscribe an SQS Queue to an SNS Topic. If we look at the SNS Topic which exists in each region and each account - check the subscriptions tab as pictured below: 

You will need to create this subscription if it doesn't exist already - if a pending subscription has expired it may not be present. The protocol is sqs and the endpoint is the ARN of your Scan Queue - located in your Shared Services Primary Region.

Remember: You must enable Raw Message Delivery on the SNS Subscription if you are creating this manually

To confirm the subscription - follow the AWS Documentation as instructed here to request the url you use in the Confirm Subscription portion of your member account

Want to know more about StackZone and how to make your cloud management simple and secure?

Check our how it works section with easy to follow videos or just create your own StackZone Account here