StackZone Feature: CloudWatch Alarms

Here you can enable Amazon CloudWatch Alarms which are constructed and deployed by StackZone. Amazon CloudWatch Alarms can perform one or more actions based on the value of a metric or expression relative to a threshold over a number of time periods. When a metric meets a criteria, the alarm is activated and will change state to IN ALARM.

This article is aimed to show you what CloudWatch Alarms can be created via StackZone, once each Alarm is enabled in the StackZone Console, it is created in every enabled region and in every managed AWS Account that you have. Automatically and creating many alarms at speed - saving you a lot of time in configurational work.

Amazon CloudWatch Alarms can be set up in your AWS Account to track specific metrics over a designated time period. Once those metrics or thresholds have been met, the CloudWatch Alarm will trigger - which can then do all sorts of actions as a response. They can be simplistic, such as sending an email to notify you an Alarm is in the Alarm state, or it can trigger other resources such as a Lambda Function, which could perform actions to help quell the current alarm.

For more about Amazon CloudWatch Alarms, be sure to head on over to their official documentation which covers many area's of this extensive service.

Alarm Description: This CloudWatch Alarm is aimed to monitor your Amazon EC2 Security Groups and notify you when changes are made to any Security Group resource.

Trigger: An API call is made to create, update or delete a Security Group

Alarm Threshold: SecurityGroupEventCount >= 1 for 1 datapoints within 5 minutes

Filter Pattern: ($.eventName = AuthorizeSecurityGroupIngress) || ($.eventName

= AuthorizeSecurityGroupEgress) || ($.eventName = RevokeSecurityGroupIngress)

|| ($.eventName = RevokeSecurityGroupEgress) || ($.eventName = CreateSecurityGroup)

|| ($.eventName = DeleteSecurityGroup)

How to Investigate: We will need to first note the Account and Region for where this CloudWatch Alarm triggered. With this information, go to that account's CloudTrail and ensure the region matches the region from which the Alarm triggered.

Click Event History. Here we can choose a more detailed API call or Event Source to try and find out more about what triggered this alarm. We can change the filter to be Event Source = ec2.amazonaws.com for a broad spectrum of recent api calls. However if we want more granularity on the specific API Call which would have triggered this alarm, we will need to set the filter to Event Name = 

• AuthorizeSecurityGroupIngress
• AuthorizeSecurityGroupEgress
• RevokeSecurityGroupIngress
• RevokeSecurityGroupEgress
• CreateSecurityGroup
• DeleteSecurityGroup

Alarm Description: This CloudWatch Alarm is aimed to monitor your Amazon Network Access Control Lists and notify you when changes are made to any Network ACL's.

Trigger: An API call is made to create, update or delete a Network ACL

Alarm Threshold: NetworkAclEventCount >= 1 for 1 datapoints within 5 minutes

Filter Pattern: ($.eventName = CreateNetworkAcl) || ($.eventName = CreateNetworkAclEntry)

|| ($.eventName = DeleteNetworkAcl) || ($.eventName = DeleteNetworkAclEntry)

|| ($.eventName = ReplaceNetworkAclEntry) || ($.eventName = ReplaceNetworkAclAssociation)

How to Investigate: We will need to first note the Account and Region for where this CloudWatch Alarm triggered. With this information, go to that account's CloudTrail and ensure the region matches the region from which the Alarm triggered.

Click Event History. Here we can choose a more detailed API call or Event Source to try and find out more about what triggered this alarm. We can change the filter to be Event Source = ec2.amazonaws.com for a broad spectrum of recent api calls. However if we want more granularity on the specific API Call which would have triggered this alarm, we will need to set the filter to Event Name = 

• CreateNetworkAcl
• CreateNetworkAclEntry
• DeleteNetworkAcl
• DeleteNetworkAclEntry
• ReplaceNetworkAclEntry
• ReplaceNetworkAclAssociation

Alarm Description: This CloudWatch Alarm is aimed to monitor your Amazon Network Gateways and notify you when changes are made to a Customer or Internet Gateway

Trigger: An API call is made to create, update or delete a Customer or Internet Gateway

Alarm Threshold: GatewayEventCount >= 1 for 1 datapoints within 5 minutes

Filter Pattern: ($.eventName = CreateCustomerGateway) || ($.eventName = DeleteCustomerGateway)

|| ($.eventName = AttachInternetGateway) || ($.eventName = CreateInternetGateway)

|| ($.eventName = DeleteInternetGateway) || ($.eventName = DetachInternetGateway)

How to Investigate: We will need to first note the Account and Region for where this CloudWatch Alarm triggered. With this information, go to that account's CloudTrail and ensure the region matches the region from which the Alarm triggered.

Click Event History. Here we can choose a more detailed API call or Event Source to try and find out more about what triggered this alarm. We can change the filter to be Event Source = ec2.amazonaws.com for a broad spectrum of recent api calls. However if we want more granularity on the specific API Call which would have triggered this alarm, we will need to set the filter to Event Name =

• CreateCustomerGateway
• DeleteCustomerGateway
• AttachInternetGateway
• CreateInternetGateway
• DeleteInternetGateway
• DetachInternetGateway

Alarm Description: This CloudWatch Alarm is aimed to monitor your Amazon Virtual Private Networks (VPC) and notify you when changes are made to any VPC, VPC Peering Connection or VPC connection to classic

Trigger: An API call is made to create, update or delete a VPC, VPC peering connection or VPC connection to classic

Alarm Threshold: VpcEventCount >= 1 for 1 datapoints within 5 minutes

Filter Pattern: ($.eventName = CreateVpc) || ($.eventName = DeleteVpc) ||

($.eventName = ModifyVpcAttribute) || ($.eventName = AcceptVpcPeeringConnection)

|| ($.eventName = CreateVpcPeeringConnection) || ($.eventName = DeleteVpcPeeringConnection)

|| ($.eventName = RejectVpcPeeringConnection) || ($.eventName = AttachClassicLinkVpc)

|| ($.eventName = DetachClassicLinkVpc) || ($.eventName = DisableVpcClassicLink)

|| ($.eventName = EnableVpcClassicLink)

How to Investigate: We will need to first note the Account and Region for where this CloudWatch Alarm triggered. With this information, go to that account's CloudTrail and ensure the region matches the region from which the Alarm triggered.

Click Event History. Here we can choose a more detailed API call or Event Source to try and find out more about what triggered this alarm. We can change the filter to be Event Source = ec2.amazonaws.com for a broad spectrum of recent api calls. However if we want more granularity on the specific API Call which would have triggered this alarm, we will need to set the filter to Event Name =

• CreateVpc
• DeleteVpc
• ModifyVpcAttribute
• AcceptVpcPeeringConnection
• CreateVpcPeeringConnection
• DeleteVpcPeeringConnection
• RejectVpcPeeringConnection
• AttachClassicLinkVpc
• DetachClassicLinkVpc
• DisableVpcClassicLink
• EnableVpcClassicLink

Alarm Description: This CloudWatch Alarm is aimed to monitor your Amazon EC2 Instances and notify you when changes are made to any EC2 Instances

Trigger: An API call is made to create, terminate, start, stop or reboot an EC2 instance.

Alarm Threshold: EC2InstanceEventCount >= 1 for 1 datapoints within 5 minutes

Filter Pattern: ($.eventName = RunInstances) || ($.eventName = RebootInstances)

|| ($.eventName = StartInstances) || ($.eventName = StopInstances) || ($.eventName

= TerminateInstances)

How to Investigate: We will need to first note the Account and Region for where this CloudWatch Alarm triggered. With this information, go to that account's CloudTrail and ensure the region matches the region from which the Alarm triggered.

Click Event History. Here we can choose a more detailed API call or Event Source to try and find out more about what triggered this alarm. We can change the filter to be Event Source = ec2.amazonaws.com for a broad spectrum of recent api calls. However if we want more granularity on the specific API Call which would have triggered this alarm, we will need to set the filter to Event Name =

• RunInstances
• RebootInstances
• StartInstances
• StopInstances
• TerminateInstances

Alarm Description: This CloudWatch Alarm is aimed to monitor your Amazon CloudTrail trails and notify you when changes are made to any CloudTrail trail.

Trigger: An API call is made to create, update or delete a CloudTrail trail, or to start or stop logging to a trail

Alarm Threshold: CloudTrailEventCount >= 1 for 1 datapoints within 5 minutes

Filter Pattern: ($.eventName = CreateTrail) || ($.eventName = UpdateTrail)

|| ($.eventName = DeleteTrail) || ($.eventName = StartLogging) || ($.eventName = StopLogging

How to Investigate: We will need to first note the Account and Region for where this CloudWatch Alarm triggered. With this information, go to that account's CloudTrail and ensure the region matches the region from which the Alarm triggered.

Click Event History. Here we can choose a more detailed API call or Event Source to try and find out more about what triggered this alarm. We can change the filter to be Event Source = cloudtrail.amazonaws.com for a broad spectrum of recent api calls. However if we want more granularity on the specific API Call which would have triggered this alarm, we will need to set the filter to Event Name =

• CreateTrail
• UpdateTrail
• DeleteTrail
• StartLogging
• StopLogging

Alarm Description: This CloudWatch Alarm is aimed to monitor authentication into your AWS Accounts' Console and notify you when any failed authentications are logged.

Trigger: When an unauthenticated API call is made to sign into the AWS Console

Alarm Threshold: ConsoleSignInFailureCount >= 3 for 1 datapoints within 5 minutes

Filter Pattern: ($.eventName = ConsoleLogin) && ($.errorMessage = "Failed authentication")

How to Investigate: We will need to first note the Account for where this CloudWatch Alarm triggered. With this information, go to that account's CloudTrail and ensure the region is set to us-east-1, or your StackZone primary region, which gets all IAM calls logged.

Click Event History. Here we can choose a more detailed API call or Event Source to try and find out more about what triggered this alarm. We can change the filter to be Event Source = signin.amazonaws.com for a broad spectrum of recent api calls. However if we want more granularity on the specific API Call which would have triggered this alarm, we will need to set the filter to Event Name = ConsoleLogin

Alarm Description: This CloudWatch Alarm is aimed to monitor your Amazon Identity and Access Management (IAM) and notify you when changes are made to any IAM Policies.

Trigger: An API call is made to create, update or delete an IAM Policy

Alarm Threshold: IAMPolicyEventCount >= 1 for 1 datapoints within 5 minutes

Filter Pattern: ($.eventName=DeleteGroupPolicy)||($.eventName=DeleteRolePolicy)||($.eventName=DeleteUserPolicy)||($.eventName=PutGroupPolicy)||($.eventName=PutRolePolicy)||($.eventName=PutUserPolicy)||($.eventName=CreatePolicy)||($.eventName=DeletePolicy)||($.eventName=CreatePolicyVersion)||($.eventName=DeletePolicyVersion)||($.eventName=AttachRolePolicy)||($.eventName=DetachRolePolicy)||($.eventName=AttachUserPolicy)||($.eventName=DetachUserPolicy)||($.eventName=AttachGroupPolicy)||($.eventName=DetachGroupPolicy)

How to Investigate: We will need to first note the Account for where this CloudWatch Alarm triggered. With this information, go to that account's CloudTrail and ensure the region is set to us-east-1, or your StackZone primary region, which gets all IAM calls logged.

Click Event History. Here we can choose a more detailed API call or Event Source to try and find out more about what triggered this alarm. We can change the filter to be Event Source = iam.amazonaws.com for a broad spectrum of recent api calls. However if we want more granularity on the specific API Call which would have triggered this alarm, we will need to set the filter to Event Name =

• DeleteGroupPolicy
• DeleteRolePolicy
• DeleteUserPolicy
• PutGroupPolicy
• PutRolePolicy
• PutUserPolicy
• CreatePolicy
• DeletePolicy
• CreatePolicyVersion
• DeletePolicyVersion
• AttachRolePolicy
• DetachRolePolicy
• AttachUserPolicy
• DetachUserPolicy
• AttachGroupPolicy
• DetachGroupPolicy

Alarm Description: This CloudWatch Alarm is aimed to monitor your Amazon AWS Root Account and notify you when an authentication call is successful for the Root Account.

Trigger: An authentication call is successful for the Root Account of the AWS Account in question.

Alarm Threshold: RootLoginEventCount >= 1 for 1 datapoints within 5 minutes

Filter Pattern: { $.userIdentity.type = "Root" && $.userIdentity.invokedBy NOT EXISTS && $.eventType != "AwsServiceEvent" }

How to Investigate: We will need to first note the Account for where this CloudWatch Alarm triggered. With this information, go to that account's CloudTrail and ensure the region is set to us-east-1, or your StackZone primary region, which gets all IAM calls logged.

Click Event History. Here we can choose a more detailed API call or Event Source to try and find out more about what triggered this alarm. We can change the filter to be Event Source = signin.amazonaws.com for a broad spectrum of recent api calls. From here we can see all of the Sign-In API calls - where you may see "root" listed as the User Name from the returned list

However, this may not be sufficient as you may have a wealth of other sign in's from other non-root users. Another way to investigate this is to change the filter to User name = root. This is not a method I can recommend however as it will pull back all AWS Service Events too.

CloudWatch Logs will be a very good place to go to ensure we find the appropriate event which has triggered this Alarm. To do this, head on over to your CloudWatch Dashboard in the AWS Console, ensuring your region is set to your Primary region.

In your list of Log Groups, search for the CloudTrail prefix, so you can see the full log group CloudTrail/Landing-Zone-Logs/

Once here, you will see a number of Log streams. You will want to choose the stream with your most recent event time written to, in our case the Log stream is $Account_ID_CloudTrail_eu-west-2_4

By adding the Filter Pattern to search through the logs, we see the below outcome:

Alarm Description: This CloudWatch Alarm is aimed to monitor your Amazon EC2 Instance AMI's and notify you when one of your Amazon AMI's is made public.

Trigger: An Amazon AMI you own in one of your AWS Account has had it's visibility changed to public.

Alarm Threshold: ModifyImageAttribute >= 1 for 1 datapoints within 5 minutes

Filter Pattern: ($.eventName = ModifyImageAttribute) && ($.requestParameters.launchPermission.add.items.group = all)

How to Investigate: We will need to first note the Account and Region for where this CloudWatch Alarm triggered. With this information, go to that account's CloudTrail and ensure the region matches the region from which the Alarm triggered.

Click Event History. Here we can choose a more detailed API call or Event Source to try and find out more about what triggered this alarm. We can change the filter to be Event Source = ec2.amazonaws.com for a broad spectrum of recent api calls. However if we want more granularity on the specific API Call which would have triggered this alarm, we will need to set the filter to Event Name = ModifyImageAttribute

Alarm Description: This CloudWatch Alarm is aimed to monitor your Amazon VPC Network Traffic and notify you when there are rejected SSH connections within an Amazon VPC.

Trigger: A rejected number of SSH connection attempts are triggered within an hour. By default this is 10 within an hour.

Alarm Threshold: RejectedSSHCount >= 10 for 1 datapoints within 1 hour

Filter Pattern: [version, account, eni, source, destination, srcport, destport="22",

protocol="6", packets, bytes, windowstart, windowend, action="REJECT",

flowlogstatus]

How to Investigate: To investigate this Alarm thoroughly, we need to head to CloudWatch Logs instead of CloudTrail, as we are searching within VPC Flow Logs and not API calls logged within CloudTrail.

Head to your CloudWatch Dashboard within the AWS Console. You will need to locate the VPC Flow Logs CloudWatch Logs logstream for which this alarm has been triggered. By using the Filter Pattern listed above, we can focus on the REJECTION actions for Port 22 for any ENI in the VPC. Here below we can see 6 rejections for an SSH connection within a few minutes. If there were 4 or more extra rejections such as this, the CloudWatch Alarm would have triggered.

Alarm Description: This CloudWatch Alarm is aimed to monitor your Amazon AWS Console Logins and notify you when an AWS Console Sign-In has been successful for an account without MFA linked.

Trigger: An authorized AWS Console Login has been actioned without the use of MFA.

Alarm Threshold: ConsoleSignInNoMFACount >= 1 for 1 datapoints within 5 minutes

Filter Pattern: { $.eventName = "ConsoleLogin" && $.additionalEventData.MFAUsed = "No" }

How to Investigate: We will need to first note the Account for where this CloudWatch Alarm triggered. With this information, go to that account's CloudTrail and ensure the region is set to us-east-1, which gets all IAM calls logged.

Click Event History. Here we can choose a more detailed API call or Event Source to try and find out more about what triggered this alarm. We can change the filter to be Event Source = signin.amazonaws.com for a broad spectrum of recent api calls. However if we want more granularity on the specific API Call which would have triggered this alarm, we will need to set the filter to Event Name = ConsoleLogin

NOTE: This works well with IAM User sign in's, but will alarm incorrectly if monitoring an AWS SSO Login - even though the SSO User has used MFA as part of the login process. Consider disabling this Alarm if you use AWS SSO Users.

Alarm Description: This CloudWatch Alarm is aimed to monitor your Amazon Network Route Tables and notify you when changes are made to any Route Table's

Trigger: An API call is made to create, update or delete a Route Table

Alarm Threshold: RouteTableEventCount >= 1 for 1 datapoints within 5 minutes

Filter Pattern: ($.eventName = AssociateRouteTable) || ($.eventName = CreateRoute) ||

($.eventName = CreateRouteTable) || ($.eventName = DeleteRoute) ||

($.eventName = DeleteRouteTable) || ($.eventName = ReplaceRoute) ||

($.eventName = ReplaceRouteTableAssociation) || ($.eventName = DisassociateRouteTable)

How to Investigate: We will need to first note the Account and Region for where this CloudWatch Alarm triggered. With this information, go to that account's CloudTrail and ensure the region matches the region from which the Alarm triggered.

Click Event History. Here we can choose a more detailed API call or Event Source to try and find out more about what triggered this alarm. We can change the filter to be Event Source = ec2.amazonaws.com for a broad spectrum of recent api calls. However if we want more granularity on the specific API Call which would have triggered this alarm, we will need to set the filter to Event Name =

• AssociateRouteTable
• CreateRoute
• CreateRouteTable
• DeleteRoute
• DeleteRouteTable
• ReplaceRoute
• ReplaceRouteTableAssociation
• DisassociateRouteTable

Alarm Description: This CloudWatch Alarm is aimed to monitor your Amazon Key Management Store (KMS) and notify you when any customer created CMK's get disabled or scheduled for deletion

Trigger: An API call is made to disable or schedule deletion of a customer created CMK (Customer Managed Key)

Alarm Threshold: CMKEventCount >= 1 for 1 datapoints within 5 minutes

Filter Pattern: ($.eventSource = kms.amazonaws.com) && (($.eventName=DisableKey) ||

($.eventName=ScheduleKeyDeletion))

How to Investigate: We will need to first note the Account and Region for where this CloudWatch Alarm triggered. With this information, go to that account's CloudTrail and ensure the region matches the region from which the Alarm triggered.

Click Event History. Here we can choose a more detailed API call or Event Source to try and find out more about what triggered this alarm. We can change the filter to be Event Source = kms.amazonaws.com for a broad spectrum of recent api calls. However if we want more granularity on the specific API Call which would have triggered this alarm, we will need to set the filter to Event Name =

• DisableKey
• ScheduleKeyDeletion

Alarm Description: This CloudWatch Alarm is aimed to monitor your Amazon Elastic File Systems and notify you when an unencrypted EFS File System is discovered.

Trigger: An Unencrypted EFS File System is discovered

Alarm Threshold: UnencryptedFileSystemCreatedEventCount >= 1 for 1 datapoints within 1 minute

Filter Pattern: ($.eventName = CreateFileSystem) && ($.responseElements.encrypted IS FALSE)

How to Investigate: We will need to first note the Account and Region for where this CloudWatch Alarm triggered. With this information, go to that account's CloudTrail and ensure the region matches the region from which the Alarm triggered.

Click Event History. Here we can choose a more detailed API call or Event Source to try and find out more about what triggered this alarm. We can change the filter to be Event Source = elasticfilesystem.amazonaws.com for a broad spectrum of recent api calls. However if we want more granularity on the specific API Call which would have triggered this alarm, we will need to set the filter to Event Name = CreateFileSystem

We can see in the event from this recent example that the EFS File System was created, and was encrypted. This would not trigger this CloudWatch Alarm.

Note: It is not possible to create an unencrypted EFS File System from the AWS Console. But it may be possible to create one in this state via CLI or IAC tools

Alarm Description: This CloudWatch Alarm is aimed to monitor your Amazon Elastic Block Storage (EBS) Encryption and notify you when EBS Default Encryption has been disabled.

Trigger: An API call is made to disable EBS Encryption by default.

Alarm Threshold: EBSEventCount >= 1 for 1 datapoints within 5 minutes

Filter Pattern: ($.eventSource = ec2.amazonaws.com) && ($.eventName=DisableEbsEncryptionByDefault)

How to Investigate: We will need to first note the Account and Region for where this CloudWatch Alarm triggered. With this information, go to that account's CloudTrail and ensure the region matches the region from which the Alarm triggered.

Click Event History. Here we can choose a more detailed API call or Event Source to try and find out more about what triggered this alarm. We can change the filter to be Event Source = ec2.amazonaws.com for a broad spectrum of recent api calls. However if we want more granularity on the specific API Call which would have triggered this alarm, we will need to set the filter to Event Name = DisableEbsEncryptionByDefault

Alarm Description: This CloudWatch Alarm is aimed to monitor your Amazon Organization and notify you when changes are made to any part of your Amazon Organization.

Trigger: An API call is made to create, update or delete an Organization, OU or Policy.

Alarm Threshold: OrganizationsEventCount >= 1 for 1 datapoints within 5 minutes

Filter Pattern: ($.eventSource = organizations.amazonaws.com) && (($.eventName=CreateAccount) ||

($.eventName=InviteAccountToOrganization) || ($.eventName=DeletePolicy) || ($.eventName=CreatePolicy) || ($.eventName=DeleteOrganizationalUnit) || ($.eventName=CreateOrganizationalUnit) || ($.eventName=UpdatePolicy))

How to Investigate: We will need to first note the Account and Region for where this CloudWatch Alarm triggered. With this information, go to that account's CloudTrail and ensure the region matches the region from which the Alarm triggered.

Note: This will only appear within the us-east-1 CloudTrail Logs.

Click Event History. Here we can choose a more detailed API call or Event Source to try and find out more about what triggered this alarm. We can change the filter to be Event Source = organizations.amazonaws.com for a broad spectrum of recent api calls. However if we want more granularity on the specific API Call which would have triggered this alarm, we will need to set the filter to Event Name =

• CreateAccount
• InviteAccountToOrganization
• DeletePolicy
• CreatePolicy
• DeleteOrganizationalUnit
• CreateOrganizationalUnit
• UpdatePolicy

Note: Some of these event names such as UpdatePolicy and CreatePolicy will have additional event sources in your CloudTrail Event History, such as iam.amazonaws.com or potentially others. This could convolute your investigations so it's important to know which event sources you need to investigate this properly.

This will also mean that single events such as UpdatePolicy for example, which do not originate from AWS Organizations will not trigger this CloudWatch Alarm.

Alarm Description: This CloudWatch Alarm is aimed to monitor your Key Management Service (KMS) keys and notify you when a key that has been scheduled for deletion is used. When in alarm it will create an OpsItem with Severity 2 under the Security category.

Trigger: A KMS key deletion is scheduled from the AWS Console o through an API call

Alarm Threshold: KMSUsageEventCount >= 1 for 1 datapoints within 5 minutes

Filter Pattern: ($.eventSource = kms* && $.errorMessage = "* is pending deletion.")

How to Investigate: We will need to first note the Account and Region for where this CloudWatch Alarm triggered. With this information, go to that account's CloudTrail and ensure the region matches the region from which the Alarm triggered.

Click Event History. Here we can choose a more detailed API call or Event Source to try and find out more about what triggered this alarm. We can change the filter to be Event Source = kms.amazonaws.com for a broad spectrum of recent API calls. However if we want more granularity on the specific API Call which would have triggered this alarm, we will need to set the filter to Event Name = ScheduleKeyDeletion to filter for keys that has been scheduled for deletion and might be used.

All alerts generated by this feature can be collected in a single SNS topic within the StackZone security account.

If you want to receive all of your alerts in one place, or if you want to send them to a specific email address, we highly recommend you to check the Security Shared Topic documentation.

Documentation Notes:

7th February 2023 - as of v4.30.0 of StackZone we removed the Amazon EC2 Large Instance Change Alarm from this feature.

Want to know more about StackZone and how to make your cloud management simple and secure?

Check our how it works section with easy to follow videos or just create your own StackZone Account here