Share CloudWatch logs with another AWS account

This solution will use a Kinesis data stream in the account receiving the logs, since per AWS docs Kinesis streams are currently the only resource supported as a destination for cross-account subscriptions. With current prices and the default configuration of 1 shard, the Kinesis data stream should have a cost of around $12 a month. This may vary depending on the volume of logs you are sharing, and you might need to create additional shards for the data stream. You can find more information on Kinesis pricing here.

To share log data across accounts, you need to establish a log data sender and receiver:

• Log data sender—gets the destination information from the recipient and lets CloudWatch Logs know that it is ready to send its log events to the specified destination. In the procedures in the rest of this section, the log data sender is shown with a fictional AWS account number of 111111111111.

• Log data recipient—sets up a destination that encapsulates a Kinesis stream and lets CloudWatch Logs know that the recipient wants to receive log data. The recipient then shares the information about his destination with the sender. In the procedures in the rest of this section, the log data recipient is shown with a fictional AWS account number of 999999999999.

To start receiving log events from cross-account users, the log data recipient first creates a CloudWatch Logs destination. Each destination consists of the following key elements:

Target ARN: The Amazon Resource Name (ARN) of the AWS resource that you want to use as the destination of the subscription feed.

Access policy: An IAM policy document (in JSON format, written using IAM policy grammar) that governs the set of users that are allowed to write to your destination.

Keep in mind, the originating log group and the destination Kinesis data stream must be in the same region. The destination log group (or any other resource which the Kinesis data stream points to) can be in any region.

All steps in this procedure are to be done in the log data recipient account.

To create a destination

1. Create a destination stream in Kinesis. At a command prompt, type:

2. Wait until the Kinesis stream becomes active. You can use the aws kinesis describe-stream command to check the StreamDescription.StreamStatus property. In addition, take note of the StreamDescription.StreamARN value because it will be passed to CloudWatch Logs later:

It might take a minute or two for your stream to show up in the active state.

3. Create the IAM role that will grant CloudWatch Logs the permission to put data into your Kinesis stream. First, you'll need to create a trust policy in a file ~/TrustPolicyForCWL.json. Use a text editor to create this policy file, do not use the IAM console.

4. Use the aws iam create-role command to create the IAM role, specifying the trust policy file. Take note of the returned Role.Arn value because that will also be passed to CloudWatch Logs later:

5. Create a permissions policy to define which actions CloudWatch Logs can perform on your account. First, you'll use a text editor to create a permissions policy in a file ~/PermissionsForCWL.json:

6. Associate the permissions policy with the role using the aws iam put-role-policy command:

7. After the Kinesis stream is in the active state and you have created the IAM role, you can create the CloudWatch Logs destination.

      a. This step will not associate an access policy with your destination and is only the first step out of two that completes a destination creation. Make a note of the DestinationArn that is returned in the payload:

      b. After step 7a is complete, in the log data recipient account, associate an access policy with the destination. This policy enables the log data sender account (111111111111) to access the destination in the log data recipient account (999999999999). You can use a text editor to put this policy in the ~/AccessPolicy.json file:

      c. This creates a policy that defines who has write access to the destination. This policy must specify the logs:PutSubscriptionFilter action to access the destination. Cross-account users will use the PutSubscriptionFilter action to send log events to the destination:

This access policy allows the root user of the AWS Account with ID 111111111111 to call PutSubscriptionFilter against the destination with ARN arn:aws:logs:region:999999999999:destination:testDestination. Any other user's attempt to call PutSubscriptionFilter against this destination will be rejected.

In order to perform the put-subscription-filter command as defined in the bottom section, we need to create a local role which is required for cross-account log subscriptions using AWS Organizations

Note: This is for cross-account sharing when both AWS Accounts are part of the same AWS Organization

1. Create the following trust policy in a local .json file. Use a text editor to create this policy file. Let's call it /TrustPolicyForMySubscription.json

2. Create the IAM role that uses the above policy.

3. Create a permissions policy to define the actions that CloudWatch Logs can perform on your account. We will only need one api action which is the PutLogEvents action.

  a. First, use a text editor to create a local policy file. Let's call it /PermissionsForMySubscription.json

  b. Enter the following command in the CLI to associate the above Permissions Policy, with the IAM role which we created in this section's Step 2.

Once this is complete, we can now move onto the below section, Create a Subscription Filter, with the knowledge of which role we are going to use for the final flag, --role-arn

For more information on this step, you can check the official AWS Documentation

After you create a destination, the log data recipient account can share the destination ARN (arn:aws:logs:us-east-1:999999999999:destination:testDestination) with other AWS accounts so that they can send log events to the same destination. These other sending accounts users then create a subscription filter on their respective log groups against this destination. The subscription filter immediately starts the flow of real-time log data from the chosen log group to the specified destination.

In the following example, a subscription filter is created in a sending account. the filter is associated with a log group containing AWS CloudTrail events so that every logged activity made by "Root" AWS credentials is delivered to the destination you previously created. That destination encapsulates a Kinesis stream called "RecipientStream". For more information about how to send AWS CloudTrail events to CloudWatch Logs, see Sending CloudTrail Events to CloudWatch Logs in the AWS CloudTrail User Guide.

The log group and the destination must be in the same AWS region. However, the destination can point to an AWS resource such as a Kinesis stream that is located in a different region.

For more information you can check the AWS documentation.

Want to know more about StackZone and how to make your cloud management simple and secure?

Check our how it works section with easy to follow videos or just create your own StackZone Account here