How can I analyze my CloudTrail Logs with CloudWatch Logs Insights?
I want to analyze my Amazon CloudTrail Logs using Amazon CloudWatch Logs Insights. How can I do this?
"I want to analyze my Amazon CloudTrail Logs using Amazon CloudWatch Logs Insights. How can I do this?Short descriptionYou can configure CloudTrail to log to CloudWatch Logs. Doing so allows you to use CloudWatch Logs Insights to analyze the CloudTrail logs to monitor specific account activity.To demonstrate Logs Insights ability to analyze CloudTrail logs, the following resolution contains sample queries. These queries cover the most common use cases:Isolating specific log fields.Filtering on different conditions.Aggregating events.Building a time series.ResolutionThe following queries explore Amazon Simple Storage Service (Amazon S3) bucket and object activity. By default, CloudTrail doesn't capture S3 data events. You can turn on event logging in CloudTrail. For more information, see Enabling CloudTrail event logging for S3 buckets and objects.You can build on these example queries to create additional and more complex Logs Insights queries aligned to your use case. You can also integrate queries with your CloudWatch dashboard to visualize your queries as charts and graphs alongside related metrics.Query 1: Latest eventsObjectiveRetrieves the most recent CloudTrail Log events with the default @timestamp and @message fields.Query#Retrieve the most recent CloudTrail eventsfields @timestamp, @message| sort @timestamp desc| limit 2Results@timestamp@message2022-02-18 17:52:31.118{"eventVersion":"1.08","userIdentity":{"type":"AssumedRole","principalId":"AROAWZKRRJU47ARZN7ECC:620d7d78144334d6933c27195cae2a98", "arn":"arn:aws:sts::123456789012:assumed- role/Amazon_EventBridge_Invoke_Run_Command_371790151/620d7d78144334d6933c27195cae2a98","accountId":"123456789012", "accessKeyId":"ASIAWZKRRJU4Y45M4SC6","sessionContext":{"sessionIssuer": {"type":"Role","principalId":"AROAWZKRRJU47ARZN7ECC","arn":"arn:aws:iam::123456789012:role/service- role/Amazon_EventBridge_Invoke_Run_Command_371790151","accountId":"123456789012","userName": "Amazon_EventBridge_Invoke_Run_Command_371790151" (output truncated)2022-02-18 17:51:52.137{"eventVersion":"1.08","userIdentity":{"type":"AssumedRole","principalId":"AROAWZKRRJU43YP4FHR2N:StateManagerService","arn":"arn:aws:sts::123456789012:assumed-role/AWSServiceRoleForAmazonSSM/StateManagerService","accountId":"123456789012","sessionContext":{"sessionIssuer":{"type":"Role","principalId":"AROAWZKRRJU43YP4FHR2N","arn":"arn:aws:iam::123456789012:role/aws-service-role/ssm.amazonaws.com/AWSServiceRoleForAmazonSSM","accountId":"123456789012","userName":"AWSServiceRoleForAmazonSSM"}, "webIdFederationData":{},"attributes":{"creationDate":"2022-02-18T17:50:06Z","mfaAuthenticated":"false"}},"invokedBy":"ssm.amazonaws.com"},"eventTime":"2022-02-18T17:50:06Z","eventSource":"ec2.amazonaws.com","eventName":"DescribeInstances","awsRegion":"eu-west-1","sourceIPAddress":"ssm.amazonaws.com","userAgent":"ssm.amazonaws.com","requestParameters":{"maxResults":50,"instancesSet": (output truncated)Query 2: Break out individual fieldsObjectiveIsolate the individual fields in @message.Display select fields in the CloudTrail event.Query#Breakout Individual Fieldsfields @timestamp, awsRegion, eventCategory, eventSource, eventName, eventType, sourceIPAddress, userIdentity.type| sort @timestamp desc| limit 2Results@timestampawsRegioneventCategoryeventSourceeventNameeventTypesourceIPAddressuserIdentity.type2022-02-18 18:00:09.647ca-central-1Managementsts.amazonaws.comAssumeRoleAwsApiCallcloudtrail.amazonaws.comAWSService2022-02-18 18:00:09.647ca-central-1Managementsts.amazonaws.comAssumeRoleAwsApiCallcloudtrail.amazonaws.comAWSServiceQuery 3: Filter by Amazon Elastic Compute Cloud (Amazon EC2) RunInstancesObjectiveRetrieve specific fields in the Cloudtrail event.Rename fields with more meaningful labels.Filter the latest EC2 instances launched in this account based on API call.Query#EC2: Recently Launched Instancesfields eventTime, eventName as API, responseElements.instancesSet.items.0.instanceId as InstanceID, userIdentity.sessionContext.sessionIssuer.type as IssuerType, userIdentity.type as IdentityType, userIdentity.sessionContext.sessionIssuer.userName as userName| filter eventName = 'RunInstances'| sort eventTime desc| limit 2ResultseventTimeAPIInstanceIDIssuerTypeIdentityTypeuserName2022-02-18T17:36:38ZRunInstancesi-0325b4d6ae4e93c75RoleAssumedRoleAWSServiceRoleForAutoScaling2022-02-18T13:45:18ZRunInstancesi-04d17a8425b7cb59aRoleAssumedRoleAWSServiceRoleForAutoScalingQuery 4: Filter by console login: Most recentObjectiveRetrieve specific fields in the Cloudtrail event.Rename fields with more meaningful labels.Filter the latest logins via the AWS Console based on API call.Query#Console Login: Most Recent API Callsfields eventTime, eventName, responseElements.ConsoleLogin as Response, userIdentity.arn as ARN, userIdentity.type as User_Type| filter eventName = 'ConsoleLogin'| sort eventTime desc| limit 10ResultseventTimeeventNameResponseARNUser_Type2022-02-18T17:35:44ZConsoleLoginSuccessarn:aws:iam::123456789012:user/test_userIAMUser2022-02-17T13:53:58ZConsoleLoginSuccessarn:aws:sts::123456789012:assumed-role/Admin/test_userAssumedRoleQuery 5: Filter by console login: Failed authenticationObjectiveRetrieve specific fields (for example, username, user type, source IP) in the Cloudtrail event.Rename fields with more meaningful labels.Filter the latest unsuccessful logins via the AWS Console.Query#ConsoleLogin: Filter on Failed Loginsfields eventTime, eventName, responseElements.ConsoleLogin as Response, userIdentity.userName as User, userIdentity.type as User_Type, sourceIPAddress, errorMessage| filter eventName = 'ConsoleLogin' and responseElements.ConsoleLogin = 'Failure'| sort eventTime desc| limit 10ResultseventTimeeventNameResponseUserUser_TypesourceIPAddresserrorMessage2022-02-18T20:10:55ZConsoleLoginFailureechoIAMUser12.34.56.89Failed authentication2022-02-18T20:10:43ZConsoleLoginFailureechoIAMUser12.34.56.89Failed authenticationQuery 6: Filter by Amazon Simple Storage Solution (Amazon S3) object uploadObjectiveRetrieve specific fields in the Cloudtrail event.Rename fields with more meaningful labels.Filter on API call and target S3 bucket.Query#Filter PutObject API Calls on a specific S3 Bucketfields @timestamp, eventName as API, requestParameters.bucketName as BucketName, requestParameters.key as Key, userIdentity.sessionContext.sessionIssuer.userName as UserName| filter eventName = 'PutObject' and BucketName = 'target-s3-bucket'| sort @timestamp desc| limit 2Results@timestampAPIBucketNameKeyUserName2022-02-12 17:16:07.415PutObjecttest_bucket1w4r9Hg4V7g.jpg2022-02-12 16:29:43.470PutObjecttest_bucket26wyBy0hBoB.jpgQuery 7: Summarize S3 activityObjectiveFilter based on S3 service.Aggregate all the matching events based on count statistic.Splice the results based on API, S3 bucket, and key.Use the stats command to rename fields.Sort by descending order.Query#S3 Activity: Bucket & Key Detailsfilter eventSource = 's3.amazonaws.com'| stats count(*) as Hits by eventName as API, requestParameters.bucketName as BucketName, requestParameters.key as Key| sort Hits desc| limit 5ResultsAPIBucketNameKeyHitsListAccessPoints44GetBucketAclteam1-ctrail-multi-region27GetBucketAclteam2-dub-cloudtrail27GetBucketAclaws-cloudtrail-logs-123456789012-ba940dd726GetObjectdevsupport-prodrdscr/individual/12345678901218Query 8: Summarize AWS Key Management Service (AWS KMS) decrypt activityObjectiveFilter based on KMS service and Decrypt API.Use the fields command to rename fields and aggregate on the user-friendly names.Aggregate all the matching events based on count statistic.Splice the results based on KMS Key and User.Sort by descending order.Query#KMS Decrypt Activity: Key & User Detailsfields resources.0.ARN as KMS_Key, userIdentity.sessionContext.sessionIssuer.userName as User| filter eventSource='kms.amazonaws.com' and eventName='Decrypt'| stats count(*) as Hits by KMS_Key, User| sort Hits desc| limit 2ResultsKMS_KeyUserHitsarn:aws:kms:us-east-1:123456789012:key/03f2923d-e213-439d-92cf-cbb444bd85bdAWSServiceRoleForConfig12arn:aws:kms:us-east-1:123456789012:key/03f2923d-e213-439d-92cf-cbb444bd85bdFoxTrot-1UQJBODTWZYZ68Query 9: Summarize API calls with errorsObjectiveFilter based on the presence of the errorCode field.Aggregate all of the matching events based on the count statistic.Splice the results based on AWS Service, API, and errorCode.Use the stats command to rename fields.Sort by largest number of matches.Query#Summarize API Calls with Errorsfilter ispresent(errorCode)| stats count(*) as Num_of_Events by eventSource as AWS_Service, eventName as API, errorCode| sort Num_of_Events desc| limit 5ResultsAWS_ServiceAPIerrorCodeNum_of_Eventss3.amazonaws.comGetBucketPublicAccessBlockNoSuchPublicAccessBlockConfiguration79lambda.amazonaws.comGetLayerVersionPolicy20181031ResourceNotFoundException66s3.amazonaws.comGetBucketPolicyStatusNoSuchBucketPolicy60s3.amazonaws.comHeadBucketAccessDenied47logs.amazonaws.comCreateLogStreamResourceNotFoundException21Query 10: Summarize S3 API calls with error codesObjectiveFilter based on S3 Service and the presence of errorCode field.Aggregate all the matching events based on the count statistic.Splice the results based on errorCode and errorMessage.Sort by largest number of matches.Query#S3: Summarize Error Codesfilter eventSource = 's3.amazonaws.com' and ispresent(errorCode)| stats count(*) as Hits by errorCode, errorMessage| sort Hits desc| limit 5ResultserrorCodeerrorMessageHitsAccessDeniedAccess Denied86NoSuchBucketPolicyThe bucket policy does not exist80NoSuchPublicAccessBlockConfigurationThe public access block configuration was not found79ObjectLockConfigurationNotFoundErrorObject Lock configuration does not exist for this bucket3ServerSideEncryptionConfigurationNotFoundErrorThe server side encryption configuration was not found3Query 11: Summarize AccessDenied/UnauthorizedOperation API calls by AWS service, API, and AWS Identity and Access Management (IAM) userObjectiveFilter on either AccessDenied or UnauthorizedOperation CloudTrail events.Aggregate all the matching events based on count statistic.Splice the results based on errorCode, AWS Service, API, and IAM user/role.Use the stats command to rename fields.Sort in descending order.Query#Summarize AccessDenied/UnauthorizedOperation API Calls by AWS Service, API, IAM Userfilter (errorCode='AccessDenied' or errorCode='UnauthorizedOperation')| stats count(*) as NumberOfEvents by errorCode, eventSource as AWS_Service, eventName as API, userIdentity.type as IdentityType, userIdentity.invokedBy as InvokedBy| sort NumberOfEvents desc| limit 10ResultserrorCodeAWS_ServiceAPIIdentityTypeInvokedByNumberOfEventsAccessDenieds3.amazonaws.comHeadBucketAWSServicedelivery.logs.amazonaws.com83AccessDenieds3.amazonaws.comGetObjectAssumedRole9Query 12: Time series: KMS hourly call volumeObjectiveFilter based on KMS Service and Decrypt API.Aggregate all the matching events into 1 hour bins.Visualize the results on a line graph.Query#KMS: Hourly Decrypt Call Volumefilter eventSource='kms.amazonaws.com' and eventName='Decrypt'| stats count(*) as Hits by bin(1h)Resultsbin(1h)Hits2022-02-18 19:00:00.000162022-02-18 18:00:00.000252022-02-18 17:00:00.000282022-02-18 16:00:00.000142022-02-18 15:00:00.00016Related informationMonitor AWS CloudTrail log data in Amazon CloudWatch (Video)Add query to dashboard or export query resultsCloudWatch Logs Insights sample queriesCloudWatch Logs Insights query commandsFollow"