Why isn't the unified CloudWatch agent pushing log events?
"I'm using the unified CloudWatch agent to push logs to Amazon CloudWatch Logs. However, I can't see the logs in the CloudWatch Logs console. Why isn't the unified CloudWatch agent pushing log events?"
"I'm using the unified CloudWatch agent to push logs to Amazon CloudWatch Logs. However, I can't see the logs in the CloudWatch Logs console. Why isn't the unified CloudWatch agent pushing log events?Short descriptionThe following issues can prevent the unified CloudWatch agent from pushing log events:Out-of-sync metadata caused by creating an Amazon Machine Image (AMI) after the CloudWatch agent is installedUsing an outdated version of the CloudWatch agentFailure to connect to the CloudWatch Logs endpointIncorrect account, Region, or log group configurationsInsufficient AWS Identity and Access Management (IAM) permissionsCloudWatch agent run errorsTimestamp issuesResolutionNote: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.Important: The following troubleshooting steps don't apply to Container Insights. For details on troubleshooting Container Insights, see Troubleshooting Container Insights and How can I troubleshoot pod status in Amazon EKS?Review the installation method for the CloudWatch agentIt's a best practice to install the CloudWatch agent at launch using AWS CloudFormation, AWS Systems Manager Agent (SSM Agent), user data scripts, or the AWS CLI. It is also a best practice to create an AMI before installing the CloudWatch agent. AMIs typically capture unique information from the original instance. Metadata becomes out of sync, and this state can lead to the CloudWatch agent not working as intended. Out-of-sync metadata is the reason that many Windows instances require Sysprep when working with AMI. For more information, see How can I use Sysprep to create and install custom reusable Windows AMIs in Amazon EC2?Confirm that you're using the latest version of the CloudWatch agentDownload and review the README files for the CloudWatch agent release notes and latest version number. If you're using an older version of the CloudWatch agent, be sure to upgrade. The latest version might include updates that resolve the issue that you're experiencing.Test connectivity to your CloudWatch Logs endpointTest connectivity to the CloudWatch Logs endpoint using either of the following commands:telnet logs.<region>.amazonaws.com 443nc -zv logs.<region>.amazonaws.com 443If you get connectivity failures, be sure that:The security group and network access control list (ACL) rules allow connectivityYour instance can reach the public endpoint using an internet gateway or a network address translation (NAT) gatewayIf you're using VPC endpoints, the endpoint resolves to a VPC IP and the endpoint security group allows access from the source instanceReview your account, Region, and log group configurationsIn the CloudWatch agent configuration file:Be sure that the specified Region matches the console RegionVerify that logs are checked in the correct accountOptionally, you can use the common-config.toml file to override system defaults for the CloudWatch agent. These system defaults include the proxy, Region, and credential information for the agent. The file is available in the following locations.Linux:/opt/aws/amazon-cloudwatch-agent/etc/common-config.toml-or-/etc/amazon/amazon-cloudwatch-agent/common-config.tomlWindows:$Env:ProgramData\Amazon\AmazonCloudWatchAgent\common-config.tomlCheck your IAM permissionsThe CloudWatch agent uses credentials from either the IAM user or IAM role policy to push log events to the CloudWatch service. Before a log event can be published, you must create a log group and log stream. If there's no log group or log stream, the CloudWatch agent creates them.Confirm that your policy includes the following IAM permissions:"logs:CreateLogGroup","logs:CreateLogStream","logs:PutLogEvents","logs:DescribeLogStreams"Add any missing IAM permissions to the user policy or the role policy.Note: When creating IAM roles and users, it's a best practice to use the CloudWatchAgentServerPolicy and CloudWatchAgentAdminPolicy policies created by Amazon rather than custom policies.Resolve CloudWatch agent run errorsVerify that the CloudWatch agent is running. If the agent isn't running, check the log file for errors and resolve them. Log files are located in the following locations.Linux:/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.logWindows:$Env:ProgramData\Amazon\AmazonCloudWatchAgent\Logs\amazon-cloudwatch-agent.logNote: Logs might be specified in a custom logfile location. Check the agent configuration file to identify any custom log locations.In the agent configuration file, enable verbose debug logging using the debug parameter. If you're using the run_as_user parameter, confirm that the user has permissions to the log location path. Without the necessary permissions, the CloudWatch agent can't write logs to the location.Resolve timestamp issuesCheck for log event timestamps that are older than 14 days or more than two hours in the future. The PutLogEvents command doesn't allow log batches in either time frame.Also, verify that the system time service on the instance is correctly configured. For more information, see Set the time for your Linux instance or Set the time for a Windows instance.Related informationTroubleshooting the CloudWatch agentFollow"