How do I troubleshoot issues when I configure SSM Agent to use a proxy for managing my Amazon EC2 instance?
I receive errors when I use a proxy with SSM Agent to manage my Amazon Elastic Compute Cloud (Amazon EC2) instance with AWS Systems Manager. How can I troubleshoot proxy issues?
"I receive errors when I use a proxy with SSM Agent to manage my Amazon Elastic Compute Cloud (Amazon EC2) instance with AWS Systems Manager. How can I troubleshoot proxy issues?Short descriptionYou can use Systems Manager to manage your Amazon EC2 instances. However, if the instances are behind a proxy, then you must configure SSM Agent to communicate with the Amazon endpoints through your proxy. If SSM Agent isn’t correctly configured, you receive an error message similar to the following:Post https://ssm.RegionID.amazonaws.com/: proxyconnect tcp: dial tcp xxx.xxx.xxx.xxx:yyyy: i/o timeoutResolutionTo troubleshoot proxy issues, follow these steps:Verify that the EC2 instance can access your proxy serverEC2 Windows instancesYou can use the following Windows PowerShell command to verify connectivity to your proxy. In the following example, replace hostname with your proxy hostname, and replace port with your proxy port.> Test-NetConnection hostname -port portExpected output:ComputerName     : hostnameRemoteAddress    : xxx.xxx.xxx.xxxRemotePort       : portInterfaceAlias   : EthernetSourceAddress    : YYY.YYY.YYY.YYYTcpTestSucceeded : TruePS C:\Windows\system32>EC2 Linux instancesYou can use the following telnet or netcat commands to verify connectivity to your proxy. In the following examples, replace hostname with your proxy hostname, and replace port with your proxy port.Telnet$ telnet hostname portExpected output:Trying xxx.xxx.xxx.xxx...Connected to hostname.Escape character is '^]'.Netcat$ nc -vz hostname port'Expected output:Ncat: Version 7.50 ( https://nmap.org/ncat )Ncat: Connected to xxx.xxx.xxx.xxx:YYYY.Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds.Note: Netcat doesn't come preinstalled on Amazon EC2 instances. To manually install Netcat, see Ncat on the Nmap website.Verify that your EC2 instance can reach the Systems Manager endpoint through the proxy serverRun the following command to confirm that your instance can reach the Systems Manager endpoint through your proxy server. Receiving an http error code 4xx indicates that you can reach the endpoint through your proxy server.Note: In the following command examples, replace RegionID with your AWS Region, replace hostname with your proxy hostname, and replace port with your proxy port.EC2 Windows instances> (invoke-webrequest https://ssm.RegionID.amazonaws.com -DisableKeepAlive -UseBasicParsing -Method head -Proxy http://hostname:port )Expected output:invoke-webrequest : The remote server returned an error: (404) Not Found.EC2 Linux instances$  curl -k --proxy http://hostname:port -m 5 -s -o /dev/null -w "%{http_code}" https://ssm.RegionID.amazonaws.comExpected output:404Verify that SSM Agent is configured to use your proxy informationEC2 Windows instancesYou must review the SSM Agent log to verify that the proxy settings are applied as shown in the following example. Enter the following command:> type C:\ProgramData\Amazon\SSM\Logs\amazon-ssm-agent.log | findstr -i "proxy"Expected output:2021-03-18 19:06:16 INFO Getting IE proxy configuration for current user: The operation completed successfully.2021-03-18 19:06:16 INFO Getting WinHTTP proxy default configuration: The operation completed successfully.2021-03-18 19:06:16 INFO Proxy environment variables:2021-03-18 19:06:16 INFO http_proxy: hostname:port2021-03-18 19:06:16 INFO https_proxy:2021-03-18 19:06:16 INFO no_proxy: 169.254.169.254For more information, see Configure SSM Agent to use a proxy for Windows Server instances.EC2 Linux instancesFirst, run the following command to verify whether SSM Agent currently uses the required proxy variables by checking the process environment variable:sudo cat /proc/$(pidof amazon-ssm-agent)/environ | xargs -0 -L1 -I{} echo {}Expected output:[root@host123~]# sudo cat /proc/$(pidof amazon-ssm-agent)/environ | xargs -0 -L1 -I{} echo {}LANG=en_US.UTF-8PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/binhttp_proxy=http://Hostname:Porthttps_proxy=http://Hostname:Portno_proxy=169.254.169.254Then, if the output does not include the proxy variables, follow the guidance below for your instance type to verify and update the relevant files.EC2 Linux instances based on an upstart such as Ubuntu 14.04Verify that the amazon-ssm-agent.override file contains the required configurations. Enter the following command and confirm that your output matches the example expected output. For more information, see Configure SSM Agent to use a proxy (upstart).Important: If you update the amazon-ssm-agent.override file, then be sure to restart SSM Agent after the file is edited.$ cat  /etc/init/amazon-ssm-agent.overrideExpected output for an HTTP proxy server:env http_proxy=http://hostname:portenv https_proxy=http://hostname:portenv no_proxy=169.254.169.254Expected output for an HTTPS proxy server:env http_proxy=http://hostname:portenv https_proxy=https://hostname:portenv no_proxy=169.254.169.254EC2 Linux instances based on Ubuntu 16.04 or later with SSM Agent installed using a snapEnter the following command and confirm that the proxy information matches the example expected output. For more information, see Configure SSM Agent to use a proxy (systemd).Important: If you update the amazon-ssm-agent.override file, then be sure to restart SSM Agent after the file is edited.$ cat /etc/systemd/system/snap.amazon-ssm-agent.amazon-ssm-agent.service.d/override.confExpected output for an HTTP proxy server:[Service]Environment="http_proxy=http://hostname:port"Environment="https_proxy=http://hostname:port"Environment="no_proxy=169.254.169.254"Expected output for an HTTPS proxy server:[Service]Environment="http_proxy=http://hostname:port"Environment="https_proxy=https://hostname:port"Environment="no_proxy=169.254.169.254"EC2 Linux instances based on Amazon Linux 2Enter the following command and confirm that the proxy information matches the example expected output. For more information, see Configure SSM Agent to use a proxy (systemd).Important: If you update the amazon-ssm-agent.override file, then be sure to restart SSM Agent after the file is edited.$ cat /etc/systemd/system/amazon-ssm-agent.service.d/override.confExpected output for an HTTP proxy server:[Service]Environment="http_proxy=http://hostname:port"Environment="https_proxy=http://hostname:port"Environment="no_proxy=169.254.169.254"Expected output for an HTTPS proxy server:[Service]Environment="http_proxy=http://hostname:port"Environment="https_proxy=https://hostname:port"Environment="no_proxy=169.254.169.254"EC2 Linux instances based on other operating systemsEnter the following command and confirm that the proxy information matches the example expected output. For more information, see Configure SSM Agent to use a proxy (systemd).Important: If you update the amazon-ssm-agent.override file, then be sure to restart SSM Agent after the file is edited.$ cat /etc/systemd/system/amazon-ssm-agent.service.d/amazon-ssm-agent.overrideExpected output for an HTTP proxy server:[Service]Environment="http_proxy=http://hostname:port"Environment="https_proxy=http://hostname:port"Environment="no_proxy=169.254.169.254"Expected output for an HTTPS proxy server:[Service]Environment="http_proxy=http://hostname:port"Environment="https_proxy=https://hostname:port"Environment="no_proxy=169.254.169.254"Related informationWorking with SSM AgentTroubleshooting SSM AgentFollow"