{"action":"created","issue":{"url":"https://api.github.com/repos/code-dot-org/code-dot-org/issues/4207","labels_url":"https://api.github.com/repos/code-dot-org/code-dot-org/issues/4207/labels{/name}","comments_url":"https://api.github.com/repos/code-dot-org/code-dot-org/issues/4207/comments","events_url":"https://api.github.com/repos/code-dot-org/code-dot-org/issues/4207/events","html_url":"https://github.com/code-dot-org/code-dot-org/pull/4207","id":108384485,"number":4207,"title":"CloudFront CDN configuration and Varnish refactor","user":{"login":"wjordan","id":56541,"avatar_url":"https://avatars.githubusercontent.com/u/56541?v=3","gravatar_id":"","url":"https://api.github.com/users/wjordan","html_url":"https://github.com/wjordan","followers_url":"https://api.github.com/users/wjordan/followers","following_url":"https://api.github.com/users/wjordan/following{/other_user}","gists_url":"https://api.github.com/users/wjordan/gists{/gist_id}","starred_url":"https://api.github.com/users/wjordan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wjordan/subscriptions","organizations_url":"https://api.github.com/users/wjordan/orgs","repos_url":"https://api.github.com/users/wjordan/repos","events_url":"https://api.github.com/users/wjordan/events{/privacy}","received_events_url":"https://api.github.com/users/wjordan/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"philbogle","id":5198586,"avatar_url":"https://avatars.githubusercontent.com/u/5198586?v=3","gravatar_id":"","url":"https://api.github.com/users/philbogle","html_url":"https://github.com/philbogle","followers_url":"https://api.github.com/users/philbogle/followers","following_url":"https://api.github.com/users/philbogle/following{/other_user}","gists_url":"https://api.github.com/users/philbogle/gists{/gist_id}","starred_url":"https://api.github.com/users/philbogle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/philbogle/subscriptions","organizations_url":"https://api.github.com/users/philbogle/orgs","repos_url":"https://api.github.com/users/philbogle/repos","events_url":"https://api.github.com/users/philbogle/events{/privacy}","received_events_url":"https://api.github.com/users/philbogle/received_events","type":"User","site_admin":false},"milestone":null,"comments":7,"created_at":"2015-09-25T18:36:39Z","updated_at":"2015-10-02T21:06:30Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/code-dot-org/code-dot-org/pulls/4207","html_url":"https://github.com/code-dot-org/code-dot-org/pull/4207","diff_url":"https://github.com/code-dot-org/code-dot-org/pull/4207.diff","patch_url":"https://github.com/code-dot-org/code-dot-org/pull/4207.patch"},"body":"1. Introduces an application-specific HTTP cache-configuration hash that will be used for provisioning/configuring our HTTP cache layers moving forward.\r\n\r\n2. Refactors the Varnish configuration to take advantage of the cache-configuration hash, which separates the application-specific configuration from the Varnish-specific VCL code. I've also taken this refactoring opportunity to upgrade Varnish to 4.0.3 which is the [current-supported release](https://www.varnish-cache.org/releases) (Varnish 3 was retired in April 2015).\r\n\r\n3. Introduces a CloudFront CDN configuration script which uses the HTTP cache-configuration hash to create/update an environment-specific CloudFront distribution using the AWS SDK for Ruby.\r\n\r\n4. Adds a set of Chef integration tests to the refactored `cdo-varnish` cookbook, to ensure that no regressions have been introduced during the refactoring.\r\n\r\n5. `Rack::SslEnforcer` middleware will be added to all environments including production, to ensure that the HTTP `Strict-Transport-Security` response header is set from the application layer rather than from the HTTP-cache layer (this was previously being served by Varnish in non-development environments). HTTPS redirection is still handled by the HTTP-cache layer (CloudFront or Varnish) in non-development environments.\r\n\r\nAside from the middleware change, none of these changes affect the web-application code directly. The CloudFront CDN and Varnish cookbook are two separate pieces that are deployed independently of our webapp-deployment process. Steps for each rollout are documented below:\r\n\r\n## CloudFront ([wiki](http://wiki.code.org/display/PROD/CDN+setup))\r\n\r\nSetup steps for CloudFront CDN rollout across environments:\r\n \r\n(In the below instructions, replace `ENV` with the environment name, e.g. `ENV-studio.code.org` is `staging-studio.code.org` in staging, with the exceptions of code.org and studio.code.org in production. `[num]`/`[abc]`/`[xyz]` are placeholders for arbitrary ID strings.)\r\n\r\n- Run the `update_cloudfront` script with `RAILS_ENV` and AWS credentials provided in the environment. This will create/update the environment-specific CloudFront distribution based on the latest HTTP cache configuration in the codebase.\r\n- In Route 53, add 2 new `A ALIAS` entries for the dashboard/pegasus ELB endpoints, that will serve as the origin domains for the CloudFront distributions:\r\n```\r\nENV-dashboard.code.org A ALIAS dualstack.ENV-dashboard-[num].us-east-1.elb.amazonaws.com.\r\nENV-pegasus.code.org A ALIAS dualstack.ENV-pegasus-[num].us-east-1.elb.amazonaws.com.\r\n```\r\n- Test that the new hostnames serve the correct content when loading `ENV-dashboard.code.org` and `ENV-pegasus.code.org` directly.\r\n- To switch on the CDN integration, update the `A` entries from the existing ELB endpoints to the corresponding CloudFront ones:\r\n```\r\nENV-studio.code.org A ALIAS [abc].cloudfront.net.\r\nENV.code.org A ALIAS [xyz].cloudfront.net.\r\n```\r\nNote: When testing, it might be a good idea to set the TTL to a relatively low number of seconds, so that a rollback will be applied without too much delay if necessary.\r\n\r\n- To rollback the CDN integration, simply update the `A` entries back to point directly to the ELB:\r\n```\r\nENV-studio.code.org A ALIAS dualstack.ENV-dashboard-[num].us-east-1.elb.amazonaws.com.\r\nENV.code.org A ALIAS dualstack.ENV-pegasus-[num].us-east-1.elb.amazonaws.com.\r\n```\r\n\r\n## Varnish ([wiki](http://wiki.code.org/display/PROD/Chef-managed+Configuration+Changes))\r\nFor the Varnish update rollout I plan to test out a new versioned-cookbook deployment process:\r\n\r\n- First, set the cookbook constraint to the existing cookbook version in all relevant environments one of these three ways:\r\n  - Adjust the `cookbook_versions` property from `knife environment edit [ENV]`;\r\n  - Adjust the \"cookbook constraints\" section in the `policy->environments` section of the Chef management console;\r\n  - Running `knife cookbook upload [cookbook] -E [ENV]` with the existing cookbook present.\r\n- Increase the cookbook version in `metadata.rb`, and make all cookbook changes.\r\n- Upload the newly-versioned cookbook and update the environment-specific cookbook constraints one at a time, through one of the three ways listed above."},"comment":{"url":"https://api.github.com/repos/code-dot-org/code-dot-org/issues/comments/145155362","html_url":"https://github.com/code-dot-org/code-dot-org/pull/4207#issuecomment-145155362","issue_url":"https://api.github.com/repos/code-dot-org/code-dot-org/issues/4207","id":145155362,"user":{"login":"philbogle","id":5198586,"avatar_url":"https://avatars.githubusercontent.com/u/5198586?v=3","gravatar_id":"","url":"https://api.github.com/users/philbogle","html_url":"https://github.com/philbogle","followers_url":"https://api.github.com/users/philbogle/followers","following_url":"https://api.github.com/users/philbogle/following{/other_user}","gists_url":"https://api.github.com/users/philbogle/gists{/gist_id}","starred_url":"https://api.github.com/users/philbogle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/philbogle/subscriptions","organizations_url":"https://api.github.com/users/philbogle/orgs","repos_url":"https://api.github.com/users/philbogle/repos","events_url":"https://api.github.com/users/philbogle/events{/privacy}","received_events_url":"https://api.github.com/users/philbogle/received_events","type":"User","site_admin":false},"created_at":"2015-10-02T21:06:30Z","updated_at":"2015-10-02T21:06:30Z","body":"Looking now."}}