{"action":"created","issue":{"url":"https://api.github.com/repos/projectcalico/calico-docker/issues/415","labels_url":"https://api.github.com/repos/projectcalico/calico-docker/issues/415/labels{/name}","comments_url":"https://api.github.com/repos/projectcalico/calico-docker/issues/415/comments","events_url":"https://api.github.com/repos/projectcalico/calico-docker/issues/415/events","html_url":"https://github.com/projectcalico/calico-docker/issues/415","id":98866908,"number":415,"title":"calico IP-in-IP tunnel device uses incorrect MTU in GCE","user":{"login":"Symmetric","id":1524910,"avatar_url":"https://avatars.githubusercontent.com/u/1524910?v=3","gravatar_id":"","url":"https://api.github.com/users/Symmetric","html_url":"https://github.com/Symmetric","followers_url":"https://api.github.com/users/Symmetric/followers","following_url":"https://api.github.com/users/Symmetric/following{/other_user}","gists_url":"https://api.github.com/users/Symmetric/gists{/gist_id}","starred_url":"https://api.github.com/users/Symmetric/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Symmetric/subscriptions","organizations_url":"https://api.github.com/users/Symmetric/orgs","repos_url":"https://api.github.com/users/Symmetric/repos","events_url":"https://api.github.com/users/Symmetric/events{/privacy}","received_events_url":"https://api.github.com/users/Symmetric/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/projectcalico/calico-docker/labels/bug","name":"bug","color":"fc2929"}],"state":"open","locked":false,"assignee":{"login":"tomdee","id":6635,"avatar_url":"https://avatars.githubusercontent.com/u/6635?v=3","gravatar_id":"","url":"https://api.github.com/users/tomdee","html_url":"https://github.com/tomdee","followers_url":"https://api.github.com/users/tomdee/followers","following_url":"https://api.github.com/users/tomdee/following{/other_user}","gists_url":"https://api.github.com/users/tomdee/gists{/gist_id}","starred_url":"https://api.github.com/users/tomdee/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tomdee/subscriptions","organizations_url":"https://api.github.com/users/tomdee/orgs","repos_url":"https://api.github.com/users/tomdee/repos","events_url":"https://api.github.com/users/tomdee/events{/privacy}","received_events_url":"https://api.github.com/users/tomdee/received_events","type":"User","site_admin":false},"milestone":null,"comments":5,"created_at":"2015-08-04T01:05:59Z","updated_at":"2015-08-05T08:32:26Z","closed_at":null,"body":"The tunl0 device uses MTU 1480 unconditionally. In GCE, the network MTU is 1460, so the correct MTU for the tunl0 device is 1440 in this environment.\r\n\r\nWith the incorrect value of 1480, packets are dropped after encapsulation, but before being transmitted on the VM's external interface. One might expect the kernel to generate an ICMP \"fragmentation required\" in this case, but that does not happen.\r\n\r\nThis issue is exacerbated by the kernel's TSO/GSO function, which can coalesce outbound packets into a single jumbo frame (to be segmented by the NIC) to improve transmission rates. This feature (enabled by default on most kernels, including GCE's Ubuntu and CoreOS images) can cause a burst of small packets (e.g. those produced by a traffic generator) to be glued into a single MTU-sized packet, which since the MTU is incorrect, will also be dropped after encapsulation.\r\n\r\nCheapest fix would be to pick an arbitrary number <= 1440 and use that as the tunl0 MTU in all cases. Could alternatively try to determine the value dynamically by inspecting the interfaces on the machine -- but that could be tricky. pMTU detection is available in the kernel, but this seems to be discouraged for security reasons."},"comment":{"url":"https://api.github.com/repos/projectcalico/calico-docker/issues/comments/127914605","html_url":"https://github.com/projectcalico/calico-docker/issues/415#issuecomment-127914605","issue_url":"https://api.github.com/repos/projectcalico/calico-docker/issues/415","id":127914605,"user":{"login":"plwhite","id":8448749,"avatar_url":"https://avatars.githubusercontent.com/u/8448749?v=3","gravatar_id":"","url":"https://api.github.com/users/plwhite","html_url":"https://github.com/plwhite","followers_url":"https://api.github.com/users/plwhite/followers","following_url":"https://api.github.com/users/plwhite/following{/other_user}","gists_url":"https://api.github.com/users/plwhite/gists{/gist_id}","starred_url":"https://api.github.com/users/plwhite/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/plwhite/subscriptions","organizations_url":"https://api.github.com/users/plwhite/orgs","repos_url":"https://api.github.com/users/plwhite/repos","events_url":"https://api.github.com/users/plwhite/events{/privacy}","received_events_url":"https://api.github.com/users/plwhite/received_events","type":"User","site_admin":false},"created_at":"2015-08-05T08:32:26Z","updated_at":"2015-08-05T08:32:26Z","body":"The solution Shaun came up with was to set the default MTU low enough to work with GCE. Thus although you might want to allow tuning it, you don't absolutely have to in order to get IP in IP to work."}}