{"action":"opened","issue":{"url":"https://api.github.com/repos/docker/docker/issues/16376","labels_url":"https://api.github.com/repos/docker/docker/issues/16376/labels{/name}","comments_url":"https://api.github.com/repos/docker/docker/issues/16376/comments","events_url":"https://api.github.com/repos/docker/docker/issues/16376/events","html_url":"https://github.com/docker/docker/issues/16376","id":107020839,"number":16376,"title":"Docker aufs issues when running lots of `pull` and `run` commands in parallel","user":{"login":"zviadm","id":517937,"avatar_url":"https://avatars.githubusercontent.com/u/517937?v=3","gravatar_id":"","url":"https://api.github.com/users/zviadm","html_url":"https://github.com/zviadm","followers_url":"https://api.github.com/users/zviadm/followers","following_url":"https://api.github.com/users/zviadm/following{/other_user}","gists_url":"https://api.github.com/users/zviadm/gists{/gist_id}","starred_url":"https://api.github.com/users/zviadm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zviadm/subscriptions","organizations_url":"https://api.github.com/users/zviadm/orgs","repos_url":"https://api.github.com/users/zviadm/repos","events_url":"https://api.github.com/users/zviadm/events{/privacy}","received_events_url":"https://api.github.com/users/zviadm/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-09-17T16:18:17Z","updated_at":"2015-09-17T16:18:18Z","closed_at":null,"body":"I am running docker in a test environment, where there are multiple worker processes continuously pulling and running different containers.\r\n\r\nAfter some point, some of the machines start throwing errors like this one:\r\n```\r\nError response from daemon: error creating aufs mount to /mnt/docker/aufs/mnt/f2fca0b7ba13476a3f6b89ac986c9eae7e3876640dbe641439218a89c5875540-init: invalid argument\r\n```\r\n\r\nOr, this one:\r\n```\r\nEXEC[docker]: [pull XXXX:XXXX/isotester:main]\r\nmain: Pulling from isotester\r\n3c2870d005ed: Pulling fs layer\r\n3c2870d005ed: Pulling fs layer\r\n9d276fcfbea2: Already exists\r\n238ca624d2ea: Already exists\r\n8615058f3e1f: Already exists\r\n9610cfc68e8d: Already exists\r\n7d152d6a4550: Already exists\r\neb86f0592ac6: Already exists\r\n0f36a61b7011: Already exists\r\n859b7f97b67f: Already exists\r\ne960c44cc53d: Already exists\r\n19b3b8ea9fdb: Already exists\r\nbe95702c4500: Already exists\r\nc22e383b9bf7: Already exists\r\nfb68523634d7: Already exists\r\nff621d625f79: Already exists\r\n323ccbe757ac: Already exists\r\nc515664bbada: Already exists\r\n7d08fcb10882: Already exists\r\n9b8b262231c6: Already exists\r\ncd91592a40ca: Already exists\r\n07aee4985daa: Already exists\r\n7118b7235dc9: Already exists\r\n102840b21d2c: Already exists\r\n3c2870d005ed: Layer already being pulled by another client. Waiting.\r\n3c2870d005ed: Verifying Checksum\r\n3c2870d005ed: Download complete\r\n3c2870d005ed: Download complete\r\nPulling repository oak1b-i-880e6248.aws.dbxnw.net:5002/isotester\r\nError: image isotester:main not found\r\n```\r\n\r\nRestarting docker daemon makes the issues go away. Until it gets into this state again after few hours or days of running.\r\n\r\n```\r\nuname -a\r\nLinux oak1b-i-21d5a0e1 3.16.0-33-generic #44~14.04.1-Ubuntu SMP Fri Mar 13 10:33:29 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux\r\n```\r\n\r\n```\r\ndocker info\r\nContainers: 2\r\nImages: 78\r\nStorage Driver: aufs\r\n Root Dir: /mnt/docker/aufs\r\n Backing Filesystem: xfs\r\n Dirs: 87\r\n Dirperm1 Supported: true\r\nExecution Driver: native-0.2\r\nLogging Driver: json-file\r\nKernel Version: 3.16.0-33-generic\r\nOperating System: Ubuntu precise (12.04.5 LTS)\r\nCPUs: 32\r\nTotal Memory: 58.97 GiB\r\nName: XXXX\r\nID: 26V6:LVAB:DJ4R:GERJ:KS2T:IT3O:VLBE:KZUI:RF5I:57FF:BUCO:P3CU\r\nWARNING: No swap limit support\r\n```\r\n\r\nRunning on EC2 boxes.\r\n\r\n\r\nFollowing issues also seem potentially related:\r\nhttps://github.com/kubernetes/kubernetes/issues/10959\r\nhttps://github.com/docker/docker/issues/13742\r\n"}}