{"action":"created","issue":{"url":"https://api.github.com/repos/coreos/rkt/issues/1042","labels_url":"https://api.github.com/repos/coreos/rkt/issues/1042/labels{/name}","comments_url":"https://api.github.com/repos/coreos/rkt/issues/1042/comments","events_url":"https://api.github.com/repos/coreos/rkt/issues/1042/events","html_url":"https://github.com/coreos/rkt/issues/1042","id":88407928,"number":1042,"title":"rkt fails on Debian GNU/Linux 7 (wheezy) with unclear error","user":{"login":"alban","id":27575,"avatar_url":"https://avatars.githubusercontent.com/u/27575?v=3","gravatar_id":"","url":"https://api.github.com/users/alban","html_url":"https://github.com/alban","followers_url":"https://api.github.com/users/alban/followers","following_url":"https://api.github.com/users/alban/following{/other_user}","gists_url":"https://api.github.com/users/alban/gists{/gist_id}","starred_url":"https://api.github.com/users/alban/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alban/subscriptions","organizations_url":"https://api.github.com/users/alban/orgs","repos_url":"https://api.github.com/users/alban/repos","events_url":"https://api.github.com/users/alban/events{/privacy}","received_events_url":"https://api.github.com/users/alban/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/coreos/rkt/labels/bug","name":"bug","color":"fc2929"},{"url":"https://api.github.com/repos/coreos/rkt/labels/stage1","name":"stage1","color":"bfd4f2"},{"url":"https://api.github.com/repos/coreos/rkt/labels/ux","name":"ux","color":"fef2c0"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":8,"created_at":"2015-06-15T12:19:31Z","updated_at":"2015-06-18T13:21:49Z","closed_at":null,"body":"Debian GNU/Linux 7 (wheezy)\r\n\r\nIt fails without error message:\r\n\r\n```\r\n$ sudo bin/rkt --insecure-skip-verify   --debug run -interactive docker://busybox\r\nrkt: fetching image from file:///tmp/rkt-ci-debian78/bin/stage1.aci\r\nrkt: found image in local store, skipping fetching from docker://busybox\r\n2015/06/15 12:06:41 Preparing stage1\r\n2015/06/15 12:06:45 Writing image manifest\r\n2015/06/15 12:06:45 Loading image sha512-e17fc826c76437429a085f2c276d3feb4312dd67f57390cefaf582739e8c8c19\r\n2015/06/15 12:06:46 Writing pod manifest\r\n2015/06/15 12:06:46 Setting up stage1\r\n2015/06/15 12:06:46 Wrote filesystem to /var/lib/rkt/pods/run/3393cb47-43c5-4185-ac68-c85b98692f33\r\n2015/06/15 12:06:46 Pivoting to filesystem /var/lib/rkt/pods/run/3393cb47-43c5-4185-ac68-c85b98692f33\r\n2015/06/15 12:06:46 Execing /init\r\nSpawning container rkt-3393cb47-43c5-4185-ac68-c85b98692f33 on /var/lib/rkt/pods/run/3393cb47-43c5-4185-ac68-c85b98692f33/stage1/rootfs.\r\nPress ^] three times within 1s to kill container.\r\n$ echo $?\r\n1\r\n```\r\n\r\nWhen using `SYSTEMD_LOG_LEVEL=debug`, the error message is unclear:\r\n\r\n```\r\n$ sudo SYSTEMD_LOG_LEVEL=debug bin/rkt --insecure-skip-verify   --debug run -interactive docker://busybox\r\n(...)\r\nPress ^] three times within 1s to kill container.\r\nnamespace helper failed with error code 1.\r\n```\r\n\r\nWhen using strace, we see that the nspawn error is related to cgroups:\r\n\r\n```\r\n[pid 19528] open(\"/proc/cgroups\", O_RDONLY|O_CLOEXEC) = 5\r\n[pid 19528] fstat(5, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0\r\n[pid 19528] mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f798f4d7000\r\n[pid 19528] read(5, \"#subsys_name\\thierarchy\\tnum_cgroups\\tenabled\\ncpuset\\t4\\t34\\t1\\ncpu\\t2\\t34\\t1\\ncpuacct\\t3\\t34\\t1\\nmemory\\t0\\t1\\t0\\ndevices\\t5\\t34\\t1\\nfreezer\\t6\\t34\\t1\\nnet_cls\\t7\\t3\r\n4\\t1\\nblkio\\t1\\t34\\t1\\nperf_event\\t9\\t34\\t1\\nnet_prio\\t8\\t34\\t1\\n\", 1024) = 188\r\n[pid 19528] open(\"/dev/urandom\", O_RDONLY|O_NOCTTY|O_CLOEXEC) = 8\r\n[pid 19528] read(8, \"\\30\\36\\226\\30\\177\\317[\\372J\\241\\367\\31h#\\323t\", 16) = 16\r\n[pid 19528] close(8)                    = 0\r\n[pid 19528] read(5, \"\", 1024)           = 0\r\n[pid 19528] close(5)                    = 0\r\n[pid 19528] munmap(0x7f798f4d7000, 4096) = 0\r\n[pid 19528] readlinkat(AT_FDCWD, \"/sys/fs/cgroup/blkio\", 0x7f7990330920, 99) = -1 ENOENT (No such file or directory)\r\n[pid 19528] writev(2, [{\"Failed to read link /sys/fs/cgroup/blkio: No such file or directory\", 67}, {\"\\n\", 1}], 2) = 68\r\n[pid 19528] exit_group(1)               = ?\r\n```\r\n\r\nIndeed, the cgroup subsystems are not mounted:\r\n```\r\n$ cat /proc/cgroups \r\n#subsys_name    hierarchy       num_cgroups     enabled\r\ncpuset  4       34      1\r\ncpu     2       34      1\r\ncpuacct 3       34      1\r\nmemory  0       1       0\r\ndevices 5       34      1\r\nfreezer 6       34      1\r\nnet_cls 7       34      1\r\nblkio   1       34      1\r\nperf_event      9       34      1\r\nnet_prio        8       34      1\r\n$ ls -l /sys/fs/cgroup/\r\ntotal 0\r\ndrwxr-xr-x 2 root root 60 Jun 15 10:21 cgmanager\r\n```\r\n\r\nSimilar issue to #996.\r\n\r\nThe solution might be on https://wiki.debian.org/LXC#Prepare_the_host but rkt+nspawn should give better error messages."},"comment":{"url":"https://api.github.com/repos/coreos/rkt/issues/comments/113154089","html_url":"https://github.com/coreos/rkt/issues/1042#issuecomment-113154089","issue_url":"https://api.github.com/repos/coreos/rkt/issues/1042","id":113154089,"user":{"login":"alban","id":27575,"avatar_url":"https://avatars.githubusercontent.com/u/27575?v=3","gravatar_id":"","url":"https://api.github.com/users/alban","html_url":"https://github.com/alban","followers_url":"https://api.github.com/users/alban/followers","following_url":"https://api.github.com/users/alban/following{/other_user}","gists_url":"https://api.github.com/users/alban/gists{/gist_id}","starred_url":"https://api.github.com/users/alban/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alban/subscriptions","organizations_url":"https://api.github.com/users/alban/orgs","repos_url":"https://api.github.com/users/alban/repos","events_url":"https://api.github.com/users/alban/events{/privacy}","received_events_url":"https://api.github.com/users/alban/received_events","type":"User","site_admin":false},"created_at":"2015-06-18T13:21:49Z","updated_at":"2015-06-18T13:21:49Z","body":"The missing error message is fixed by https://github.com/systemd/systemd/issues/207 https://github.com/systemd/systemd/pull/272. It will be in systemd-v221.\r\n\r\n\r\nI now have correct error messages:\r\n\r\n```\r\nSpawning container rkt-8fa79c17-35ea-48bb-b13a-9d1b2257fef1 on /tmp/datadir-712183490/pods/run/8fa79c17-35ea-48bb-b13a-9d1b2257fef1/stage1/rootfs.\r\nPress ^] three times within 1s to kill container.\r\n/etc/localtime is not a symlink, not updating container timezone.\r\n\r\nFailed to read link /sys/fs/cgroup/freezer: No such file or directory\r\n```\r\n\r\nI don't know if we should do something else about this."}}