{"action":"created","issue":{"url":"https://api.github.com/repos/SnabbCo/snabbswitch/issues/588","labels_url":"https://api.github.com/repos/SnabbCo/snabbswitch/issues/588/labels{/name}","comments_url":"https://api.github.com/repos/SnabbCo/snabbswitch/issues/588/comments","events_url":"https://api.github.com/repos/SnabbCo/snabbswitch/issues/588/events","html_url":"https://github.com/SnabbCo/snabbswitch/issues/588","id":99637043,"number":588,"title":"Dockerization of CI","user":{"login":"lukego","id":13791,"avatar_url":"https://avatars.githubusercontent.com/u/13791?v=3","gravatar_id":"","url":"https://api.github.com/users/lukego","html_url":"https://github.com/lukego","followers_url":"https://api.github.com/users/lukego/followers","following_url":"https://api.github.com/users/lukego/following{/other_user}","gists_url":"https://api.github.com/users/lukego/gists{/gist_id}","starred_url":"https://api.github.com/users/lukego/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lukego/subscriptions","organizations_url":"https://api.github.com/users/lukego/orgs","repos_url":"https://api.github.com/users/lukego/repos","events_url":"https://api.github.com/users/lukego/events{/privacy}","received_events_url":"https://api.github.com/users/lukego/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/SnabbCo/snabbswitch/labels/enhancement","name":"enhancement","color":"84b6eb"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-08-07T12:16:17Z","updated_at":"2015-08-31T12:26:53Z","closed_at":null,"body":"I propose that we Dockerize our test and CI infrastructure. That is, we would use Docker to make the test setup completely bootstrappable (everything done with Dockerfiles) and also instantly installable (as a versioned prebuilt binary from http://hub.docker.com/). Then that we actually execute CI tests using Dockerfiles and expand the set of hardware and configurations that we test.\r\n\r\nThis is following on from related threads:\r\n\r\n1. #574 troubleshooting of an NFV test scenario.\r\n2. [Docker for the lab?](https://groups.google.com/forum/#!topic/snabb-devel/zSWRH3Zq5Mo) on snabb-devel.\r\n3. [Nix, NixOS, NixOps](https://groups.google.com/forum/#!topic/snabb-devel/8BTftZL0oMo) on snabb-devel.\r\n\r\nEverything that we have done by hand would be automated. It would become taboo to rely on instructions in README files and manually created binaries: our new religion would be automation and reproducibility.\r\n\r\nThis requires work and especially for the NFV application:\r\n\r\n1. Create a `Dockerfile` that builds a complete NFV guest operating system. For example with a specific Linux kernel version, kernel configuration, and Ubuntu version. (This would be generalized to support more kernels, kernel configurations, distros, and operating systems.) The build results (e.g. `bzImage` and `disk.img`) would be stored read-only inside the container and could be accessed externally from other containers via Docker \"volumes\". The container with completed build would be uploaded to http://hub.docker.com/ as free storage of the image files with proper naming and versioning.\r\n2. Change `test_env` to access image files via a Docker \"volume\" (that the user can override e.g. for Ubuntu vs FreeBSD guest) and to use the system-installed `qemu` (etc) instead of manually building its own.\r\n3. Create a `Dockerfile` to run the CI test suite (or a well-defined subset thereof). The \"build\" phase of this container would install all necessary software including building a specific version of QEMU. The \"run\" phase would checkout Snabb Switch (version provided as env/arg) and execute the tests and post the results to Github. See [draft Dockerfile](https://gist.github.com/lukego/850a037c7d9ecdf594af).\r\n\r\n@eugeneia you might be interested in taking the lead on this? (or on shooting the idea down :-))"},"comment":{"url":"https://api.github.com/repos/SnabbCo/snabbswitch/issues/comments/136358697","html_url":"https://github.com/SnabbCo/snabbswitch/issues/588#issuecomment-136358697","issue_url":"https://api.github.com/repos/SnabbCo/snabbswitch/issues/588","id":136358697,"user":{"login":"eugeneia","id":4933566,"avatar_url":"https://avatars.githubusercontent.com/u/4933566?v=3","gravatar_id":"","url":"https://api.github.com/users/eugeneia","html_url":"https://github.com/eugeneia","followers_url":"https://api.github.com/users/eugeneia/followers","following_url":"https://api.github.com/users/eugeneia/following{/other_user}","gists_url":"https://api.github.com/users/eugeneia/gists{/gist_id}","starred_url":"https://api.github.com/users/eugeneia/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eugeneia/subscriptions","organizations_url":"https://api.github.com/users/eugeneia/orgs","repos_url":"https://api.github.com/users/eugeneia/repos","events_url":"https://api.github.com/users/eugeneia/events{/privacy}","received_events_url":"https://api.github.com/users/eugeneia/received_events","type":"User","site_admin":false},"created_at":"2015-08-31T12:26:53Z","updated_at":"2015-08-31T12:26:53Z","body":"After tinkering on this for a while here is a progress update. My working branch can be inspected  here: https://github.com/SnabbCo/snabbswitch/compare/master...eugeneia:simplify-test\r\n\r\nTo just see the docker related changes see: https://github.com/SnabbCo/snabbswitch/commit/03ba0a710808fc4dbd8bd5eb84901e8f4e57e4d9\r\n\r\nFirst of all a summary of whats done (docker-wise):\r\n\r\n* There is a new top-level target `make docker` that will build a docker image containing a full Snabb Switch test environment.\r\n* In `src/` you can call `scripts/dock.sh <command>` to run `command` on the current Snabb Switch tree within a fresh docker container based on mentioned image.\r\n\r\nIssues I have encountered include:\r\n\r\n* The [Dockerfile](https://github.com/SnabbCo/snabbswitch/blob/03ba0a710808fc4dbd8bd5eb84901e8f4e57e4d9/Dockerfile) is small but the [script to build the test assets](https://github.com/SnabbCo/snabbswitch/blob/03ba0a710808fc4dbd8bd5eb84901e8f4e57e4d9/src/scripts/make-assets.sh) is \"big\". The reason is that the asset VM images are not docker images and docker is not designed to build \"child images\" (it would be useful if there was an easy way to turn docker images into disk images to be used by qemu). To build the VM images we use `mount -o loop ...`, `chroot` and possibly other (e.g. `mkfs`?) privileged commands. Since `docker build` can not be run in privileged mode, we need to invoke the asset building in a `docker run` followed by a `docker commit`. See https://github.com/docker/docker/issues/1916\r\n* The original idea of using volumes to share test assets does not work out, because volumes are bound to containers (instances of images) and can not be shared on DockerHub which is for sharing images.\r\n* I see a 30% performance degradation in the packetblaster/dpdk benchmark when run in docker. I have not yet been able to figure out why.\r\n* The resulting docker image is 5GB..."}}