{"action":"created","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/1150","labels_url":"https://api.github.com/repos/fchollet/keras/issues/1150/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/1150/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/1150/events","html_url":"https://github.com/fchollet/keras/pull/1150","id":120101689,"number":1150,"title":"Speed up keras test infrastructure","user":{"login":"olegsinyavskiy","id":2086260,"avatar_url":"https://avatars.githubusercontent.com/u/2086260?v=3","gravatar_id":"","url":"https://api.github.com/users/olegsinyavskiy","html_url":"https://github.com/olegsinyavskiy","followers_url":"https://api.github.com/users/olegsinyavskiy/followers","following_url":"https://api.github.com/users/olegsinyavskiy/following{/other_user}","gists_url":"https://api.github.com/users/olegsinyavskiy/gists{/gist_id}","starred_url":"https://api.github.com/users/olegsinyavskiy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/olegsinyavskiy/subscriptions","organizations_url":"https://api.github.com/users/olegsinyavskiy/orgs","repos_url":"https://api.github.com/users/olegsinyavskiy/repos","events_url":"https://api.github.com/users/olegsinyavskiy/events{/privacy}","received_events_url":"https://api.github.com/users/olegsinyavskiy/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2015-12-03T06:10:51Z","updated_at":"2015-12-04T21:12:42Z","closed_at":"2015-12-03T18:34:44Z","pull_request":{"url":"https://api.github.com/repos/fchollet/keras/pulls/1150","html_url":"https://github.com/fchollet/keras/pull/1150","diff_url":"https://github.com/fchollet/keras/pull/1150.diff","patch_url":"https://github.com/fchollet/keras/pull/1150.patch"},"body":"Hello,\r\n\r\nThank you for this framework! It seems to be the best one on the market:) These changes speeds up travis testing time 2 times using some pytest and travis configuration options.\r\n\r\nSummary of changes:\r\n - py.test is configured to display test profiling information that shows 10 slowest tests. This would allow additional speed ups if anyone has ideas on some particular test. The slowest test is usually cifar dataset test and tensorflow convolutions. It seems that there are some other IT tests that could be sped up.\r\n - py.test is configured to run with pytest-xdist with 2 processes in parallel because travis does provide multicore support (1.5 cores) and because the slowest cifar test spends time on download which can run in parallel with other tests.\r\n - travis is configured to split backend tests into test matrix to make parallel theano vs tensorflow testing as opposed to rerun all the tests twice for python 2.7.\r\n - pickle filenames in tests are renamed to avoid clashes during multiprocessing\r\n\r\nThings that I tried or it would be nice to try in the future: \r\n - use travis container infrastructure that allows caching. This would speed up starting time and speed up dataset tests enormously. Unfortunately 14.04 is not supported with containers yet and tensorflow doesn't work on 12.04. It seems that tensorflow can be patched, but testing on 12.04 is a bad idea anyway.\r\n - create richer travis test matrix (for example, split core keras tests from integration and dataset tests)\r\n - do not test everything on every backend. Test everything of theano and have only core tests for tensorflow. At least it seems possible to avoid running IT tests (like datasets) on every backend since these are independent of the backend.\r\n - create shorter dataset versions and shorter IT tests (e.g. 5% of cifar images) with appropriate integration tests like trainers and examples that learn only part of the task.\r\n"},"comment":{"url":"https://api.github.com/repos/fchollet/keras/issues/comments/162083143","html_url":"https://github.com/fchollet/keras/pull/1150#issuecomment-162083143","issue_url":"https://api.github.com/repos/fchollet/keras/issues/1150","id":162083143,"user":{"login":"EderSantana","id":3902480,"avatar_url":"https://avatars.githubusercontent.com/u/3902480?v=3","gravatar_id":"","url":"https://api.github.com/users/EderSantana","html_url":"https://github.com/EderSantana","followers_url":"https://api.github.com/users/EderSantana/followers","following_url":"https://api.github.com/users/EderSantana/following{/other_user}","gists_url":"https://api.github.com/users/EderSantana/gists{/gist_id}","starred_url":"https://api.github.com/users/EderSantana/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/EderSantana/subscriptions","organizations_url":"https://api.github.com/users/EderSantana/orgs","repos_url":"https://api.github.com/users/EderSantana/repos","events_url":"https://api.github.com/users/EderSantana/events{/privacy}","received_events_url":"https://api.github.com/users/EderSantana/received_events","type":"User","site_admin":false},"created_at":"2015-12-04T21:12:42Z","updated_at":"2015-12-04T21:12:42Z","body":"@olegsinyavskiy we have several examples in the `examples` folder (that I personally copy and past a lot from). You can also write a tutorials section on the wiki. But if you plan something different let us know. What did you have in mind?"}}