{"action":"created","issue":{"url":"https://api.github.com/repos/hashbangcode/vlad/issues/190","labels_url":"https://api.github.com/repos/hashbangcode/vlad/issues/190/labels{/name}","comments_url":"https://api.github.com/repos/hashbangcode/vlad/issues/190/comments","events_url":"https://api.github.com/repos/hashbangcode/vlad/issues/190/events","html_url":"https://github.com/hashbangcode/vlad/issues/190","id":65688973,"number":190,"title":"Automated Testing","user":{"login":"philipnorton42","id":150767,"avatar_url":"https://avatars.githubusercontent.com/u/150767?v=3","gravatar_id":"","url":"https://api.github.com/users/philipnorton42","html_url":"https://github.com/philipnorton42","followers_url":"https://api.github.com/users/philipnorton42/followers","following_url":"https://api.github.com/users/philipnorton42/following{/other_user}","gists_url":"https://api.github.com/users/philipnorton42/gists{/gist_id}","starred_url":"https://api.github.com/users/philipnorton42/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/philipnorton42/subscriptions","organizations_url":"https://api.github.com/users/philipnorton42/orgs","repos_url":"https://api.github.com/users/philipnorton42/repos","events_url":"https://api.github.com/users/philipnorton42/events{/privacy}","received_events_url":"https://api.github.com/users/philipnorton42/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2015-04-01T13:52:23Z","updated_at":"2015-04-06T20:43:55Z","closed_at":null,"body":"Ok, so now that we have a password free version of Vlad we can start some automated testing. I would prefer some way of testing every commit and getting a report if anything went wrong. Future plans would be to supply different settings files but for now I just want to test a default install.\r\n\r\n_Travis CI_\r\nI've tried (for a few hours) to get Travis CI to do set up a vagrant environment and run Vlad. This doesn't work. There is nothing specifically saying 'Virtualbox/Vagrant is not supported' but after trying all sorts of things to get it working there is simply no way this can work. What is annoying is that it ~almost~ works. Travis CI is a great service though and I was impressed by what it did do.\r\n\r\n_Jenkins_\r\nThis is definitely possible, but might take a little bit of setup, and will most likely be self hosted. The difficulty here is that we can't use a VM to run the tests and as such the hosting costs are quite high. We could set it up on a local machine somewhere, but I don't currently have one spare and it might even be hosted on my broadband connection. Frankly, I don't really want to have to support a public facing machine like this so I would prefer some form of hosted solution.\r\n\r\n_Anything else_\r\nSo, are there any hosted solutions that would allow us to test a vagrant build? I went for Travis CI originally, but it doesn't support what we need so if people can post potential solutions then I'm happy to setup the integrations."},"comment":{"url":"https://api.github.com/repos/hashbangcode/vlad/issues/comments/90233860","html_url":"https://github.com/hashbangcode/vlad/issues/190#issuecomment-90233860","issue_url":"https://api.github.com/repos/hashbangcode/vlad/issues/190","id":90233860,"user":{"login":"philipnorton42","id":150767,"avatar_url":"https://avatars.githubusercontent.com/u/150767?v=3","gravatar_id":"","url":"https://api.github.com/users/philipnorton42","html_url":"https://github.com/philipnorton42","followers_url":"https://api.github.com/users/philipnorton42/followers","following_url":"https://api.github.com/users/philipnorton42/following{/other_user}","gists_url":"https://api.github.com/users/philipnorton42/gists{/gist_id}","starred_url":"https://api.github.com/users/philipnorton42/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/philipnorton42/subscriptions","organizations_url":"https://api.github.com/users/philipnorton42/orgs","repos_url":"https://api.github.com/users/philipnorton42/repos","events_url":"https://api.github.com/users/philipnorton42/events{/privacy}","received_events_url":"https://api.github.com/users/philipnorton42/received_events","type":"User","site_admin":false},"created_at":"2015-04-06T20:43:55Z","updated_at":"2015-04-06T20:43:55Z","body":"Ok, after much messing about (and actually breaking Vlad in a couple of places) I have reverted the automated running of the ansible tasks and just left the syntax checking bit in. I'd started to go down the route of altering Vlad to allow the tests to run, which isn't really ideal :)\r\n\r\nCurrently, travis will check to make sure that there are no syntax errors in the code, which is great. But we still need to figure out some way of running an end to end test on a system in a automated (or semi-automated) way."}}