{"action":"created","issue":{"url":"https://api.github.com/repos/bokeh/bokeh/issues/2528","labels_url":"https://api.github.com/repos/bokeh/bokeh/issues/2528/labels{/name}","comments_url":"https://api.github.com/repos/bokeh/bokeh/issues/2528/comments","events_url":"https://api.github.com/repos/bokeh/bokeh/issues/2528/events","html_url":"https://github.com/bokeh/bokeh/issues/2528","id":93850096,"number":2528,"title":"Selenium Testing w/ Browserstack","user":{"login":"mistakevin","id":4411115,"avatar_url":"https://avatars.githubusercontent.com/u/4411115?v=3","gravatar_id":"","url":"https://api.github.com/users/mistakevin","html_url":"https://github.com/mistakevin","followers_url":"https://api.github.com/users/mistakevin/followers","following_url":"https://api.github.com/users/mistakevin/following{/other_user}","gists_url":"https://api.github.com/users/mistakevin/gists{/gist_id}","starred_url":"https://api.github.com/users/mistakevin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mistakevin/subscriptions","organizations_url":"https://api.github.com/users/mistakevin/orgs","repos_url":"https://api.github.com/users/mistakevin/repos","events_url":"https://api.github.com/users/mistakevin/events{/privacy}","received_events_url":"https://api.github.com/users/mistakevin/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/bokeh/bokeh/labels/type:%20discussion","name":"type: discussion","color":"2659A0"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-07-08T17:50:32Z","updated_at":"2015-07-13T15:33:18Z","closed_at":null,"body":"Using Selenium Webdriver along with the Selenium IDE and [Browserstack](browserstack.com) gives a great workflow for testing in the browser. The [Selenium IDE](http://docs.seleniumhq.org/download/) (scroll down until you see \"Selenium IDE\") is a FireFox add-on that lets you record actions in the browser and assert elements are present by locating and interacting with DOM elements:\r\n![screen shot 2015-07-08 at 12 37 37 pm](https://cloud.githubusercontent.com/assets/4411115/8577419/2bacc100-256e-11e5-8b2d-68fd5d7370d3.png)\r\nThis will export python code. With a few adjustments for browserstack, will look like [this](https://github.com/bokeh/bokeh/blob/mistakevin/feature/selenium/bokeh/tests/selenium/test.py)\r\n\r\nThen when ran you are able to watch a visual log (or actually interact with!) the test as it is happening on browserstack.com:\r\n![selenium](https://cloud.githubusercontent.com/assets/4411115/8577505/c483536c-256e-11e5-9b5a-a1591ee07a71.gif)\r\n\r\nAs well as in the terminal:\r\n```\r\n12:23 $ python bokeh/tests/selenium/test.py\r\n.\r\n----------------------------------------------------------------------\r\nRan 1 test in 15.321s\r\n\r\nOK\r\n```\r\n\r\nWe can set this up on our own without browserstack. In the past, I have used [CircleCI](circleci.com) for all testing by launching a selenium server and actually making a video recording of the DOM interaction, viewable for every build. The selenium server is clunky though, and setting it up with multiple versions of multiple browsers on multiple OSs would be a nightmare, which is where browserstack comes in. Browserstack also supports local testing as seen in the gif."},"comment":{"url":"https://api.github.com/repos/bokeh/bokeh/issues/comments/120969893","html_url":"https://github.com/bokeh/bokeh/issues/2528#issuecomment-120969893","issue_url":"https://api.github.com/repos/bokeh/bokeh/issues/2528","id":120969893,"user":{"login":"mistakevin","id":4411115,"avatar_url":"https://avatars.githubusercontent.com/u/4411115?v=3","gravatar_id":"","url":"https://api.github.com/users/mistakevin","html_url":"https://github.com/mistakevin","followers_url":"https://api.github.com/users/mistakevin/followers","following_url":"https://api.github.com/users/mistakevin/following{/other_user}","gists_url":"https://api.github.com/users/mistakevin/gists{/gist_id}","starred_url":"https://api.github.com/users/mistakevin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mistakevin/subscriptions","organizations_url":"https://api.github.com/users/mistakevin/orgs","repos_url":"https://api.github.com/users/mistakevin/repos","events_url":"https://api.github.com/users/mistakevin/events{/privacy}","received_events_url":"https://api.github.com/users/mistakevin/received_events","type":"User","site_admin":false},"created_at":"2015-07-13T15:33:18Z","updated_at":"2015-07-13T15:33:18Z","body":"Saucelabs would be a great alternative, and pro-opensource is always a great stance. Browerstack also has [free testing for open source projects](https://www.browserstack.com/accounts/subscriptions).\r\n\r\nI was initially pulled toward Browserstack due to the [local testing](https://www.browserstack.com/local-testing) feature, keeping all of us developers from having more environment setup, and being able to rapidly test our branch on demand. Though I just saw that Saucelabs also shares that [functionality](https://docs.saucelabs.com/reference/sauce-connect/)!\r\n\r\nAt my last job, we had a non-programming QA team that we taught how to use the Selenium IDE. We ran PHPUnit tests which (awesomely) ran native Selenese 1 (HTML) tests. These worked about 95% of the time on the Selenium Server, and the main issues we had were timing related (selenium does not integrate with javascript, so no callbacks) and were largely patched by adding a few millisecond pause.\r\n\r\nThough, we should definitely keep to tweaking in python. The workflow I was thinking was to build out the skeleton test in the IDE, export it to python, and tweak it while remotely testing our branch.\r\n\r\nAlso, Saucelabs makes a [Selenium Builder](https://docs.saucelabs.com/reference/selenium-builder/) extension for FireFox that builds Selenium 1 & 2 and can [run it remotely](https://docs.saucelabs.com/reference/selenium-builder/#running-scripts-on-sauce-labs) with a click of a button without exporting anything, so that you can test faster before ultimately exporting to python.\r\n\r\nAs for testing canvas, we can dive into [fabric.js](http://fabricjs.com/) which features a Canvas-to-SVG parser. SVG elements would be Selenium testable, and would offer the feature to users to export to SVG. But I'm not sure how much of a work it would take to convert our existing canvas infrastructure."}}