{"action":"created","issue":{"url":"https://api.github.com/repos/syl20bnr/spacemacs/issues/2454","labels_url":"https://api.github.com/repos/syl20bnr/spacemacs/issues/2454/labels{/name}","comments_url":"https://api.github.com/repos/syl20bnr/spacemacs/issues/2454/comments","events_url":"https://api.github.com/repos/syl20bnr/spacemacs/issues/2454/events","html_url":"https://github.com/syl20bnr/spacemacs/pull/2454","id":97931508,"number":2454,"title":"Extended testing framework + modeline tests","user":{"login":"TheBB","id":619375,"avatar_url":"https://avatars.githubusercontent.com/u/619375?v=3","gravatar_id":"","url":"https://api.github.com/users/TheBB","html_url":"https://github.com/TheBB","followers_url":"https://api.github.com/users/TheBB/followers","following_url":"https://api.github.com/users/TheBB/following{/other_user}","gists_url":"https://api.github.com/users/TheBB/gists{/gist_id}","starred_url":"https://api.github.com/users/TheBB/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TheBB/subscriptions","organizations_url":"https://api.github.com/users/TheBB/orgs","repos_url":"https://api.github.com/users/TheBB/repos","events_url":"https://api.github.com/users/TheBB/events{/privacy}","received_events_url":"https://api.github.com/users/TheBB/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2015-07-29T13:09:21Z","updated_at":"2015-07-31T23:24:56Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/syl20bnr/spacemacs/pulls/2454","html_url":"https://github.com/syl20bnr/spacemacs/pull/2454","diff_url":"https://github.com/syl20bnr/spacemacs/pull/2454.diff","patch_url":"https://github.com/syl20bnr/spacemacs/pull/2454.patch"},"body":"This is just me playing around with ideas for #2440, #110.\r\n\r\nWith this PR, to add tests to a layer, you have to do three things:\r\n\r\n1. Make a subdirectory of `tests` corresponding to your layer, with some test files. (See `/tests/core` or `/tests/spacemacs` in this PR for examples).\r\n2. Write an `init.el` in that directory as well. This should be a minimal `.spacemacs` file for enabling your layer. See `/tests/spacemacs/init.el` for the extreme case of nothing.\r\n3. Write a `Makefile` in that directory that lists the files you just wrote. See `/tests/core/Makefile` and `/tests/spacemacs/Makefile` for examples. This should be literally nothing other than copying `/spacemacs/Makefile` (don't do the one from core), replacing the files with the actual files you wrote in step 1. You may have to add some more `../` to the `include` as well though. The `LOAD_FILES` are relative to `~/.emacs.d/` (and there should be a very good reason if this is not just `init.el`), while the `TEST_FILES` are relative to the test directory.\r\n4. Add the layer to the list of tests in `travis/run_build.sh`.\r\n\r\nTo run tests, you should go into the test directory for the layer you want to test and execute `make test`. This will run the tests naïvely. That is, it just runs the setup you currently have on your computer, and it ignores the `init.el` file you wrote in step 2. If you don't have the layer enabled in your personal dotfile, for example, you should expect the tests to fail. This is done to prevent testing from making changes in the developer's `~/.emacs.d/elpa`. It's intended as a quickie mode compared to Travis.\r\n\r\nTravis, on the other hand, will go through core as well as all layers (skipping those without a makefile) to run all tests. For each layer, it will run emacs once with the environment variable `SPACEMACSDIR` set to `/tests/<layer>/`. This should make it choose the right dotfile for each test. This should **not** be done in each makefile individually, because that would cause test runs locally to mess up the elpa folder!\r\n\r\nAdditionally, Travis will delete the `elpa` and `.cache` folders between each layer test. Is this necessary? Well… maybe not. Spacemacs should of course ensure that the relevant packages are available for all runs, even if you use a different dotfile. I could probably remove this if we see the CI builds take too long. Right now it takes just short of a minute to do the whole build as it stands on this branch, which includes:\r\n\r\n* Cloning the repo\r\n* Installing emacs\r\n* Running the core tests\r\n* Installing all the packages required by the spacemacs layer\r\n\r\n**Notes:**\r\n\r\n* As it stands, this PR doesn't include any actual _tests_, except for a dummy that does basically nothing.\r\n* I have added the `ci` branch to the Travis file for testing (the testing), but I can remove that when it's time to merge.\r\n* Coverage would be cool. But later. :-D\r\n\r\nComments? I don't expect this to be the \"done deal\" but it's perhaps a decent starting point for discussion."},"comment":{"url":"https://api.github.com/repos/syl20bnr/spacemacs/issues/comments/126834136","html_url":"https://github.com/syl20bnr/spacemacs/pull/2454#issuecomment-126834136","issue_url":"https://api.github.com/repos/syl20bnr/spacemacs/issues/2454","id":126834136,"user":{"login":"robbyoconnor","id":23088,"avatar_url":"https://avatars.githubusercontent.com/u/23088?v=3","gravatar_id":"","url":"https://api.github.com/users/robbyoconnor","html_url":"https://github.com/robbyoconnor","followers_url":"https://api.github.com/users/robbyoconnor/followers","following_url":"https://api.github.com/users/robbyoconnor/following{/other_user}","gists_url":"https://api.github.com/users/robbyoconnor/gists{/gist_id}","starred_url":"https://api.github.com/users/robbyoconnor/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/robbyoconnor/subscriptions","organizations_url":"https://api.github.com/users/robbyoconnor/orgs","repos_url":"https://api.github.com/users/robbyoconnor/repos","events_url":"https://api.github.com/users/robbyoconnor/events{/privacy}","received_events_url":"https://api.github.com/users/robbyoconnor/received_events","type":"User","site_admin":false},"created_at":"2015-07-31T23:24:56Z","updated_at":"2015-07-31T23:24:56Z","body":"@justbur -- why not make a linter for `.spacemacs` -- rather than full blown tests? "}}