{"action":"created","issue":{"url":"https://api.github.com/repos/pseudomuto/riquid/issues/2","labels_url":"https://api.github.com/repos/pseudomuto/riquid/issues/2/labels{/name}","comments_url":"https://api.github.com/repos/pseudomuto/riquid/issues/2/comments","events_url":"https://api.github.com/repos/pseudomuto/riquid/issues/2/events","html_url":"https://github.com/pseudomuto/riquid/pull/2","id":111900511,"number":2,"title":"Tokenization is now a thing!","user":{"login":"pseudomuto","id":4748863,"avatar_url":"https://avatars.githubusercontent.com/u/4748863?v=3","gravatar_id":"","url":"https://api.github.com/users/pseudomuto","html_url":"https://github.com/pseudomuto","followers_url":"https://api.github.com/users/pseudomuto/followers","following_url":"https://api.github.com/users/pseudomuto/following{/other_user}","gists_url":"https://api.github.com/users/pseudomuto/gists{/gist_id}","starred_url":"https://api.github.com/users/pseudomuto/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pseudomuto/subscriptions","organizations_url":"https://api.github.com/users/pseudomuto/orgs","repos_url":"https://api.github.com/users/pseudomuto/repos","events_url":"https://api.github.com/users/pseudomuto/events{/privacy}","received_events_url":"https://api.github.com/users/pseudomuto/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-10-16T19:59:23Z","updated_at":"2015-10-16T20:02:18Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/pseudomuto/riquid/pulls/2","html_url":"https://github.com/pseudomuto/riquid/pull/2","diff_url":"https://github.com/pseudomuto/riquid/pull/2.diff","patch_url":"https://github.com/pseudomuto/riquid/pull/2.patch"},"body":"@tahnok \r\n\r\nAfter waaaay too much time, I was able to get a tokenizer set up. It works similarly to Ruby's `String#split` in that it will split the string using the provided `Regex`, but also include the results in the split.\r\n\r\nSo for example, ` {% comment %} My comment {% endcomment %} Some Text\" will break down to:\r\n\r\n* `\" \"`\r\n* `\"{% comment %}\"`\r\n* `\"  My comment \"`\r\n* `\"{% endcomment %}`\r\n* `\" Some Text\"`\r\n\r\nThere's nothing here that enforces any kind of grammar. We can tackle that when we get to the parser.\r\n\r\n**The Shitty Bit**\r\n\r\nFor now, it collects the mapped string slices since dealing with `Map<Iter<'t, (usize, usize)>, FnMut(blah blah blah` was fighteningly difficult. Eventually we can work on pushing the collect call further up the call stack."},"comment":{"url":"https://api.github.com/repos/pseudomuto/riquid/issues/comments/148820315","html_url":"https://github.com/pseudomuto/riquid/pull/2#issuecomment-148820315","issue_url":"https://api.github.com/repos/pseudomuto/riquid/issues/2","id":148820315,"user":{"login":"pseudomuto","id":4748863,"avatar_url":"https://avatars.githubusercontent.com/u/4748863?v=3","gravatar_id":"","url":"https://api.github.com/users/pseudomuto","html_url":"https://github.com/pseudomuto","followers_url":"https://api.github.com/users/pseudomuto/followers","following_url":"https://api.github.com/users/pseudomuto/following{/other_user}","gists_url":"https://api.github.com/users/pseudomuto/gists{/gist_id}","starred_url":"https://api.github.com/users/pseudomuto/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pseudomuto/subscriptions","organizations_url":"https://api.github.com/users/pseudomuto/orgs","repos_url":"https://api.github.com/users/pseudomuto/repos","events_url":"https://api.github.com/users/pseudomuto/events{/privacy}","received_events_url":"https://api.github.com/users/pseudomuto/received_events","type":"User","site_admin":false},"created_at":"2015-10-16T20:02:18Z","updated_at":"2015-10-16T20:02:18Z","body":"@ibawt on a scale from :hankey: to :100: how amazing is this?"}}