{"action":"created","issue":{"url":"https://api.github.com/repos/GSA/analytics.usa.gov/issues/111","labels_url":"https://api.github.com/repos/GSA/analytics.usa.gov/issues/111/labels{/name}","comments_url":"https://api.github.com/repos/GSA/analytics.usa.gov/issues/111/comments","events_url":"https://api.github.com/repos/GSA/analytics.usa.gov/issues/111/events","html_url":"https://github.com/GSA/analytics.usa.gov/issues/111","id":54735671,"number":111,"title":"*All* top domains over 7 days","user":{"login":"konklone","id":4592,"avatar_url":"https://avatars.githubusercontent.com/u/4592?v=3","gravatar_id":"","url":"https://api.github.com/users/konklone","html_url":"https://github.com/konklone","followers_url":"https://api.github.com/users/konklone/followers","following_url":"https://api.github.com/users/konklone/following{/other_user}","gists_url":"https://api.github.com/users/konklone/gists{/gist_id}","starred_url":"https://api.github.com/users/konklone/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/konklone/subscriptions","organizations_url":"https://api.github.com/users/konklone/orgs","repos_url":"https://api.github.com/users/konklone/repos","events_url":"https://api.github.com/users/konklone/events{/privacy}","received_events_url":"https://api.github.com/users/konklone/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-01-19T08:01:14Z","updated_at":"2015-01-20T02:55:43Z","closed_at":null,"body":"The new filtered profile should have 7 days' worth of data pretty soon. We can then export a pretty nice list of every domain that appears in it. It's well below 10,000, which is the maximum set size. And many have 0 visits, for some reason, so you could filter them to `visits>0` and cut it down further.\r\n\r\nSince this is a great research dataset, it should be available in CSV and JSON. We can include it in the data listing, even if it's not turned into some epic graph.\r\n\r\nThere are staging domains and stuff in here, but any of those sites could just refrain from doing DAP reporting from non-final domains."},"comment":{"url":"https://api.github.com/repos/GSA/analytics.usa.gov/issues/comments/70597068","html_url":"https://github.com/GSA/analytics.usa.gov/issues/111#issuecomment-70597068","issue_url":"https://api.github.com/repos/GSA/analytics.usa.gov/issues/111","id":70597068,"user":{"login":"konklone","id":4592,"avatar_url":"https://avatars.githubusercontent.com/u/4592?v=3","gravatar_id":"","url":"https://api.github.com/users/konklone","html_url":"https://github.com/konklone","followers_url":"https://api.github.com/users/konklone/followers","following_url":"https://api.github.com/users/konklone/following{/other_user}","gists_url":"https://api.github.com/users/konklone/gists{/gist_id}","starred_url":"https://api.github.com/users/konklone/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/konklone/subscriptions","organizations_url":"https://api.github.com/users/konklone/orgs","repos_url":"https://api.github.com/users/konklone/repos","events_url":"https://api.github.com/users/konklone/events{/privacy}","received_events_url":"https://api.github.com/users/konklone/received_events","type":"User","site_admin":false},"created_at":"2015-01-20T02:55:43Z","updated_at":"2015-01-20T02:55:43Z","body":"Totally agree that it needs discussion with the DAP team - and, looking at the data, it'll need a bit more effort than the other reports if we want it to feel cleaner (like removing IP addresses). I'm going to send some samples to the DAP team, as I'm not sure the new profile is doing as much whitelisting as they're hoping for.\r\n\r\nStill, I want to make sure we get some sort of substantive data release out soon. The data we've got so far is great and people are going to love it, but it also isn't deep enough for anyone to do any interesting research on. We'll get some very different wins out of a larger dataset than we will the live snapshots, and this one seems to me like the highest ROI of that nature."}}