{"action":"created","comment":{"url":"https://api.github.com/repos/swift-nav/libswiftnav/pulls/comments/32778029","id":32778029,"diff_hunk":"@@ -67,12 +68,73 @@ START_TEST(test_lsq)\n }\n END_TEST\n \n+START_TEST(test_sos_innov)","path":"tests/check_amb_kf.c","position":12,"original_position":12,"commit_id":"7b0841a27fc3ead592e4e40f8e2f7a5a0a3765bb","original_commit_id":"8702ceb0be20c936b2f216faf05cf27bb297c657","user":{"login":"imh","id":391217,"avatar_url":"https://avatars.githubusercontent.com/u/391217?v=3","gravatar_id":"","url":"https://api.github.com/users/imh","html_url":"https://github.com/imh","followers_url":"https://api.github.com/users/imh/followers","following_url":"https://api.github.com/users/imh/following{/other_user}","gists_url":"https://api.github.com/users/imh/gists{/gist_id}","starred_url":"https://api.github.com/users/imh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/imh/subscriptions","organizations_url":"https://api.github.com/users/imh/orgs","repos_url":"https://api.github.com/users/imh/repos","events_url":"https://api.github.com/users/imh/events{/privacy}","received_events_url":"https://api.github.com/users/imh/received_events","type":"User","site_admin":false},"body":"In the case of this function, at least, it's just wrapping a bit of matrix algebra together. I think that would be overkill.","created_at":"2015-06-18T21:01:53Z","updated_at":"2015-06-18T21:01:53Z","html_url":"https://github.com/swift-nav/libswiftnav/pull/168#discussion_r32778029","pull_request_url":"https://api.github.com/repos/swift-nav/libswiftnav/pulls/168","_links":{"self":{"href":"https://api.github.com/repos/swift-nav/libswiftnav/pulls/comments/32778029"},"html":{"href":"https://github.com/swift-nav/libswiftnav/pull/168#discussion_r32778029"},"pull_request":{"href":"https://api.github.com/repos/swift-nav/libswiftnav/pulls/168"}}},"pull_request":{"url":"https://api.github.com/repos/swift-nav/libswiftnav/pulls/168","id":37414214,"html_url":"https://github.com/swift-nav/libswiftnav/pull/168","diff_url":"https://github.com/swift-nav/libswiftnav/pull/168.diff","patch_url":"https://github.com/swift-nav/libswiftnav/pull/168.patch","issue_url":"https://api.github.com/repos/swift-nav/libswiftnav/issues/168","number":168,"state":"open","locked":false,"title":"Added outlier detection and robustness to the KF and amb test","user":{"login":"imh","id":391217,"avatar_url":"https://avatars.githubusercontent.com/u/391217?v=3","gravatar_id":"","url":"https://api.github.com/users/imh","html_url":"https://github.com/imh","followers_url":"https://api.github.com/users/imh/followers","following_url":"https://api.github.com/users/imh/following{/other_user}","gists_url":"https://api.github.com/users/imh/gists{/gist_id}","starred_url":"https://api.github.com/users/imh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/imh/subscriptions","organizations_url":"https://api.github.com/users/imh/orgs","repos_url":"https://api.github.com/users/imh/repos","events_url":"https://api.github.com/users/imh/events{/privacy}","received_events_url":"https://api.github.com/users/imh/received_events","type":"User","site_admin":false},"body":"Because of cycle slips, false locks, and general mismodelling brought about by changing satellite sets, I opted not to use a data editing scheme, which would have allowed the possibility of locking up the filter forever and worked poorly in testing. Instead, I opted to use a more robust measurement update that essentially allows for heavier tails in the measurement model. If the measurement is good enough, it will be acted upon as normal, and the effect of this change is continuous and smooth. \"Good\" here refers to an adaptively scaled weighted sum of squares of KF innovations, weighted by their variances.\r\nThe KF also now returns whether it thought this data point was an outlier.\r\n\r\n### Sacrifices made:\r\n- We weight the SOS by their variances instead of by the whole covariance. We saved a few matrix multiplies and the solution of a system of equations.\r\n- We are scaling by a slightly heuristic term, with connections to Huber's minimax estimators, but the connection is still kinda loose and ad hoc. It was simpler to implement and seems good enough.\r\n- We implement the KF gain scaling in a slightly not-statistically-optimal way, but again, it seems good enough. The important thing here is that the covariances still update properly to match the mean.\r\n\r\n### Some things to note:\r\n- I got about 1 in 1000 false positives in early testing (SITL). They are mostly independent and therefore spaced out.\r\n- I use an adaptive filter to decide the scale of what is and isn't an outlier, which dramatically decreases the incidence of long series of false positives.\r\n- The timescale of adaptation is quite short, so this handles cycle slips/change points quite well, but won't handle long streams of outliers well. That said, there was an order of magnitude between the cutoff and the smallest outlier in my (small) training set. If we decrease the frequency of cycle slips, we can choose a longer timescale.\r\n- When using a median filter rather than basic 1st (0th?) order IIR for the adaptation in the classifier, the median had a slightly lower false positive rate (FPR) for the same same FNR with less sensitivity between FNR and FPR. However, the median filter would have been more expensive and harder to implement.\r\n\r\n### Plots/metrics:\r\nIn this plot, we are using the short timescale IIR with false positive rate a little over 1e-3, basically where it hooks over:\r\n![fpr versus safety](https://cloud.githubusercontent.com/assets/391217/8096775/8ad16482-0f8f-11e5-921b-b14602ff5a51.png)\r\n\r\nTo connect a point on those curves to an actual cutoff, we consult this graph (this is for the short timeframe IIR, used in this PR):\r\n![fpr versus cutoff](https://cloud.githubusercontent.com/assets/391217/8097238/9f722dae-0f94-11e5-983b-c4e5bdcbfa73.png)\r\n\r\n\r\ncc @kovach, @fnoble, @mookerji \r\n","created_at":"2015-06-10T23:05:23Z","updated_at":"2015-06-18T21:01:53Z","closed_at":null,"merged_at":null,"merge_commit_sha":"6f1ffe35ec81bb174dbb5958a040dae9a9eb4ee6","assignee":{"login":"imh","id":391217,"avatar_url":"https://avatars.githubusercontent.com/u/391217?v=3","gravatar_id":"","url":"https://api.github.com/users/imh","html_url":"https://github.com/imh","followers_url":"https://api.github.com/users/imh/followers","following_url":"https://api.github.com/users/imh/following{/other_user}","gists_url":"https://api.github.com/users/imh/gists{/gist_id}","starred_url":"https://api.github.com/users/imh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/imh/subscriptions","organizations_url":"https://api.github.com/users/imh/orgs","repos_url":"https://api.github.com/users/imh/repos","events_url":"https://api.github.com/users/imh/events{/privacy}","received_events_url":"https://api.github.com/users/imh/received_events","type":"User","site_admin":false},"milestone":null,"commits_url":"https://api.github.com/repos/swift-nav/libswiftnav/pulls/168/commits","review_comments_url":"https://api.github.com/repos/swift-nav/libswiftnav/pulls/168/comments","review_comment_url":"https://api.github.com/repos/swift-nav/libswiftnav/pulls/comments{/number}","comments_url":"https://api.github.com/repos/swift-nav/libswiftnav/issues/168/comments","statuses_url":"https://api.github.com/repos/swift-nav/libswiftnav/statuses/7b0841a27fc3ead592e4e40f8e2f7a5a0a3765bb","head":{"label":"imh:kf_robustness","ref":"kf_robustness","sha":"7b0841a27fc3ead592e4e40f8e2f7a5a0a3765bb","user":{"login":"imh","id":391217,"avatar_url":"https://avatars.githubusercontent.com/u/391217?v=3","gravatar_id":"","url":"https://api.github.com/users/imh","html_url":"https://github.com/imh","followers_url":"https://api.github.com/users/imh/followers","following_url":"https://api.github.com/users/imh/following{/other_user}","gists_url":"https://api.github.com/users/imh/gists{/gist_id}","starred_url":"https://api.github.com/users/imh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/imh/subscriptions","organizations_url":"https://api.github.com/users/imh/orgs","repos_url":"https://api.github.com/users/imh/repos","events_url":"https://api.github.com/users/imh/events{/privacy}","received_events_url":"https://api.github.com/users/imh/received_events","type":"User","site_admin":false},"repo":{"id":18895865,"name":"libswiftnav","full_name":"imh/libswiftnav","owner":{"login":"imh","id":391217,"avatar_url":"https://avatars.githubusercontent.com/u/391217?v=3","gravatar_id":"","url":"https://api.github.com/users/imh","html_url":"https://github.com/imh","followers_url":"https://api.github.com/users/imh/followers","following_url":"https://api.github.com/users/imh/following{/other_user}","gists_url":"https://api.github.com/users/imh/gists{/gist_id}","starred_url":"https://api.github.com/users/imh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/imh/subscriptions","organizations_url":"https://api.github.com/users/imh/orgs","repos_url":"https://api.github.com/users/imh/repos","events_url":"https://api.github.com/users/imh/events{/privacy}","received_events_url":"https://api.github.com/users/imh/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/imh/libswiftnav","description":"A portable C library implementing GNSS related functions and algorithms.","fork":true,"url":"https://api.github.com/repos/imh/libswiftnav","forks_url":"https://api.github.com/repos/imh/libswiftnav/forks","keys_url":"https://api.github.com/repos/imh/libswiftnav/keys{/key_id}","collaborators_url":"https://api.github.com/repos/imh/libswiftnav/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/imh/libswiftnav/teams","hooks_url":"https://api.github.com/repos/imh/libswiftnav/hooks","issue_events_url":"https://api.github.com/repos/imh/libswiftnav/issues/events{/number}","events_url":"https://api.github.com/repos/imh/libswiftnav/events","assignees_url":"https://api.github.com/repos/imh/libswiftnav/assignees{/user}","branches_url":"https://api.github.com/repos/imh/libswiftnav/branches{/branch}","tags_url":"https://api.github.com/repos/imh/libswiftnav/tags","blobs_url":"https://api.github.com/repos/imh/libswiftnav/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/imh/libswiftnav/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/imh/libswiftnav/git/refs{/sha}","trees_url":"https://api.github.com/repos/imh/libswiftnav/git/trees{/sha}","statuses_url":"https://api.github.com/repos/imh/libswiftnav/statuses/{sha}","languages_url":"https://api.github.com/repos/imh/libswiftnav/languages","stargazers_url":"https://api.github.com/repos/imh/libswiftnav/stargazers","contributors_url":"https://api.github.com/repos/imh/libswiftnav/contributors","subscribers_url":"https://api.github.com/repos/imh/libswiftnav/subscribers","subscription_url":"https://api.github.com/repos/imh/libswiftnav/subscription","commits_url":"https://api.github.com/repos/imh/libswiftnav/commits{/sha}","git_commits_url":"https://api.github.com/repos/imh/libswiftnav/git/commits{/sha}","comments_url":"https://api.github.com/repos/imh/libswiftnav/comments{/number}","issue_comment_url":"https://api.github.com/repos/imh/libswiftnav/issues/comments{/number}","contents_url":"https://api.github.com/repos/imh/libswiftnav/contents/{+path}","compare_url":"https://api.github.com/repos/imh/libswiftnav/compare/{base}...{head}","merges_url":"https://api.github.com/repos/imh/libswiftnav/merges","archive_url":"https://api.github.com/repos/imh/libswiftnav/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/imh/libswiftnav/downloads","issues_url":"https://api.github.com/repos/imh/libswiftnav/issues{/number}","pulls_url":"https://api.github.com/repos/imh/libswiftnav/pulls{/number}","milestones_url":"https://api.github.com/repos/imh/libswiftnav/milestones{/number}","notifications_url":"https://api.github.com/repos/imh/libswiftnav/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/imh/libswiftnav/labels{/name}","releases_url":"https://api.github.com/repos/imh/libswiftnav/releases{/id}","created_at":"2014-04-17T22:50:38Z","updated_at":"2015-05-05T17:55:26Z","pushed_at":"2015-06-16T22:59:12Z","git_url":"git://github.com/imh/libswiftnav.git","ssh_url":"git@github.com:imh/libswiftnav.git","clone_url":"https://github.com/imh/libswiftnav.git","svn_url":"https://github.com/imh/libswiftnav","homepage":"","size":8042,"stargazers_count":0,"watchers_count":0,"language":"C","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"swift-nav:master","ref":"master","sha":"24f85e95aaaebbea61543b57ceb66238d0e2e410","user":{"login":"swift-nav","id":1069835,"avatar_url":"https://avatars.githubusercontent.com/u/1069835?v=3","gravatar_id":"","url":"https://api.github.com/users/swift-nav","html_url":"https://github.com/swift-nav","followers_url":"https://api.github.com/users/swift-nav/followers","following_url":"https://api.github.com/users/swift-nav/following{/other_user}","gists_url":"https://api.github.com/users/swift-nav/gists{/gist_id}","starred_url":"https://api.github.com/users/swift-nav/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/swift-nav/subscriptions","organizations_url":"https://api.github.com/users/swift-nav/orgs","repos_url":"https://api.github.com/users/swift-nav/repos","events_url":"https://api.github.com/users/swift-nav/events{/privacy}","received_events_url":"https://api.github.com/users/swift-nav/received_events","type":"Organization","site_admin":false},"repo":{"id":7901696,"name":"libswiftnav","full_name":"swift-nav/libswiftnav","owner":{"login":"swift-nav","id":1069835,"avatar_url":"https://avatars.githubusercontent.com/u/1069835?v=3","gravatar_id":"","url":"https://api.github.com/users/swift-nav","html_url":"https://github.com/swift-nav","followers_url":"https://api.github.com/users/swift-nav/followers","following_url":"https://api.github.com/users/swift-nav/following{/other_user}","gists_url":"https://api.github.com/users/swift-nav/gists{/gist_id}","starred_url":"https://api.github.com/users/swift-nav/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/swift-nav/subscriptions","organizations_url":"https://api.github.com/users/swift-nav/orgs","repos_url":"https://api.github.com/users/swift-nav/repos","events_url":"https://api.github.com/users/swift-nav/events{/privacy}","received_events_url":"https://api.github.com/users/swift-nav/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/swift-nav/libswiftnav","description":"A portable C library implementing GNSS related functions and algorithms.","fork":false,"url":"https://api.github.com/repos/swift-nav/libswiftnav","forks_url":"https://api.github.com/repos/swift-nav/libswiftnav/forks","keys_url":"https://api.github.com/repos/swift-nav/libswiftnav/keys{/key_id}","collaborators_url":"https://api.github.com/repos/swift-nav/libswiftnav/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/swift-nav/libswiftnav/teams","hooks_url":"https://api.github.com/repos/swift-nav/libswiftnav/hooks","issue_events_url":"https://api.github.com/repos/swift-nav/libswiftnav/issues/events{/number}","events_url":"https://api.github.com/repos/swift-nav/libswiftnav/events","assignees_url":"https://api.github.com/repos/swift-nav/libswiftnav/assignees{/user}","branches_url":"https://api.github.com/repos/swift-nav/libswiftnav/branches{/branch}","tags_url":"https://api.github.com/repos/swift-nav/libswiftnav/tags","blobs_url":"https://api.github.com/repos/swift-nav/libswiftnav/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/swift-nav/libswiftnav/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/swift-nav/libswiftnav/git/refs{/sha}","trees_url":"https://api.github.com/repos/swift-nav/libswiftnav/git/trees{/sha}","statuses_url":"https://api.github.com/repos/swift-nav/libswiftnav/statuses/{sha}","languages_url":"https://api.github.com/repos/swift-nav/libswiftnav/languages","stargazers_url":"https://api.github.com/repos/swift-nav/libswiftnav/stargazers","contributors_url":"https://api.github.com/repos/swift-nav/libswiftnav/contributors","subscribers_url":"https://api.github.com/repos/swift-nav/libswiftnav/subscribers","subscription_url":"https://api.github.com/repos/swift-nav/libswiftnav/subscription","commits_url":"https://api.github.com/repos/swift-nav/libswiftnav/commits{/sha}","git_commits_url":"https://api.github.com/repos/swift-nav/libswiftnav/git/commits{/sha}","comments_url":"https://api.github.com/repos/swift-nav/libswiftnav/comments{/number}","issue_comment_url":"https://api.github.com/repos/swift-nav/libswiftnav/issues/comments{/number}","contents_url":"https://api.github.com/repos/swift-nav/libswiftnav/contents/{+path}","compare_url":"https://api.github.com/repos/swift-nav/libswiftnav/compare/{base}...{head}","merges_url":"https://api.github.com/repos/swift-nav/libswiftnav/merges","archive_url":"https://api.github.com/repos/swift-nav/libswiftnav/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/swift-nav/libswiftnav/downloads","issues_url":"https://api.github.com/repos/swift-nav/libswiftnav/issues{/number}","pulls_url":"https://api.github.com/repos/swift-nav/libswiftnav/pulls{/number}","milestones_url":"https://api.github.com/repos/swift-nav/libswiftnav/milestones{/number}","notifications_url":"https://api.github.com/repos/swift-nav/libswiftnav/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/swift-nav/libswiftnav/labels{/name}","releases_url":"https://api.github.com/repos/swift-nav/libswiftnav/releases{/id}","created_at":"2013-01-29T21:32:13Z","updated_at":"2015-06-01T02:31:46Z","pushed_at":"2015-06-18T17:52:26Z","git_url":"git://github.com/swift-nav/libswiftnav.git","ssh_url":"git@github.com:swift-nav/libswiftnav.git","clone_url":"https://github.com/swift-nav/libswiftnav.git","svn_url":"https://github.com/swift-nav/libswiftnav","homepage":"","size":17228,"stargazers_count":33,"watchers_count":33,"language":"C","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":43,"mirror_url":null,"open_issues_count":54,"forks":43,"open_issues":54,"watchers":33,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/swift-nav/libswiftnav/pulls/168"},"html":{"href":"https://github.com/swift-nav/libswiftnav/pull/168"},"issue":{"href":"https://api.github.com/repos/swift-nav/libswiftnav/issues/168"},"comments":{"href":"https://api.github.com/repos/swift-nav/libswiftnav/issues/168/comments"},"review_comments":{"href":"https://api.github.com/repos/swift-nav/libswiftnav/pulls/168/comments"},"review_comment":{"href":"https://api.github.com/repos/swift-nav/libswiftnav/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/swift-nav/libswiftnav/pulls/168/commits"},"statuses":{"href":"https://api.github.com/repos/swift-nav/libswiftnav/statuses/7b0841a27fc3ead592e4e40f8e2f7a5a0a3765bb"}}}}