{"action":"created","comment":{"url":"https://api.github.com/repos/quasar-analytics/quasar/pulls/comments/45005020","id":45005020,"diff_hunk":"@@ -0,0 +1,240 @@\n+package quasar\n+package fs\n+\n+import quasar.Predef._\n+import quasar.fp._\n+\n+import monocle.std.{disjunction => D}\n+\n+import scalaz._, Scalaz._\n+import scalaz.stream._\n+import pathy.Path._\n+\n+sealed trait WriteFile[A]\n+\n+object WriteFile {\n+  final case class WriteHandle(run: Long) extends AnyVal\n+\n+  object WriteHandle {\n+    implicit val writeHandleShow: Show[WriteHandle] =\n+      Show.showFromToString\n+\n+    implicit val writeHandleOrder: Order[WriteHandle] =\n+      Order.orderBy(_.run)\n+  }\n+\n+  final case class Open(file: AbsFile[Sandboxed])\n+    extends WriteFile[FileSystemError \\/ WriteHandle]\n+\n+  final case class Write(h: WriteHandle, chunk: Vector[Data])\n+    extends WriteFile[Vector[FileSystemError]]\n+\n+  final case class Close(h: WriteHandle)\n+    extends WriteFile[Unit]\n+\n+  final class Ops[S[_]](implicit S0: Functor[S], S1: WriteFileF :<: S) {\n+    import FileSystemError._, PathError2._\n+    import ManageFile.MoveSemantics\n+\n+    type F[A]    = Free[S, A]\n+    type M[A]    = FileSystemErrT[F, A]\n+    type G[E, A] = EitherT[F, E, A]\n+\n+    /** Returns a write handle for the specified file which may be used to\n+      * append data to the file it represents, creating it if necessary.\n+      *\n+      * Care must be taken to `close` the handle when it is no longer needed\n+      * to avoid potential resource leaks.\n+      */\n+    def open(file: AbsFile[Sandboxed]): M[WriteHandle] =\n+      EitherT(lift(Open(file)))\n+\n+    /** Write a chunk of data to the file represented by the write handle.\n+      *\n+      * Attempts to write as much of the chunk as possible, even if parts of\n+      * it fail, any such failures will be returned in the output `Vector`\n+      * An empty `Vector` means the entire chunk was written successfully.\n+      */\n+    def write(h: WriteHandle, chunk: Vector[Data]): F[Vector[FileSystemError]] =","path":"core/src/main/scala/quasar/fs/WriteFile.scala","position":58,"original_position":58,"commit_id":"b088851694da57052fb1b938ee8c31dc8877ef0c","original_commit_id":"b088851694da57052fb1b938ee8c31dc8877ef0c","user":{"login":"mossprescott","id":6147371,"avatar_url":"https://avatars.githubusercontent.com/u/6147371?v=3","gravatar_id":"","url":"https://api.github.com/users/mossprescott","html_url":"https://github.com/mossprescott","followers_url":"https://api.github.com/users/mossprescott/followers","following_url":"https://api.github.com/users/mossprescott/following{/other_user}","gists_url":"https://api.github.com/users/mossprescott/gists{/gist_id}","starred_url":"https://api.github.com/users/mossprescott/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mossprescott/subscriptions","organizations_url":"https://api.github.com/users/mossprescott/orgs","repos_url":"https://api.github.com/users/mossprescott/repos","events_url":"https://api.github.com/users/mossprescott/events{/privacy}","received_events_url":"https://api.github.com/users/mossprescott/received_events","type":"User","site_admin":false},"body":"This has always been kind of an unsatisfying way of returning errors. I can think of two things that can come back: trying to save non-obj values in MongoDB, and random runtime errors. Those seem like two very different sorts of problems, one being either a client error or a server bug, depending on how you look at it, and the other suggesting that the whole request maybe should be treated as a failure. Unfortunately I don't have a better idea.","created_at":"2015-11-17T00:07:35Z","updated_at":"2015-11-17T00:07:35Z","html_url":"https://github.com/quasar-analytics/quasar/pull/986#discussion_r45005020","pull_request_url":"https://api.github.com/repos/quasar-analytics/quasar/pulls/986","_links":{"self":{"href":"https://api.github.com/repos/quasar-analytics/quasar/pulls/comments/45005020"},"html":{"href":"https://github.com/quasar-analytics/quasar/pull/986#discussion_r45005020"},"pull_request":{"href":"https://api.github.com/repos/quasar-analytics/quasar/pulls/986"}}},"pull_request":{"url":"https://api.github.com/repos/quasar-analytics/quasar/pulls/986","id":50580861,"html_url":"https://github.com/quasar-analytics/quasar/pull/986","diff_url":"https://github.com/quasar-analytics/quasar/pull/986.diff","patch_url":"https://github.com/quasar-analytics/quasar/pull/986.patch","issue_url":"https://api.github.com/repos/quasar-analytics/quasar/issues/986","number":986,"state":"open","locked":false,"title":"[SD-983] Filesystem algebras and tests","user":{"login":"wemrysi","id":32185,"avatar_url":"https://avatars.githubusercontent.com/u/32185?v=3","gravatar_id":"","url":"https://api.github.com/users/wemrysi","html_url":"https://github.com/wemrysi","followers_url":"https://api.github.com/users/wemrysi/followers","following_url":"https://api.github.com/users/wemrysi/following{/other_user}","gists_url":"https://api.github.com/users/wemrysi/gists{/gist_id}","starred_url":"https://api.github.com/users/wemrysi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wemrysi/subscriptions","organizations_url":"https://api.github.com/users/wemrysi/orgs","repos_url":"https://api.github.com/users/wemrysi/repos","events_url":"https://api.github.com/users/wemrysi/events{/privacy}","received_events_url":"https://api.github.com/users/wemrysi/received_events","type":"User","site_admin":false},"body":"This is a mostly additive PR containing the new filesystem algebras, their MongoDB interpreters, and all the refactored tests to go along with them. Neither the rest API or REPL are using the new interpreters yet and this PR is primarily for integration and to try and break up the changes a bit so that there isn't a single enormous PR for all of SD-983 (though I know this one is pretty big).\r\n\r\nOnce the API and REPL PRs are submitted, a final PR will be submitted that deletes all the infrastructure being replaced and renames the `XXX2` variants to `XXX`.\r\n\r\nAlso, after reviewing @mossprescott 's views PR some concepts clarified a bit and as a result I am going to change the distribution of algebra terms a bit, merging the read-only bits of `ManageFile` with `ExecutePlan` into a new algebra called `QueryFile`. This doesn't change anything in this PR in a major semantic way, other than shuffling of where tests and some functions live and will tidy up some of the types a bit. So it doesn't need to hold up this PR, which is why I'm not waiting.\r\n\r\nA rough outline of what's in here\r\n* `ReadFile`, `WriteFile`, `ManageFile` and `ExecutePlan` algebras, describing the filesytem operations\r\n* `MongoDb` a thin wrapper around the async `mongo-java-driver`.\r\n* Interpreters for all the filesystem algebras in terms of `MongoDb`.\r\n* Conversion to pathy paths in all public filesystem apis (`quasar.fs.Path` is still used internally in a few places)\r\n* An in-memory filesystem interpreter useful for tests\r\n* Unit tests for all derived filesystem operations\r\n* A file system specification in the form of implementation agnostic integration tests, the idea being that if an implementation passes these tests, it should be a valid filesystem.\r\n* Implementation-specific filesystem tests for the MongoDb implementation\r\n* A side-by-side reimplementation of the regression test runner that is a bit more performant (though MongoDB is still the bottleneck here, I believe)\r\n* A thread-safe `TaskRef` cribbed from oncue/remotely\r\n* Various supporting functions in `quasar.fp` for interpreting free monads based on coproducts of functors\r\n* Broke apart regression test classes into own package and files for easier reuse.\r\n","created_at":"2015-11-12T22:56:51Z","updated_at":"2015-11-17T00:07:35Z","closed_at":null,"merged_at":null,"merge_commit_sha":"0dd07bc66cc8081bafe17c69c67394e23e5641d9","assignee":{"login":"sellout","id":33031,"avatar_url":"https://avatars.githubusercontent.com/u/33031?v=3","gravatar_id":"","url":"https://api.github.com/users/sellout","html_url":"https://github.com/sellout","followers_url":"https://api.github.com/users/sellout/followers","following_url":"https://api.github.com/users/sellout/following{/other_user}","gists_url":"https://api.github.com/users/sellout/gists{/gist_id}","starred_url":"https://api.github.com/users/sellout/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sellout/subscriptions","organizations_url":"https://api.github.com/users/sellout/orgs","repos_url":"https://api.github.com/users/sellout/repos","events_url":"https://api.github.com/users/sellout/events{/privacy}","received_events_url":"https://api.github.com/users/sellout/received_events","type":"User","site_admin":false},"milestone":null,"commits_url":"https://api.github.com/repos/quasar-analytics/quasar/pulls/986/commits","review_comments_url":"https://api.github.com/repos/quasar-analytics/quasar/pulls/986/comments","review_comment_url":"https://api.github.com/repos/quasar-analytics/quasar/pulls/comments{/number}","comments_url":"https://api.github.com/repos/quasar-analytics/quasar/issues/986/comments","statuses_url":"https://api.github.com/repos/quasar-analytics/quasar/statuses/b088851694da57052fb1b938ee8c31dc8877ef0c","head":{"label":"wemrysi:wip/SD-983-algebras-and-tests","ref":"wip/SD-983-algebras-and-tests","sha":"b088851694da57052fb1b938ee8c31dc8877ef0c","user":{"login":"wemrysi","id":32185,"avatar_url":"https://avatars.githubusercontent.com/u/32185?v=3","gravatar_id":"","url":"https://api.github.com/users/wemrysi","html_url":"https://github.com/wemrysi","followers_url":"https://api.github.com/users/wemrysi/followers","following_url":"https://api.github.com/users/wemrysi/following{/other_user}","gists_url":"https://api.github.com/users/wemrysi/gists{/gist_id}","starred_url":"https://api.github.com/users/wemrysi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wemrysi/subscriptions","organizations_url":"https://api.github.com/users/wemrysi/orgs","repos_url":"https://api.github.com/users/wemrysi/repos","events_url":"https://api.github.com/users/wemrysi/events{/privacy}","received_events_url":"https://api.github.com/users/wemrysi/received_events","type":"User","site_admin":false},"repo":{"id":40374964,"name":"quasar","full_name":"wemrysi/quasar","owner":{"login":"wemrysi","id":32185,"avatar_url":"https://avatars.githubusercontent.com/u/32185?v=3","gravatar_id":"","url":"https://api.github.com/users/wemrysi","html_url":"https://github.com/wemrysi","followers_url":"https://api.github.com/users/wemrysi/followers","following_url":"https://api.github.com/users/wemrysi/following{/other_user}","gists_url":"https://api.github.com/users/wemrysi/gists{/gist_id}","starred_url":"https://api.github.com/users/wemrysi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wemrysi/subscriptions","organizations_url":"https://api.github.com/users/wemrysi/orgs","repos_url":"https://api.github.com/users/wemrysi/repos","events_url":"https://api.github.com/users/wemrysi/events{/privacy}","received_events_url":"https://api.github.com/users/wemrysi/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/wemrysi/quasar","description":"The engine for SlamData","fork":true,"url":"https://api.github.com/repos/wemrysi/quasar","forks_url":"https://api.github.com/repos/wemrysi/quasar/forks","keys_url":"https://api.github.com/repos/wemrysi/quasar/keys{/key_id}","collaborators_url":"https://api.github.com/repos/wemrysi/quasar/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/wemrysi/quasar/teams","hooks_url":"https://api.github.com/repos/wemrysi/quasar/hooks","issue_events_url":"https://api.github.com/repos/wemrysi/quasar/issues/events{/number}","events_url":"https://api.github.com/repos/wemrysi/quasar/events","assignees_url":"https://api.github.com/repos/wemrysi/quasar/assignees{/user}","branches_url":"https://api.github.com/repos/wemrysi/quasar/branches{/branch}","tags_url":"https://api.github.com/repos/wemrysi/quasar/tags","blobs_url":"https://api.github.com/repos/wemrysi/quasar/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/wemrysi/quasar/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/wemrysi/quasar/git/refs{/sha}","trees_url":"https://api.github.com/repos/wemrysi/quasar/git/trees{/sha}","statuses_url":"https://api.github.com/repos/wemrysi/quasar/statuses/{sha}","languages_url":"https://api.github.com/repos/wemrysi/quasar/languages","stargazers_url":"https://api.github.com/repos/wemrysi/quasar/stargazers","contributors_url":"https://api.github.com/repos/wemrysi/quasar/contributors","subscribers_url":"https://api.github.com/repos/wemrysi/quasar/subscribers","subscription_url":"https://api.github.com/repos/wemrysi/quasar/subscription","commits_url":"https://api.github.com/repos/wemrysi/quasar/commits{/sha}","git_commits_url":"https://api.github.com/repos/wemrysi/quasar/git/commits{/sha}","comments_url":"https://api.github.com/repos/wemrysi/quasar/comments{/number}","issue_comment_url":"https://api.github.com/repos/wemrysi/quasar/issues/comments{/number}","contents_url":"https://api.github.com/repos/wemrysi/quasar/contents/{+path}","compare_url":"https://api.github.com/repos/wemrysi/quasar/compare/{base}...{head}","merges_url":"https://api.github.com/repos/wemrysi/quasar/merges","archive_url":"https://api.github.com/repos/wemrysi/quasar/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/wemrysi/quasar/downloads","issues_url":"https://api.github.com/repos/wemrysi/quasar/issues{/number}","pulls_url":"https://api.github.com/repos/wemrysi/quasar/pulls{/number}","milestones_url":"https://api.github.com/repos/wemrysi/quasar/milestones{/number}","notifications_url":"https://api.github.com/repos/wemrysi/quasar/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/wemrysi/quasar/labels{/name}","releases_url":"https://api.github.com/repos/wemrysi/quasar/releases{/id}","created_at":"2015-08-07T18:29:08Z","updated_at":"2015-09-14T20:43:00Z","pushed_at":"2015-11-16T23:58:23Z","git_url":"git://github.com/wemrysi/quasar.git","ssh_url":"git@github.com:wemrysi/quasar.git","clone_url":"https://github.com/wemrysi/quasar.git","svn_url":"https://github.com/wemrysi/quasar","homepage":null,"size":7883,"stargazers_count":0,"watchers_count":0,"language":"Scala","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":1,"forks":0,"open_issues":1,"watchers":0,"default_branch":"master"}},"base":{"label":"quasar-analytics:master","ref":"master","sha":"2bd9d34be1eb0151c8160a67fa5584608df75234","user":{"login":"quasar-analytics","id":15095906,"avatar_url":"https://avatars.githubusercontent.com/u/15095906?v=3","gravatar_id":"","url":"https://api.github.com/users/quasar-analytics","html_url":"https://github.com/quasar-analytics","followers_url":"https://api.github.com/users/quasar-analytics/followers","following_url":"https://api.github.com/users/quasar-analytics/following{/other_user}","gists_url":"https://api.github.com/users/quasar-analytics/gists{/gist_id}","starred_url":"https://api.github.com/users/quasar-analytics/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/quasar-analytics/subscriptions","organizations_url":"https://api.github.com/users/quasar-analytics/orgs","repos_url":"https://api.github.com/users/quasar-analytics/repos","events_url":"https://api.github.com/users/quasar-analytics/events{/privacy}","received_events_url":"https://api.github.com/users/quasar-analytics/received_events","type":"Organization","site_admin":false},"repo":{"id":16515058,"name":"quasar","full_name":"quasar-analytics/quasar","owner":{"login":"quasar-analytics","id":15095906,"avatar_url":"https://avatars.githubusercontent.com/u/15095906?v=3","gravatar_id":"","url":"https://api.github.com/users/quasar-analytics","html_url":"https://github.com/quasar-analytics","followers_url":"https://api.github.com/users/quasar-analytics/followers","following_url":"https://api.github.com/users/quasar-analytics/following{/other_user}","gists_url":"https://api.github.com/users/quasar-analytics/gists{/gist_id}","starred_url":"https://api.github.com/users/quasar-analytics/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/quasar-analytics/subscriptions","organizations_url":"https://api.github.com/users/quasar-analytics/orgs","repos_url":"https://api.github.com/users/quasar-analytics/repos","events_url":"https://api.github.com/users/quasar-analytics/events{/privacy}","received_events_url":"https://api.github.com/users/quasar-analytics/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/quasar-analytics/quasar","description":"A NoSQL analytics engine that embraces post-relational analytics and pushes computation to the data.","fork":false,"url":"https://api.github.com/repos/quasar-analytics/quasar","forks_url":"https://api.github.com/repos/quasar-analytics/quasar/forks","keys_url":"https://api.github.com/repos/quasar-analytics/quasar/keys{/key_id}","collaborators_url":"https://api.github.com/repos/quasar-analytics/quasar/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/quasar-analytics/quasar/teams","hooks_url":"https://api.github.com/repos/quasar-analytics/quasar/hooks","issue_events_url":"https://api.github.com/repos/quasar-analytics/quasar/issues/events{/number}","events_url":"https://api.github.com/repos/quasar-analytics/quasar/events","assignees_url":"https://api.github.com/repos/quasar-analytics/quasar/assignees{/user}","branches_url":"https://api.github.com/repos/quasar-analytics/quasar/branches{/branch}","tags_url":"https://api.github.com/repos/quasar-analytics/quasar/tags","blobs_url":"https://api.github.com/repos/quasar-analytics/quasar/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/quasar-analytics/quasar/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/quasar-analytics/quasar/git/refs{/sha}","trees_url":"https://api.github.com/repos/quasar-analytics/quasar/git/trees{/sha}","statuses_url":"https://api.github.com/repos/quasar-analytics/quasar/statuses/{sha}","languages_url":"https://api.github.com/repos/quasar-analytics/quasar/languages","stargazers_url":"https://api.github.com/repos/quasar-analytics/quasar/stargazers","contributors_url":"https://api.github.com/repos/quasar-analytics/quasar/contributors","subscribers_url":"https://api.github.com/repos/quasar-analytics/quasar/subscribers","subscription_url":"https://api.github.com/repos/quasar-analytics/quasar/subscription","commits_url":"https://api.github.com/repos/quasar-analytics/quasar/commits{/sha}","git_commits_url":"https://api.github.com/repos/quasar-analytics/quasar/git/commits{/sha}","comments_url":"https://api.github.com/repos/quasar-analytics/quasar/comments{/number}","issue_comment_url":"https://api.github.com/repos/quasar-analytics/quasar/issues/comments{/number}","contents_url":"https://api.github.com/repos/quasar-analytics/quasar/contents/{+path}","compare_url":"https://api.github.com/repos/quasar-analytics/quasar/compare/{base}...{head}","merges_url":"https://api.github.com/repos/quasar-analytics/quasar/merges","archive_url":"https://api.github.com/repos/quasar-analytics/quasar/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/quasar-analytics/quasar/downloads","issues_url":"https://api.github.com/repos/quasar-analytics/quasar/issues{/number}","pulls_url":"https://api.github.com/repos/quasar-analytics/quasar/pulls{/number}","milestones_url":"https://api.github.com/repos/quasar-analytics/quasar/milestones{/number}","notifications_url":"https://api.github.com/repos/quasar-analytics/quasar/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/quasar-analytics/quasar/labels{/name}","releases_url":"https://api.github.com/repos/quasar-analytics/quasar/releases{/id}","created_at":"2014-02-04T15:01:27Z","updated_at":"2015-11-05T21:42:22Z","pushed_at":"2015-11-16T23:58:24Z","git_url":"git://github.com/quasar-analytics/quasar.git","ssh_url":"git@github.com:quasar-analytics/quasar.git","clone_url":"https://github.com/quasar-analytics/quasar.git","svn_url":"https://github.com/quasar-analytics/quasar","homepage":"","size":7874,"stargazers_count":162,"watchers_count":162,"language":"Scala","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":38,"mirror_url":null,"open_issues_count":162,"forks":38,"open_issues":162,"watchers":162,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/quasar-analytics/quasar/pulls/986"},"html":{"href":"https://github.com/quasar-analytics/quasar/pull/986"},"issue":{"href":"https://api.github.com/repos/quasar-analytics/quasar/issues/986"},"comments":{"href":"https://api.github.com/repos/quasar-analytics/quasar/issues/986/comments"},"review_comments":{"href":"https://api.github.com/repos/quasar-analytics/quasar/pulls/986/comments"},"review_comment":{"href":"https://api.github.com/repos/quasar-analytics/quasar/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/quasar-analytics/quasar/pulls/986/commits"},"statuses":{"href":"https://api.github.com/repos/quasar-analytics/quasar/statuses/b088851694da57052fb1b938ee8c31dc8877ef0c"}}}}