{"action":"created","issue":{"url":"https://api.github.com/repos/sandstorm-io/capnproto/issues/247","labels_url":"https://api.github.com/repos/sandstorm-io/capnproto/issues/247/labels{/name}","comments_url":"https://api.github.com/repos/sandstorm-io/capnproto/issues/247/comments","events_url":"https://api.github.com/repos/sandstorm-io/capnproto/issues/247/events","html_url":"https://github.com/sandstorm-io/capnproto/pull/247","id":108579529,"number":247,"title":"Add TextCodec to expose an API for the schema's text format.","user":{"login":"pqu","id":227179,"avatar_url":"https://avatars.githubusercontent.com/u/227179?v=3","gravatar_id":"","url":"https://api.github.com/users/pqu","html_url":"https://github.com/pqu","followers_url":"https://api.github.com/users/pqu/followers","following_url":"https://api.github.com/users/pqu/following{/other_user}","gists_url":"https://api.github.com/users/pqu/gists{/gist_id}","starred_url":"https://api.github.com/users/pqu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pqu/subscriptions","organizations_url":"https://api.github.com/users/pqu/orgs","repos_url":"https://api.github.com/users/pqu/repos","events_url":"https://api.github.com/users/pqu/events{/privacy}","received_events_url":"https://api.github.com/users/pqu/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-09-28T02:39:20Z","updated_at":"2015-09-30T02:45:20Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/sandstorm-io/capnproto/pulls/247","html_url":"https://github.com/sandstorm-io/capnproto/pull/247","diff_url":"https://github.com/sandstorm-io/capnproto/pull/247.diff","patch_url":"https://github.com/sandstorm-io/capnproto/pull/247.patch"},"body":"I've been looking at using the schema text format as a convenient way of specifying unit test cases that contain Cap'n Proto data, and as a runtime format for user configuration (already backed by a Cap'n Proto schema). But there's no API for reading these messages at runtime (aside from rolling them into a constant in a schema and using `SchemaParser`).\r\n\r\nReading the mailing lists, I see there's resistance to exposing an API for reading text messages because it has some undesirable effects if it's abused as a long-term storage or interchange format. This is understandable, but I think there are valid use cases for using the text format at runtime, and that the caveats are no worse than if JSON or the `SchemaParser` APIs are used instead. In particular, my use cases are similar to that described in [this thread](https://groups.google.com/d/msg/capnproto/nAYn8g9rn_w/Vi-vgcJ2xokJ):\r\n * Messages are supplied (and are editable) by users,\r\n * Schema definitions (and the capnp tool) are not distributed with the built application, and\r\n * Messages are not expected to be transmitted or be viable between versions.\r\n\r\nWhile the `SchemaParser` supports reading this format, it expects to read a complete *schema*, which introduces a fair bit of setup overhead if only a single *message* needs to be read.\r\n\r\nAs with any other text format, this format is unable to meet the backwards-compatibility guarantees of the binary format—but there is some scope for customising errors, such as unknown fields, through the `ErrorHandler` (although there needs to be a better way to communicate the type of error from the `ValueTranslator`). But in my case, I prefer to treat these as hard-errors.\r\n\r\nAlso, keeping this in the `capnpc` library should hopefully deter people from using it as a robust alternative for messages."},"comment":{"url":"https://api.github.com/repos/sandstorm-io/capnproto/issues/comments/144264018","html_url":"https://github.com/sandstorm-io/capnproto/pull/247#issuecomment-144264018","issue_url":"https://api.github.com/repos/sandstorm-io/capnproto/issues/247","id":144264018,"user":{"login":"kentonv","id":4001805,"avatar_url":"https://avatars.githubusercontent.com/u/4001805?v=3","gravatar_id":"","url":"https://api.github.com/users/kentonv","html_url":"https://github.com/kentonv","followers_url":"https://api.github.com/users/kentonv/followers","following_url":"https://api.github.com/users/kentonv/following{/other_user}","gists_url":"https://api.github.com/users/kentonv/gists{/gist_id}","starred_url":"https://api.github.com/users/kentonv/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kentonv/subscriptions","organizations_url":"https://api.github.com/users/kentonv/orgs","repos_url":"https://api.github.com/users/kentonv/repos","events_url":"https://api.github.com/users/kentonv/events{/privacy}","received_events_url":"https://api.github.com/users/kentonv/received_events","type":"User","site_admin":false},"created_at":"2015-09-30T02:45:20Z","updated_at":"2015-09-30T02:45:20Z","body":"Hi Philip,\r\n\r\n> While the `SchemaParser` supports reading this format, it expects to read a complete schema, which introduces a fair bit of setup overhead if only a single message needs to be read.\r\n\r\nAre you sure that's not what you want? Using a proper schema file has a huge advantage over raw \"text format\": variable substitution. Meanwhile this \"fair bit of setup overhead\" seems pretty light to me: a file ID and a constant declaration.\r\n\r\nI've had success with this approach in Sandstorm: apps define their metadata in the form of a file called `sandstorm-pkgdef.capnp`. For example:\r\n\r\nhttps://github.com/kentonv/etherpad-lite/blob/sandstorm/sandstorm-pkgdef.capnp\r\n\r\nThis file is not parsed by the usual `capnp` tool, but rather by Sandstorm's `spk` tool, using `SchemaParser`. The ability to use variable substitution has proven surprisingly useful both for keeping the content organized and for cases of repetition.\r\n\r\nYou mention unit test cases as one use case. These tend to be highly repetitive. Would it not be useful to be able to define a set of variables for sub-objects that can be reused in each test case? (In fact, for a unit test, I would normally recommend compiling the constants into the binary rather than reading them at runtime, because why not?)"}}