{"action":"created","issue":{"url":"https://api.github.com/repos/honnibal/spaCy/issues/197","labels_url":"https://api.github.com/repos/honnibal/spaCy/issues/197/labels{/name}","comments_url":"https://api.github.com/repos/honnibal/spaCy/issues/197/comments","events_url":"https://api.github.com/repos/honnibal/spaCy/issues/197/events","html_url":"https://github.com/honnibal/spaCy/issues/197","id":119691141,"number":197,"title":"Feature Request: Vector \"File\" interface","user":{"login":"RXminuS","id":3949285,"avatar_url":"https://avatars.githubusercontent.com/u/3949285?v=3","gravatar_id":"","url":"https://api.github.com/users/RXminuS","html_url":"https://github.com/RXminuS","followers_url":"https://api.github.com/users/RXminuS/followers","following_url":"https://api.github.com/users/RXminuS/following{/other_user}","gists_url":"https://api.github.com/users/RXminuS/gists{/gist_id}","starred_url":"https://api.github.com/users/RXminuS/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/RXminuS/subscriptions","organizations_url":"https://api.github.com/users/RXminuS/orgs","repos_url":"https://api.github.com/users/RXminuS/repos","events_url":"https://api.github.com/users/RXminuS/events{/privacy}","received_events_url":"https://api.github.com/users/RXminuS/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2015-12-01T10:49:10Z","updated_at":"2015-12-04T21:27:49Z","closed_at":null,"body":"Just read in an old spaCy tutorial the following \"Future versions of spaCy will allow you to provide a file-like object, instead of a location of a [vector bin] file.\"\r\n\r\nIs this in place yet? Would love to replace standard vector file and in-memory loading with my own Redis (or any other \"shared-memory-system\") interface to allow a distributed cluster of spacy nodes to share the same \"file\". Would love to contribute, any pointers on where to start looking?"},"comment":{"url":"https://api.github.com/repos/honnibal/spaCy/issues/comments/162086235","html_url":"https://github.com/honnibal/spaCy/issues/197#issuecomment-162086235","issue_url":"https://api.github.com/repos/honnibal/spaCy/issues/197","id":162086235,"user":{"login":"honnibal","id":8059750,"avatar_url":"https://avatars.githubusercontent.com/u/8059750?v=3","gravatar_id":"","url":"https://api.github.com/users/honnibal","html_url":"https://github.com/honnibal","followers_url":"https://api.github.com/users/honnibal/followers","following_url":"https://api.github.com/users/honnibal/following{/other_user}","gists_url":"https://api.github.com/users/honnibal/gists{/gist_id}","starred_url":"https://api.github.com/users/honnibal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/honnibal/subscriptions","organizations_url":"https://api.github.com/users/honnibal/orgs","repos_url":"https://api.github.com/users/honnibal/repos","events_url":"https://api.github.com/users/honnibal/events{/privacy}","received_events_url":"https://api.github.com/users/honnibal/received_events","type":"User","site_admin":false},"created_at":"2015-12-04T21:27:49Z","updated_at":"2015-12-04T21:27:49Z","body":"> Do you have some form of basic class design for it yet which I should stick to or shall I just come up with something?\r\n\r\nI thought the answer was \"no\", but then I started writing out some \"suggestions\", and I guess I have a clearer idea than I thought :). Most of these things are demanded by consistency with the rest of the library. You can see examples of pretty much all of this in the `Vocab` and `StringsTable` classes:\r\n\r\nPython API:\r\n\r\n* Use the `__getitem__`, `__setitem__` and `__iter__` special methods. You don't necessarily have to subclass dict.\r\n* All vectors in the same `VectorTable` must be the same length. \r\n* The table should allow the user to pass in a key function, which should take a `spacy.tokens.Token` object as an argument, and return a 64-bit unsigned integer (used to key the table)\r\n* The hash will be non-reversible, and it won't be possible to iterate over the keys and get back a useful representation of the integer key. This is okay. Otherwise we'll have to store the key strings, which could occupy a lot of memory.\r\n\r\nImplementation details:\r\n* The data should be stored in a PreshMap instance.\r\n* The table should be keyed by a `feat_t` (64 bit unisnged integer), with values being `weight_t*`, i.e. raw C arrays of floats.\r\n* Allocate the memory using `cymem.Pool`"}}