{"action":"created","issue":{"url":"https://api.github.com/repos/boltdb/bolt/issues/293","labels_url":"https://api.github.com/repos/boltdb/bolt/issues/293/labels{/name}","comments_url":"https://api.github.com/repos/boltdb/bolt/issues/293/comments","events_url":"https://api.github.com/repos/boltdb/bolt/issues/293/events","html_url":"https://github.com/boltdb/bolt/issues/293","id":55811241,"number":293,"title":"Best Practices","user":{"login":"j","id":589410,"avatar_url":"https://avatars.githubusercontent.com/u/589410?v=3","gravatar_id":"","url":"https://api.github.com/users/j","html_url":"https://github.com/j","followers_url":"https://api.github.com/users/j/followers","following_url":"https://api.github.com/users/j/following{/other_user}","gists_url":"https://api.github.com/users/j/gists{/gist_id}","starred_url":"https://api.github.com/users/j/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/j/subscriptions","organizations_url":"https://api.github.com/users/j/orgs","repos_url":"https://api.github.com/users/j/repos","events_url":"https://api.github.com/users/j/events{/privacy}","received_events_url":"https://api.github.com/users/j/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/boltdb/bolt/labels/question","name":"question","color":"cc317c"}],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-01-28T20:32:28Z","updated_at":"2015-01-30T03:07:37Z","closed_at":"2015-01-30T03:07:37Z","body":"I have a few questions before diving completely head first into Bolt.\r\n\r\n* How many buckets can bolt handle, and does performance degrade with the use of many buckets?\r\n* What's the use-case of nesting buckets\r\n\r\nAnd, Dealing with relationships, which scenario would be best?\r\n\r\nA Test has many Variations.\r\n\r\n1. I have a \"Tests\" bucket, and I nest a bucket called \"Variations\" in it prefixed like, \"T:{test id}:V\", and that bucket contains it's Variations.  I can then iterate over that bucket completely for the test's variations.\r\n2. I have a \"Tests\" bucket, a \"Variations\" bucket, and a \"TestsVariations\" bucket. \"TestVariations\" has keys prefixed with the variation ID, I find those and each match is a variation. (I suppose I can also just have the key prefixed with the Test id and have the variations as the value and get all those keys from the Variations bucket.... but if this gets large, I assume it may be a problem.\r\n\r\nAm I way off?"},"comment":{"url":"https://api.github.com/repos/boltdb/bolt/issues/comments/72147108","html_url":"https://github.com/boltdb/bolt/issues/293#issuecomment-72147108","issue_url":"https://api.github.com/repos/boltdb/bolt/issues/293","id":72147108,"user":{"login":"benbjohnson","id":118015,"avatar_url":"https://avatars.githubusercontent.com/u/118015?v=3","gravatar_id":"","url":"https://api.github.com/users/benbjohnson","html_url":"https://github.com/benbjohnson","followers_url":"https://api.github.com/users/benbjohnson/followers","following_url":"https://api.github.com/users/benbjohnson/following{/other_user}","gists_url":"https://api.github.com/users/benbjohnson/gists{/gist_id}","starred_url":"https://api.github.com/users/benbjohnson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/benbjohnson/subscriptions","organizations_url":"https://api.github.com/users/benbjohnson/orgs","repos_url":"https://api.github.com/users/benbjohnson/repos","events_url":"https://api.github.com/users/benbjohnson/events{/privacy}","received_events_url":"https://api.github.com/users/benbjohnson/received_events","type":"User","site_admin":false},"created_at":"2015-01-30T03:07:37Z","updated_at":"2015-01-30T03:07:37Z","body":"> How many buckets can bolt handle, and does performance degrade with the use of many buckets?\r\n\r\nThere's not a limit on buckets. They can be inefficient with regards to space if you store a small amount of data in each one. If the size of the data in a bucket barely exceeds a 4KB page then it'll split into two leaf pages and one parent branch page. So if you have a bucket with 5KB of data it still needs three 4KB pages store it in for a total of 12KB. Space tends to even out more when you have larger data sets in a bucket.\r\n\r\n> What's the use-case of nesting buckets?\r\n\r\nThe original use case was for a database that stored web site page views grouped by session. The top level bucket stored sessions and each session was a bucket that stored all the page views. It suffered from some of the poor utilization issues mentioned above but it went *really* fast since the data was sorted exactly how I needed to read it (e.g. by session, then by timestamp).\r\n\r\nAnother use case is if you need to segment data and you don't want your code to accidentally grab data from the wrong segment. I've used buckets in multi-tenant systems to keep one user's data completely separate from another user's data without having to make a Bolt DB for each user.\r\n\r\n> And, Dealing with relationships, which scenario would be best?\r\n\r\nWhen I model relationships in Bolt I look at the cardinality of the data and how the data is accessed. There's essentially 3 models I consider:\r\n\r\n1. *Small cardinality, never accessed directly* - This would be a situation like if a `User` could have multiple names. I would just make a `users` bucket and encode the names directly into the user object that I encode.\r\n\r\n2. *Small cardinality, accessed directly* - This is a situation like the `Test` & `Variation` example you gave. A \"test\" probably doesn't have thousands of variations but the variations are probably accessed directly by a `VariationID`. I would make a `tests` bucket and a `variations` bucket and simple store the list of variation ids into the test object. I'd probably also store the parent test id into the variation object as well so you can go back and forth between the two objects. If you're using something like `gogoprotobuf` then it's really fast to decode a `Test` struct to grab your `Test.VariationIDs` list and then traverse into the \"variations\" bucket.\r\n\r\n3. *Large cardinality* - This is a situation where a parent object has thousands or tens of thousands of child objects. In this case a subbucket is probably appropriate. It gives you a nice isolated space to put the child objects. Another option is to create a single bucket to act as an index and you simply make the key `<parentID><childID>` and don't set a value. That'll let you iterate over a range of `childIDs` for a given `parentID`.\r\n\r\nOne more thing to note. I typically use integer IDs when possible because they're easy to work with, fixed size and Bolt can generate them for you per bucket. I generate them using the `Bucket.NextSequence()` call and store them using `binary.BigEndian.PutUint64()` since big-endian numbers are byte-sortable.\r\n\r\nI hope this all helps. Let me know if you have any questions.\r\n\r\n"}}