{"action":"created","issue":{"url":"https://api.github.com/repos/HelloZeroNet/ZeroNet/issues/163","labels_url":"https://api.github.com/repos/HelloZeroNet/ZeroNet/issues/163/labels{/name}","comments_url":"https://api.github.com/repos/HelloZeroNet/ZeroNet/issues/163/comments","events_url":"https://api.github.com/repos/HelloZeroNet/ZeroNet/issues/163/events","html_url":"https://github.com/HelloZeroNet/ZeroNet/issues/163","id":106409926,"number":163,"title":"Optional files","user":{"login":"HelloZeroNet","id":10350359,"avatar_url":"https://avatars.githubusercontent.com/u/10350359?v=3","gravatar_id":"","url":"https://api.github.com/users/HelloZeroNet","html_url":"https://github.com/HelloZeroNet","followers_url":"https://api.github.com/users/HelloZeroNet/followers","following_url":"https://api.github.com/users/HelloZeroNet/following{/other_user}","gists_url":"https://api.github.com/users/HelloZeroNet/gists{/gist_id}","starred_url":"https://api.github.com/users/HelloZeroNet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HelloZeroNet/subscriptions","organizations_url":"https://api.github.com/users/HelloZeroNet/orgs","repos_url":"https://api.github.com/users/HelloZeroNet/repos","events_url":"https://api.github.com/users/HelloZeroNet/events{/privacy}","received_events_url":"https://api.github.com/users/HelloZeroNet/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/HelloZeroNet/ZeroNet/labels/enhancement","name":"enhancement","color":"84b6eb"},{"url":"https://api.github.com/repos/HelloZeroNet/ZeroNet/labels/hard","name":"hard","color":"eb6420"}],"state":"open","locked":false,"assignee":{"login":"shortcutme","id":1019235,"avatar_url":"https://avatars.githubusercontent.com/u/1019235?v=3","gravatar_id":"","url":"https://api.github.com/users/shortcutme","html_url":"https://github.com/shortcutme","followers_url":"https://api.github.com/users/shortcutme/followers","following_url":"https://api.github.com/users/shortcutme/following{/other_user}","gists_url":"https://api.github.com/users/shortcutme/gists{/gist_id}","starred_url":"https://api.github.com/users/shortcutme/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shortcutme/subscriptions","organizations_url":"https://api.github.com/users/shortcutme/orgs","repos_url":"https://api.github.com/users/shortcutme/repos","events_url":"https://api.github.com/users/shortcutme/events{/privacy}","received_events_url":"https://api.github.com/users/shortcutme/received_events","type":"User","site_admin":false},"milestone":null,"comments":3,"created_at":"2015-09-14T19:17:44Z","updated_at":"2015-09-17T13:58:04Z","closed_at":null,"body":"Provide a way to define not automatically downloaded files in content.json. It will make possible to create larger sites with many image (and later videos).\r\n\r\nTo do this we can simply add `optional_files` node to content.json that is identical to current `files` node. This files only downloaded if user is request it directly (by a http request).\r\n\r\nThese files are excluded from site limit calculation (10MB by default). These files lands in a cache registry which has use-configurable size, and if the cache is runs out of the configured limit the most seeded file will be automatically removed.\r\n\r\n## Problem 1: Initial distribution\r\nIn a Multi-user imageboard site if someone uploads a new image it will not downloaded by default anyone, so he/she has to wait until someone open the site and request it.\r\n\r\nPossible solution: It would be possible to join to a site as \"sponsor\" who is downloads every optional files automatically and keeps it seeding until its necessary.\r\nAlso every user would be able to add \"share friend\" users. If your share friend uploads a file you automatically downloads his/her files and help to distribute it.\r\n\r\nThe API also adds possibility to \"pin\" (where it will not removed automatically from cache) and dislike (immediately remove) files.\r\n\r\n## Problem 2: Who has it?\r\nCurrently every peer has every file, so if you want to download a file you can request it from anyone. With optional files this is going to change: To solve this every connected peer has to keep a `hashfield` table which holds the downloaded files hashes first 4 character (only first 4 for economy reasons) eg.:\r\n```\r\npeer1 = ['8a42', 'fa11']\r\npeer2 = ['12ab', 'bbc9', 'd39a']\r\n```\r\nIf you want to download an optional file you checks if it's hash's first 4 character is in any of connected peer's hash table. If its not then you ask your connected peers to check if any of their connected peers has it. If still not found you can try to connect other peers randomly.\r\n\r\nIf you finish downloading an optional file you send the connected peers to add it to your `hashfield`, so if anyone looking for it later they will know you has it. (Probably some kind a batching is recommended)"},"comment":{"url":"https://api.github.com/repos/HelloZeroNet/ZeroNet/issues/comments/141094952","html_url":"https://github.com/HelloZeroNet/ZeroNet/issues/163#issuecomment-141094952","issue_url":"https://api.github.com/repos/HelloZeroNet/ZeroNet/issues/163","id":141094952,"user":{"login":"bashrc","id":3055239,"avatar_url":"https://avatars.githubusercontent.com/u/3055239?v=3","gravatar_id":"","url":"https://api.github.com/users/bashrc","html_url":"https://github.com/bashrc","followers_url":"https://api.github.com/users/bashrc/followers","following_url":"https://api.github.com/users/bashrc/following{/other_user}","gists_url":"https://api.github.com/users/bashrc/gists{/gist_id}","starred_url":"https://api.github.com/users/bashrc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bashrc/subscriptions","organizations_url":"https://api.github.com/users/bashrc/orgs","repos_url":"https://api.github.com/users/bashrc/repos","events_url":"https://api.github.com/users/bashrc/events{/privacy}","received_events_url":"https://api.github.com/users/bashrc/received_events","type":"User","site_admin":false},"created_at":"2015-09-17T13:58:04Z","updated_at":"2015-09-17T13:58:04Z","body":"I don't think there should be a peer hierarchy, and whether or not someone has a powerful machine should be irrelevant."}}