{"action":"opened","issue":{"url":"https://api.github.com/repos/tus/tus-resumable-upload-protocol/issues/55","labels_url":"https://api.github.com/repos/tus/tus-resumable-upload-protocol/issues/55/labels{/name}","comments_url":"https://api.github.com/repos/tus/tus-resumable-upload-protocol/issues/55/comments","events_url":"https://api.github.com/repos/tus/tus-resumable-upload-protocol/issues/55/events","html_url":"https://github.com/tus/tus-resumable-upload-protocol/issues/55","id":55340687,"number":55,"title":"Block Uploads vs. Offset Ranges","user":{"login":"Acconut","id":1375043,"avatar_url":"https://avatars.githubusercontent.com/u/1375043?v=3","gravatar_id":"","url":"https://api.github.com/users/Acconut","html_url":"https://github.com/Acconut","followers_url":"https://api.github.com/users/Acconut/followers","following_url":"https://api.github.com/users/Acconut/following{/other_user}","gists_url":"https://api.github.com/users/Acconut/gists{/gist_id}","starred_url":"https://api.github.com/users/Acconut/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Acconut/subscriptions","organizations_url":"https://api.github.com/users/Acconut/orgs","repos_url":"https://api.github.com/users/Acconut/repos","events_url":"https://api.github.com/users/Acconut/events{/privacy}","received_events_url":"https://api.github.com/users/Acconut/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/tus/tus-resumable-upload-protocol/labels/extension","name":"extension","color":"009800"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/tus/tus-resumable-upload-protocol/milestones/1","labels_url":"https://api.github.com/repos/tus/tus-resumable-upload-protocol/milestones/1/labels","id":888130,"number":1,"title":"1.0","description":"","creator":{"login":"Acconut","id":1375043,"avatar_url":"https://avatars.githubusercontent.com/u/1375043?v=3","gravatar_id":"","url":"https://api.github.com/users/Acconut","html_url":"https://github.com/Acconut","followers_url":"https://api.github.com/users/Acconut/followers","following_url":"https://api.github.com/users/Acconut/following{/other_user}","gists_url":"https://api.github.com/users/Acconut/gists{/gist_id}","starred_url":"https://api.github.com/users/Acconut/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Acconut/subscriptions","organizations_url":"https://api.github.com/users/Acconut/orgs","repos_url":"https://api.github.com/users/Acconut/repos","events_url":"https://api.github.com/users/Acconut/events{/privacy}","received_events_url":"https://api.github.com/users/Acconut/received_events","type":"User","site_admin":false},"open_issues":10,"closed_issues":15,"state":"open","created_at":"2014-12-02T15:53:38Z","updated_at":"2015-01-23T22:21:30Z","due_on":null,"closed_at":null},"comments":0,"created_at":"2015-01-23T22:21:30Z","updated_at":"2015-01-23T22:21:30Z","closed_at":null,"body":"In this issue I will propose two extensions which both aim implement the features needed for parallel uploads and non-contiguous chunks (see #3). In the end we have to choose one of them or go with something else.\r\n\r\nBefore starting I want to clarify that streaming uploads (with unknown length at the beginning) have not been included in these thoughts since you should not use them in conjunction with parallel uploads.\r\n\r\n------\r\n\r\nThe first solution uses offset ranges. Instead of defining one offset which starts at 0 and is incremented for each `PATCH` request the server would store one or multiple ranges of allowed and free offsets. These offsets will be returned in the `Offset-Range` header in the `HEAD` request replacing the `Offset` response (!) header. The client then uses this information to choose the offset and uploads the same way as it's currently implemented.\r\n\r\nHere is an example of a 300-byte file of which the second 100 bytes (100-199) have been uploaded:\r\n\r\n```\r\nHEAD /files/foo HTTP/1.1\r\n\r\nHTTP/1.1 204 No Content\r\nEnitity-Length: 300\r\nOffset-Range: 0-99, 200-299\r\n```\r\n```\r\nPATCH /files/foo HTTP/1.1\r\nContent-Length: 100\r\nOffset: 200\r\n\r\n[100 bytes]\r\n\r\nHTTP/1.1 204 No Content\r\n```\r\n```\r\nHEAD /files/foo HTTP/1.1\r\n\r\nHTTP/1.1 204 No Content\r\nEnitity-Length: 300\r\nOffset-Range: 0-99\r\n```\r\n\r\nThe range of the last 100 bytes (200-299) has been removed since this buffer has been filled successfully by the upload.\r\n\r\nWhile this solution allows the maximum of flexibility (compared to my second proposal) since you can upload at any offset (as long as it's available) it may be a though extension to implement for the servers. It has to ensure that the start of the offset range against which the chunk is uploaded is available and the end of the offset. Using the example from above you're not allowed to patch a 150-byte chunk at the offset of 0 because the bytes starting from 100 have already been written.\r\n\r\n-----\r\n\r\nThe second solution I came up with involves a bit more: When creating a new upload (using the file creation extension or somehow else) a blocksize is defined using which the file is separated into different blocks. For example, considering a file of 5KB and a blocksize of 2KB you would end up with two blocks of 2KB and a single one of 1KB. The important point is that each of the blocks has its own offset which starts at position 0 relative to the starting position of the block.\r\n\r\nConsidering the last example, the relative offset 100 of the second block would be the absolute offset of 2148: 2048 (2KB starting position of the second block) + 100 relative offset.\r\n\r\nOnly one upload is allowed at the same time per block. In this example a maximum of three parallel uploads are allowed. Each new `PATCH` request must resume where the last upload of the block has stopped, jumps are not allowed.\r\n\r\nIn following example we consider having a file of 5KB with the blocksize of 2KB. The first block is already fully uploaded (2048 bytes), the second with is filled with 100 bytes and the last one has not a single write yet. We are going to upload 100 bytes to the relative offset of 100 into the second block:\r\n\r\n```\r\nHEAD /files/bar HTTP/1.1\r\n\r\nHTTP/1.1 204 No Content\r\nEnity-Length: 5120\r\nBlocksize: 2048\r\nBlock-Offset: 2048, 100, 0\r\n```\r\n```\r\nPATCH /files/bar HTTP/1.1\r\nContent-Length: 100\r\nOffset: 2148\r\n\r\n[100 bytes]\r\n\r\nHTTP/1.1 204 No Content\r\n```\r\n```\r\nHEAD /files/bar HTTP/1.1\r\n\r\nHTTP/1.1 204 No Content\r\nEnity-Length: 5120\r\nBlocksize: 2048\r\nBlock-Offset: 2048, 200, 0\r\n```\r\n\r\n------\r\n\r\nPlease post your opinion about these solutions (I prefer the my last proposal) or any additional way we could achieve parallel and non-contiguous uploads. Also take the time to consider the work of implementations for servers and clients."}}