{"action":"created","comment":{"url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/pulls/comments/43225875","id":43225875,"diff_hunk":"@@ -0,0 +1,178 @@\n+.. _qep#[.#]:\n+\n+========================================================================\n+QGIS Enhancement ??: Auxiliary storage\n+========================================================================\n+\n+:Date: October 2015\n+:Author: Hugo Mercier\n+:Contact: hugo dot mercier at oslandia dot com\n+:Last Edited: \n+:Status:  \n+:Version:\n+\n+Summary\n+-------\n+\n+This proposal is about an evolution of QGIS aiming at storing auxiliary data\n+in a layer. Such auxiliary data are data used mostly for the needs of QGIS (symbology) and have no real\n+interest in being stored with the native raw geospatial data.\n+\n+The need arises from the restrictions existing in the manual placement of labels.\n+Manual placement of labels are possible in QGIS by setting some labeling properties (X and Y position,\n+and rotation angle optionaly) as being \"data-defined\", meaning that values come from a column (or an expression).\n+But setting this up on an existing layer requires either to add new columns to the source layer, while it is\n+not always possible or desirable. Creating another table and using joins to it is possible but the synchronisation\n+between the two tables is not possible, leading to a solution not satisfactory.\n+\n+We propose here a more integrated solution where the creation of auxiliary columns or layers\n+are brought more transparently to the user by storing them in a central repository carried by the\n+project file.\n+\n+Requirements\n+------------\n+\n+The proposed solution have the following requirements :\n+\n+- It should allow to transparently add columns of any type to a layer, without adding them to the source data source.\n+Beside usual types (string, real, integer), we may also think of geometry types. We only consider types that can be converted to atomic types (string for WKT or WKB)\n+The user should be able to edit values in these columns when the layer is in editing mode.\n+\n+- These values should not be stored with the source data, and should be carried by the project, i.e.\n+when the project file is moved or copied, the auxiliary data should still be there.\n+\n+- Adding a new row to the layer also adds a new row for each auxiliary field (with a null or default value)\n+\n+- Removing a row from the layer also removes the corresponding row of each auxiliary field\n+\n+- New GUI elements should ease the creation of auxiliary fields\n+  - on each \"data-defined\" property\n+  - possibly on groups of \"data-defined\": for manual label placement or label connectors\n+\n+- It should be possible to save auxiliary data to a new layer\n+\n+Proposed solution\n+-----------------\n+\n+We propose to add a new type of field, close to the existing \"virtual field\" (where a field can be defined as an expression).\n+Here a new type of field, \"auxiliary field\", will point to a central data storage.\n+\n+The central data storage should allow to store any kind of auxiliary column definition and data for any layer. And it should\n+be linked to, or be part of, a project file. The access to auxiliary data should be efficient.\n+\n+Core changes\n+------------\n+\n+Project file\n+------------\n+\n+The current project file format (XML) is too limited to carry an efficient storage of such auxiliary data. It should somehow be evolved\n+to include a binary indexable format. SQLite is chosen as a first candidate for this data format. But it might evolve if performance issues\n+are found.\n+","path":"QEP-auxiliary-storage.rst","position":72,"original_position":72,"commit_id":"e2b9cda51c0bad3908c9c6fe6ed6b3a44fec7b6a","original_commit_id":"e2b9cda51c0bad3908c9c6fe6ed6b3a44fec7b6a","user":{"login":"mhugo","id":1618556,"avatar_url":"https://avatars.githubusercontent.com/u/1618556?v=3","gravatar_id":"","url":"https://api.github.com/users/mhugo","html_url":"https://github.com/mhugo","followers_url":"https://api.github.com/users/mhugo/followers","following_url":"https://api.github.com/users/mhugo/following{/other_user}","gists_url":"https://api.github.com/users/mhugo/gists{/gist_id}","starred_url":"https://api.github.com/users/mhugo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mhugo/subscriptions","organizations_url":"https://api.github.com/users/mhugo/orgs","repos_url":"https://api.github.com/users/mhugo/repos","events_url":"https://api.github.com/users/mhugo/events{/privacy}","received_events_url":"https://api.github.com/users/mhugo/received_events","type":"User","site_admin":false},"body":"Yes that what I had in mind. I'll add it to the doc","created_at":"2015-10-28T08:03:10Z","updated_at":"2015-10-28T08:03:10Z","html_url":"https://github.com/qgis/QGIS-Enhancement-Proposals/pull/26#discussion_r43225875","pull_request_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/pulls/26","_links":{"self":{"href":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/pulls/comments/43225875"},"html":{"href":"https://github.com/qgis/QGIS-Enhancement-Proposals/pull/26#discussion_r43225875"},"pull_request":{"href":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/pulls/26"}}},"pull_request":{"url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/pulls/26","id":48873764,"html_url":"https://github.com/qgis/QGIS-Enhancement-Proposals/pull/26","diff_url":"https://github.com/qgis/QGIS-Enhancement-Proposals/pull/26.diff","patch_url":"https://github.com/qgis/QGIS-Enhancement-Proposals/pull/26.patch","issue_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/issues/26","number":26,"state":"open","locked":false,"title":"Advanced labeling","user":{"login":"mhugo","id":1618556,"avatar_url":"https://avatars.githubusercontent.com/u/1618556?v=3","gravatar_id":"","url":"https://api.github.com/users/mhugo","html_url":"https://github.com/mhugo","followers_url":"https://api.github.com/users/mhugo/followers","following_url":"https://api.github.com/users/mhugo/following{/other_user}","gists_url":"https://api.github.com/users/mhugo/gists{/gist_id}","starred_url":"https://api.github.com/users/mhugo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mhugo/subscriptions","organizations_url":"https://api.github.com/users/mhugo/orgs","repos_url":"https://api.github.com/users/mhugo/repos","events_url":"https://api.github.com/users/mhugo/events{/privacy}","received_events_url":"https://api.github.com/users/mhugo/received_events","type":"User","site_admin":false},"body":"Here are some new QEPs related to advanced manual labeling.\r\n\r\nIt comes from user needs (QEP writing funded by Agence de l'Eau Adour Garonne).\r\n\r\nThis is a first version of the QEPs. I've tried to translate the user needs into generic enhancements to the core. I would like to collect feedback from reviewers regarding the proposed design and architecture, before focusing on implementation details, since there might be better / simpler ways to achieve the same goals.\r\n\r\nThe user needs are:\r\n- to ease the way labels can be manually placed by the user. It requires currently to have additional fields to store X, Y (and angle) values for labels. That led me to propose a generic way to store \"auxiliary\" data (you're welcome if you have a better name) linked to a layer. See the [QEP document on auxiliary storage](https://github.com/mhugo/QGIS-Enhancement-Proposals/blob/adv_labeling/QEP-auxiliary-storage.rst)\r\n- to allow the use of a simplified geometry for the curved labeling of waterways. This leads me to the [QEP on \"label path\"](https://github.com/mhugo/QGIS-Enhancement-Proposals/blob/adv_labeling/QEP-label-path.rst)\r\n- to allow to precisely place isolated words or letters of a label. This leads me to the [QEP on \"label annotations\"](https://github.com/mhugo/QGIS-Enhancement-Proposals/blob/adv_labeling/QEP-label-annotations.rst)\r\n- to allow to hide labels when a \"mask\" layer is used. The [QEP on \"label mask\"](https://github.com/mhugo/QGIS-Enhancement-Proposals/blob/adv_labeling/QEP-label-mask.rst) is about that.\r\n\r\n","created_at":"2015-10-27T13:59:30Z","updated_at":"2015-10-28T08:03:10Z","closed_at":null,"merged_at":null,"merge_commit_sha":"a1e6eda83bb01e662e87c74f355e7598dd8ca6f4","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/pulls/26/commits","review_comments_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/pulls/26/comments","review_comment_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/pulls/comments{/number}","comments_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/issues/26/comments","statuses_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/statuses/e2b9cda51c0bad3908c9c6fe6ed6b3a44fec7b6a","head":{"label":"mhugo:adv_labeling","ref":"adv_labeling","sha":"e2b9cda51c0bad3908c9c6fe6ed6b3a44fec7b6a","user":{"login":"mhugo","id":1618556,"avatar_url":"https://avatars.githubusercontent.com/u/1618556?v=3","gravatar_id":"","url":"https://api.github.com/users/mhugo","html_url":"https://github.com/mhugo","followers_url":"https://api.github.com/users/mhugo/followers","following_url":"https://api.github.com/users/mhugo/following{/other_user}","gists_url":"https://api.github.com/users/mhugo/gists{/gist_id}","starred_url":"https://api.github.com/users/mhugo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mhugo/subscriptions","organizations_url":"https://api.github.com/users/mhugo/orgs","repos_url":"https://api.github.com/users/mhugo/repos","events_url":"https://api.github.com/users/mhugo/events{/privacy}","received_events_url":"https://api.github.com/users/mhugo/received_events","type":"User","site_admin":false},"repo":{"id":23962109,"name":"QGIS-Enhancement-Proposals","full_name":"mhugo/QGIS-Enhancement-Proposals","owner":{"login":"mhugo","id":1618556,"avatar_url":"https://avatars.githubusercontent.com/u/1618556?v=3","gravatar_id":"","url":"https://api.github.com/users/mhugo","html_url":"https://github.com/mhugo","followers_url":"https://api.github.com/users/mhugo/followers","following_url":"https://api.github.com/users/mhugo/following{/other_user}","gists_url":"https://api.github.com/users/mhugo/gists{/gist_id}","starred_url":"https://api.github.com/users/mhugo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mhugo/subscriptions","organizations_url":"https://api.github.com/users/mhugo/orgs","repos_url":"https://api.github.com/users/mhugo/repos","events_url":"https://api.github.com/users/mhugo/events{/privacy}","received_events_url":"https://api.github.com/users/mhugo/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/mhugo/QGIS-Enhancement-Proposals","description":"QEP's (QGIS Enhancement Proposals) are used in the process of creating and discussing new enhancements for QGiS","fork":true,"url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals","forks_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/forks","keys_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/keys{/key_id}","collaborators_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/teams","hooks_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/hooks","issue_events_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/issues/events{/number}","events_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/events","assignees_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/assignees{/user}","branches_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/branches{/branch}","tags_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/tags","blobs_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/git/refs{/sha}","trees_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/git/trees{/sha}","statuses_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/statuses/{sha}","languages_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/languages","stargazers_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/stargazers","contributors_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/contributors","subscribers_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/subscribers","subscription_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/subscription","commits_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/commits{/sha}","git_commits_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/git/commits{/sha}","comments_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/comments{/number}","issue_comment_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/issues/comments{/number}","contents_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/contents/{+path}","compare_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/compare/{base}...{head}","merges_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/merges","archive_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/downloads","issues_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/issues{/number}","pulls_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/pulls{/number}","milestones_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/milestones{/number}","notifications_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/labels{/name}","releases_url":"https://api.github.com/repos/mhugo/QGIS-Enhancement-Proposals/releases{/id}","created_at":"2014-09-12T13:39:38Z","updated_at":"2014-08-26T07:02:44Z","pushed_at":"2015-10-27T13:41:49Z","git_url":"git://github.com/mhugo/QGIS-Enhancement-Proposals.git","ssh_url":"git@github.com:mhugo/QGIS-Enhancement-Proposals.git","clone_url":"https://github.com/mhugo/QGIS-Enhancement-Proposals.git","svn_url":"https://github.com/mhugo/QGIS-Enhancement-Proposals","homepage":null,"size":1271,"stargazers_count":0,"watchers_count":0,"language":null,"has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"qgis:master","ref":"master","sha":"6c7e57fd90a0ec24054bb37fda6415ac49130ed6","user":{"login":"qgis","id":483444,"avatar_url":"https://avatars.githubusercontent.com/u/483444?v=3","gravatar_id":"","url":"https://api.github.com/users/qgis","html_url":"https://github.com/qgis","followers_url":"https://api.github.com/users/qgis/followers","following_url":"https://api.github.com/users/qgis/following{/other_user}","gists_url":"https://api.github.com/users/qgis/gists{/gist_id}","starred_url":"https://api.github.com/users/qgis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/qgis/subscriptions","organizations_url":"https://api.github.com/users/qgis/orgs","repos_url":"https://api.github.com/users/qgis/repos","events_url":"https://api.github.com/users/qgis/events{/privacy}","received_events_url":"https://api.github.com/users/qgis/received_events","type":"Organization","site_admin":false},"repo":{"id":23342140,"name":"QGIS-Enhancement-Proposals","full_name":"qgis/QGIS-Enhancement-Proposals","owner":{"login":"qgis","id":483444,"avatar_url":"https://avatars.githubusercontent.com/u/483444?v=3","gravatar_id":"","url":"https://api.github.com/users/qgis","html_url":"https://github.com/qgis","followers_url":"https://api.github.com/users/qgis/followers","following_url":"https://api.github.com/users/qgis/following{/other_user}","gists_url":"https://api.github.com/users/qgis/gists{/gist_id}","starred_url":"https://api.github.com/users/qgis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/qgis/subscriptions","organizations_url":"https://api.github.com/users/qgis/orgs","repos_url":"https://api.github.com/users/qgis/repos","events_url":"https://api.github.com/users/qgis/events{/privacy}","received_events_url":"https://api.github.com/users/qgis/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/qgis/QGIS-Enhancement-Proposals","description":"QEP's (QGIS Enhancement Proposals) are used in the process of creating and discussing new enhancements for QGIS","fork":false,"url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals","forks_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/forks","keys_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/keys{/key_id}","collaborators_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/teams","hooks_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/hooks","issue_events_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/issues/events{/number}","events_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/events","assignees_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/assignees{/user}","branches_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/branches{/branch}","tags_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/tags","blobs_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/git/refs{/sha}","trees_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/git/trees{/sha}","statuses_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/statuses/{sha}","languages_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/languages","stargazers_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/stargazers","contributors_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/contributors","subscribers_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/subscribers","subscription_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/subscription","commits_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/commits{/sha}","git_commits_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/git/commits{/sha}","comments_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/comments{/number}","issue_comment_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/issues/comments{/number}","contents_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/contents/{+path}","compare_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/compare/{base}...{head}","merges_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/merges","archive_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/downloads","issues_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/issues{/number}","pulls_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/pulls{/number}","milestones_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/milestones{/number}","notifications_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/labels{/name}","releases_url":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/releases{/id}","created_at":"2014-08-26T07:02:44Z","updated_at":"2015-10-02T06:59:28Z","pushed_at":"2015-10-27T23:19:33Z","git_url":"git://github.com/qgis/QGIS-Enhancement-Proposals.git","ssh_url":"git@github.com:qgis/QGIS-Enhancement-Proposals.git","clone_url":"https://github.com/qgis/QGIS-Enhancement-Proposals.git","svn_url":"https://github.com/qgis/QGIS-Enhancement-Proposals","homepage":"","size":2413,"stargazers_count":3,"watchers_count":3,"language":null,"has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":23,"mirror_url":null,"open_issues_count":17,"forks":23,"open_issues":17,"watchers":3,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/pulls/26"},"html":{"href":"https://github.com/qgis/QGIS-Enhancement-Proposals/pull/26"},"issue":{"href":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/issues/26"},"comments":{"href":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/issues/26/comments"},"review_comments":{"href":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/pulls/26/comments"},"review_comment":{"href":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/pulls/26/commits"},"statuses":{"href":"https://api.github.com/repos/qgis/QGIS-Enhancement-Proposals/statuses/e2b9cda51c0bad3908c9c6fe6ed6b3a44fec7b6a"}}}}