{"action":"created","comment":{"url":"https://api.github.com/repos/biocore/emp/pulls/comments/48457233","id":48457233,"diff_hunk":"@@ -0,0 +1,129 @@\n+\n+## Generate MIMS-compliant metadata template specific to environmental package\n+\n+Environmental package is a [MIGS/MIMS/MIMARKS](http://gensc.org/mixs) extension for reporting of measurements and observations obtained from one or more of the environments where the sample was obtained. All environmental packages listed here are described in README files generated by this script. By giving the name of the environmental package, a selection of fields can be made from the subtables, a template generated by this script for the user to record obtained values.\n+\n+Environmental packages available:\n+\n+* air\n+* built environment\n+* host-associated\n+* human-associated\n+* human-skin\n+* human-oral\n+* human-gut\n+* human-vaginal\n+* microbial mat/biofilm\n+* misc environment\n+* plant-associated\n+* sediment\n+* soil\n+* wastewater/sludge\n+* water\n+\n+### Import libraries\n+\n+\n+```python\n+import pandas as pd\n+import re\n+```\n+\n+### Define variables\n+\n+\n+```python\n+# required fields for Qiita, EBI, and MIMS (general)\n+qiita_ebi_mims_path = '../data/MIxS/Qiita_EBI_MIMS_v1.xlsx'\n+\n+# required fields for environmental packages\n+migs_mims_path = '../data/MIxS/MIGS_MIMS_v4.xls'\n+\n+# list one environmental package e.g. soil or multiple e.g. soil,water (one recommended)\n+list_of_env_pkg = 'soil'\n+\n+# number of samples for specified environmental package\n+number_of_samples = 10\n+\n+# PI name followed by study number\n+sample_prefix = 'Jones99'\n+\n+# used to pre-populate template\n+investigation_type = 'metagenome'\n+```\n+\n+### Function: return dataframe of environmental package-specific metadata items\n+\n+\n+```python\n+# a single environmental package (soil) or list can be provided (soil,water)\n+def show_items_of_env_pkg(df_env_pkg, list_of_env_pkg):\n+    df_items = df_env_pkg[df_env_pkg['Environmental package'].isin(list_of_env_pkg)]\n+    return df_items\n+```\n+\n+### Function: return dataframe of metadata template\n+\n+\n+```python\n+def create_template_for_env_pkg(df_QiitaEBI, df_MIMS, df_env_pkg, list_of_env_pkg, number_of_samples, sample_prefix):\n+    \n+    # get headers/requirement/example of Qiita-EBI/MIMS/env_pkg columns\n+    pkg_items = show_items_of_env_pkg(df_env_pkg, list_of_env_pkg)\n+    headers_env_pkg = pkg_items['Structured comment name'].values\n+    require_env_pkg = pkg_items['Requirement']\n+    example_env_pkg = pkg_items['Value syntax']\n+    headers_all = list(df_QiitaEBI.iloc[0]) + list(df_MIMS.iloc[0]) + list(headers_env_pkg)\n+    require_all = pd.concat([df_QiitaEBI.iloc[1], df_MIMS.iloc[1], require_env_pkg])\n+    example_all = pd.concat([df_QiitaEBI.iloc[2], df_MIMS.iloc[2], example_env_pkg])\n+    \n+    # populate template dataframe\n+    df_template = pd.DataFrame(columns=headers_all, dtype=object)\n+    df_template.loc['Requirement'] = require_all.values\n+    df_template.loc['Format'] = example_all.values\n+    string_of_env_pkg = re.sub(r'\\W', '.', '.'.join(list_of_env_pkg))\n+    for i in range(0, number_of_samples):\n+        df_template.loc[i+1] = ['' for x in range(len(df_template.columns))]\n+        df_template.loc[i+1]['sample_name'] = '%s.%s.%s' % (sample_prefix, string_of_env_pkg, i+1)\n+        df_template.loc[i+1]['investigation_type'] = investigation_type\n+        df_template.loc[i+1]['env_package'] = ' or '.join(list_of_env_pkg)\n+    return df_template\n+```\n+\n+### Main function: generate metadata template and readme csv files\n+\n+\n+```python\n+def generate_metadata_template(qiita_ebi_mims_path, migs_mims_path, list_of_env_pkg, number_of_samples, sample_prefix):\n+    ","path":"ipynb/metadata_template_generator.md","position":98,"original_position":98,"commit_id":"3f9fdbd07cf22cf52e8bfdba0a7b1e3cba2f5a26","original_commit_id":"3f9fdbd07cf22cf52e8bfdba0a7b1e3cba2f5a26","user":{"login":"mortonjt","id":4184797,"avatar_url":"https://avatars.githubusercontent.com/u/4184797?v=3","gravatar_id":"","url":"https://api.github.com/users/mortonjt","html_url":"https://github.com/mortonjt","followers_url":"https://api.github.com/users/mortonjt/followers","following_url":"https://api.github.com/users/mortonjt/following{/other_user}","gists_url":"https://api.github.com/users/mortonjt/gists{/gist_id}","starred_url":"https://api.github.com/users/mortonjt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mortonjt/subscriptions","organizations_url":"https://api.github.com/users/mortonjt/orgs","repos_url":"https://api.github.com/users/mortonjt/repos","events_url":"https://api.github.com/users/mortonjt/events{/privacy}","received_events_url":"https://api.github.com/users/mortonjt/received_events","type":"User","site_admin":false},"body":"[Numpy documenation standards](https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt)\r\n\r\nIts to clarify what the function does, what parameters are being pass into the function and what's being returned.","created_at":"2015-12-27T18:47:40Z","updated_at":"2015-12-27T18:47:40Z","html_url":"https://github.com/biocore/emp/pull/42#discussion_r48457233","pull_request_url":"https://api.github.com/repos/biocore/emp/pulls/42","_links":{"self":{"href":"https://api.github.com/repos/biocore/emp/pulls/comments/48457233"},"html":{"href":"https://github.com/biocore/emp/pull/42#discussion_r48457233"},"pull_request":{"href":"https://api.github.com/repos/biocore/emp/pulls/42"}}},"pull_request":{"url":"https://api.github.com/repos/biocore/emp/pulls/42","id":54426358,"html_url":"https://github.com/biocore/emp/pull/42","diff_url":"https://github.com/biocore/emp/pull/42.diff","patch_url":"https://github.com/biocore/emp/pull/42.patch","issue_url":"https://api.github.com/repos/biocore/emp/issues/42","number":42,"state":"open","locked":false,"title":"Script and data to generate metadata templates for EMP","user":{"login":"cuttlefishh","id":5685094,"avatar_url":"https://avatars.githubusercontent.com/u/5685094?v=3","gravatar_id":"","url":"https://api.github.com/users/cuttlefishh","html_url":"https://github.com/cuttlefishh","followers_url":"https://api.github.com/users/cuttlefishh/followers","following_url":"https://api.github.com/users/cuttlefishh/following{/other_user}","gists_url":"https://api.github.com/users/cuttlefishh/gists{/gist_id}","starred_url":"https://api.github.com/users/cuttlefishh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cuttlefishh/subscriptions","organizations_url":"https://api.github.com/users/cuttlefishh/orgs","repos_url":"https://api.github.com/users/cuttlefishh/repos","events_url":"https://api.github.com/users/cuttlefishh/events{/privacy}","received_events_url":"https://api.github.com/users/cuttlefishh/received_events","type":"User","site_admin":false},"body":"I've given @ackermag a script she can run on her laptop to generate the metadata templates. I've posted ipython notebook as .ipynb and .md so others can look at the code.","created_at":"2015-12-22T23:19:40Z","updated_at":"2015-12-27T18:47:40Z","closed_at":null,"merged_at":null,"merge_commit_sha":"cca216068a45d273ac9d0c27fda3ba80c7590b9a","assignee":{"login":"cuttlefishh","id":5685094,"avatar_url":"https://avatars.githubusercontent.com/u/5685094?v=3","gravatar_id":"","url":"https://api.github.com/users/cuttlefishh","html_url":"https://github.com/cuttlefishh","followers_url":"https://api.github.com/users/cuttlefishh/followers","following_url":"https://api.github.com/users/cuttlefishh/following{/other_user}","gists_url":"https://api.github.com/users/cuttlefishh/gists{/gist_id}","starred_url":"https://api.github.com/users/cuttlefishh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cuttlefishh/subscriptions","organizations_url":"https://api.github.com/users/cuttlefishh/orgs","repos_url":"https://api.github.com/users/cuttlefishh/repos","events_url":"https://api.github.com/users/cuttlefishh/events{/privacy}","received_events_url":"https://api.github.com/users/cuttlefishh/received_events","type":"User","site_admin":false},"milestone":null,"commits_url":"https://api.github.com/repos/biocore/emp/pulls/42/commits","review_comments_url":"https://api.github.com/repos/biocore/emp/pulls/42/comments","review_comment_url":"https://api.github.com/repos/biocore/emp/pulls/comments{/number}","comments_url":"https://api.github.com/repos/biocore/emp/issues/42/comments","statuses_url":"https://api.github.com/repos/biocore/emp/statuses/3f9fdbd07cf22cf52e8bfdba0a7b1e3cba2f5a26","head":{"label":"cuttlefishh:metadata","ref":"metadata","sha":"3f9fdbd07cf22cf52e8bfdba0a7b1e3cba2f5a26","user":{"login":"cuttlefishh","id":5685094,"avatar_url":"https://avatars.githubusercontent.com/u/5685094?v=3","gravatar_id":"","url":"https://api.github.com/users/cuttlefishh","html_url":"https://github.com/cuttlefishh","followers_url":"https://api.github.com/users/cuttlefishh/followers","following_url":"https://api.github.com/users/cuttlefishh/following{/other_user}","gists_url":"https://api.github.com/users/cuttlefishh/gists{/gist_id}","starred_url":"https://api.github.com/users/cuttlefishh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cuttlefishh/subscriptions","organizations_url":"https://api.github.com/users/cuttlefishh/orgs","repos_url":"https://api.github.com/users/cuttlefishh/repos","events_url":"https://api.github.com/users/cuttlefishh/events{/privacy}","received_events_url":"https://api.github.com/users/cuttlefishh/received_events","type":"User","site_admin":false},"repo":{"id":47724778,"name":"emp","full_name":"cuttlefishh/emp","owner":{"login":"cuttlefishh","id":5685094,"avatar_url":"https://avatars.githubusercontent.com/u/5685094?v=3","gravatar_id":"","url":"https://api.github.com/users/cuttlefishh","html_url":"https://github.com/cuttlefishh","followers_url":"https://api.github.com/users/cuttlefishh/followers","following_url":"https://api.github.com/users/cuttlefishh/following{/other_user}","gists_url":"https://api.github.com/users/cuttlefishh/gists{/gist_id}","starred_url":"https://api.github.com/users/cuttlefishh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cuttlefishh/subscriptions","organizations_url":"https://api.github.com/users/cuttlefishh/orgs","repos_url":"https://api.github.com/users/cuttlefishh/repos","events_url":"https://api.github.com/users/cuttlefishh/events{/privacy}","received_events_url":"https://api.github.com/users/cuttlefishh/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/cuttlefishh/emp","description":"Data, code and analysis results for the Earth Microbiome Project.","fork":true,"url":"https://api.github.com/repos/cuttlefishh/emp","forks_url":"https://api.github.com/repos/cuttlefishh/emp/forks","keys_url":"https://api.github.com/repos/cuttlefishh/emp/keys{/key_id}","collaborators_url":"https://api.github.com/repos/cuttlefishh/emp/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/cuttlefishh/emp/teams","hooks_url":"https://api.github.com/repos/cuttlefishh/emp/hooks","issue_events_url":"https://api.github.com/repos/cuttlefishh/emp/issues/events{/number}","events_url":"https://api.github.com/repos/cuttlefishh/emp/events","assignees_url":"https://api.github.com/repos/cuttlefishh/emp/assignees{/user}","branches_url":"https://api.github.com/repos/cuttlefishh/emp/branches{/branch}","tags_url":"https://api.github.com/repos/cuttlefishh/emp/tags","blobs_url":"https://api.github.com/repos/cuttlefishh/emp/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/cuttlefishh/emp/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/cuttlefishh/emp/git/refs{/sha}","trees_url":"https://api.github.com/repos/cuttlefishh/emp/git/trees{/sha}","statuses_url":"https://api.github.com/repos/cuttlefishh/emp/statuses/{sha}","languages_url":"https://api.github.com/repos/cuttlefishh/emp/languages","stargazers_url":"https://api.github.com/repos/cuttlefishh/emp/stargazers","contributors_url":"https://api.github.com/repos/cuttlefishh/emp/contributors","subscribers_url":"https://api.github.com/repos/cuttlefishh/emp/subscribers","subscription_url":"https://api.github.com/repos/cuttlefishh/emp/subscription","commits_url":"https://api.github.com/repos/cuttlefishh/emp/commits{/sha}","git_commits_url":"https://api.github.com/repos/cuttlefishh/emp/git/commits{/sha}","comments_url":"https://api.github.com/repos/cuttlefishh/emp/comments{/number}","issue_comment_url":"https://api.github.com/repos/cuttlefishh/emp/issues/comments{/number}","contents_url":"https://api.github.com/repos/cuttlefishh/emp/contents/{+path}","compare_url":"https://api.github.com/repos/cuttlefishh/emp/compare/{base}...{head}","merges_url":"https://api.github.com/repos/cuttlefishh/emp/merges","archive_url":"https://api.github.com/repos/cuttlefishh/emp/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/cuttlefishh/emp/downloads","issues_url":"https://api.github.com/repos/cuttlefishh/emp/issues{/number}","pulls_url":"https://api.github.com/repos/cuttlefishh/emp/pulls{/number}","milestones_url":"https://api.github.com/repos/cuttlefishh/emp/milestones{/number}","notifications_url":"https://api.github.com/repos/cuttlefishh/emp/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/cuttlefishh/emp/labels{/name}","releases_url":"https://api.github.com/repos/cuttlefishh/emp/releases{/id}","created_at":"2015-12-09T23:06:53Z","updated_at":"2015-12-09T23:06:54Z","pushed_at":"2015-12-27T07:48:37Z","git_url":"git://github.com/cuttlefishh/emp.git","ssh_url":"git@github.com:cuttlefishh/emp.git","clone_url":"https://github.com/cuttlefishh/emp.git","svn_url":"https://github.com/cuttlefishh/emp","homepage":"www.earthmicrobiome.org","size":805922,"stargazers_count":0,"watchers_count":0,"language":"OpenEdge ABL","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"biocore:master","ref":"master","sha":"b30c92bfb87ab5ae18a49f1c5579681722ce0d56","user":{"login":"biocore","id":6241857,"avatar_url":"https://avatars.githubusercontent.com/u/6241857?v=3","gravatar_id":"","url":"https://api.github.com/users/biocore","html_url":"https://github.com/biocore","followers_url":"https://api.github.com/users/biocore/followers","following_url":"https://api.github.com/users/biocore/following{/other_user}","gists_url":"https://api.github.com/users/biocore/gists{/gist_id}","starred_url":"https://api.github.com/users/biocore/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/biocore/subscriptions","organizations_url":"https://api.github.com/users/biocore/orgs","repos_url":"https://api.github.com/users/biocore/repos","events_url":"https://api.github.com/users/biocore/events{/privacy}","received_events_url":"https://api.github.com/users/biocore/received_events","type":"Organization","site_admin":false},"repo":{"id":5103350,"name":"emp","full_name":"biocore/emp","owner":{"login":"biocore","id":6241857,"avatar_url":"https://avatars.githubusercontent.com/u/6241857?v=3","gravatar_id":"","url":"https://api.github.com/users/biocore","html_url":"https://github.com/biocore","followers_url":"https://api.github.com/users/biocore/followers","following_url":"https://api.github.com/users/biocore/following{/other_user}","gists_url":"https://api.github.com/users/biocore/gists{/gist_id}","starred_url":"https://api.github.com/users/biocore/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/biocore/subscriptions","organizations_url":"https://api.github.com/users/biocore/orgs","repos_url":"https://api.github.com/users/biocore/repos","events_url":"https://api.github.com/users/biocore/events{/privacy}","received_events_url":"https://api.github.com/users/biocore/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/biocore/emp","description":"Data, code and analysis results for the Earth Microbiome Project.","fork":false,"url":"https://api.github.com/repos/biocore/emp","forks_url":"https://api.github.com/repos/biocore/emp/forks","keys_url":"https://api.github.com/repos/biocore/emp/keys{/key_id}","collaborators_url":"https://api.github.com/repos/biocore/emp/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/biocore/emp/teams","hooks_url":"https://api.github.com/repos/biocore/emp/hooks","issue_events_url":"https://api.github.com/repos/biocore/emp/issues/events{/number}","events_url":"https://api.github.com/repos/biocore/emp/events","assignees_url":"https://api.github.com/repos/biocore/emp/assignees{/user}","branches_url":"https://api.github.com/repos/biocore/emp/branches{/branch}","tags_url":"https://api.github.com/repos/biocore/emp/tags","blobs_url":"https://api.github.com/repos/biocore/emp/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/biocore/emp/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/biocore/emp/git/refs{/sha}","trees_url":"https://api.github.com/repos/biocore/emp/git/trees{/sha}","statuses_url":"https://api.github.com/repos/biocore/emp/statuses/{sha}","languages_url":"https://api.github.com/repos/biocore/emp/languages","stargazers_url":"https://api.github.com/repos/biocore/emp/stargazers","contributors_url":"https://api.github.com/repos/biocore/emp/contributors","subscribers_url":"https://api.github.com/repos/biocore/emp/subscribers","subscription_url":"https://api.github.com/repos/biocore/emp/subscription","commits_url":"https://api.github.com/repos/biocore/emp/commits{/sha}","git_commits_url":"https://api.github.com/repos/biocore/emp/git/commits{/sha}","comments_url":"https://api.github.com/repos/biocore/emp/comments{/number}","issue_comment_url":"https://api.github.com/repos/biocore/emp/issues/comments{/number}","contents_url":"https://api.github.com/repos/biocore/emp/contents/{+path}","compare_url":"https://api.github.com/repos/biocore/emp/compare/{base}...{head}","merges_url":"https://api.github.com/repos/biocore/emp/merges","archive_url":"https://api.github.com/repos/biocore/emp/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/biocore/emp/downloads","issues_url":"https://api.github.com/repos/biocore/emp/issues{/number}","pulls_url":"https://api.github.com/repos/biocore/emp/pulls{/number}","milestones_url":"https://api.github.com/repos/biocore/emp/milestones{/number}","notifications_url":"https://api.github.com/repos/biocore/emp/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/biocore/emp/labels{/name}","releases_url":"https://api.github.com/repos/biocore/emp/releases{/id}","created_at":"2012-07-18T23:48:09Z","updated_at":"2015-12-15T22:54:06Z","pushed_at":"2015-12-27T07:48:38Z","git_url":"git://github.com/biocore/emp.git","ssh_url":"git@github.com:biocore/emp.git","clone_url":"https://github.com/biocore/emp.git","svn_url":"https://github.com/biocore/emp","homepage":"www.earthmicrobiome.org","size":805921,"stargazers_count":10,"watchers_count":10,"language":"OpenEdge ABL","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":10,"mirror_url":null,"open_issues_count":24,"forks":10,"open_issues":24,"watchers":10,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/biocore/emp/pulls/42"},"html":{"href":"https://github.com/biocore/emp/pull/42"},"issue":{"href":"https://api.github.com/repos/biocore/emp/issues/42"},"comments":{"href":"https://api.github.com/repos/biocore/emp/issues/42/comments"},"review_comments":{"href":"https://api.github.com/repos/biocore/emp/pulls/42/comments"},"review_comment":{"href":"https://api.github.com/repos/biocore/emp/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/biocore/emp/pulls/42/commits"},"statuses":{"href":"https://api.github.com/repos/biocore/emp/statuses/3f9fdbd07cf22cf52e8bfdba0a7b1e3cba2f5a26"}}}}