{"action":"created","comment":{"url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/pulls/comments/48063414","id":48063414,"diff_hunk":"@@ -0,0 +1,155 @@\n+\"\"\"Utility to read configs for spark magic.\n+\"\"\"\n+# Copyright (c) 2015  aggftw@gmail.com\n+# Distributed under the terms of the Modified BSD License.\n+\n+import json\n+\n+from remotespark.utils.constants import Constants\n+from remotespark.utils.utils import join_paths, get_magics_home_path, ensure_path_exists, ensure_file_exists\n+\n+_overrides = None\n+\n+\n+def initialize():\n+    \"\"\"Checks if the configuration is initialized. If so, initializes the\n+    global configuration object by reading from the configuration\n+    file, overwriting the current set of overrides if there is one\"\"\"\n+    global _overrides\n+    if _overrides is None:\n+        load()\n+\n+\n+def load():\n+    \"\"\"Initializes the global configuration by reading from the configuration\n+    file, overwriting the current set of overrides if there is one\"\"\"\n+    home_path = get_magics_home_path()\n+    ensure_path_exists(home_path)\n+    config_file = join_paths(home_path, Constants.config_json)\n+    ensure_file_exists(config_file)\n+    with open(config_file, \"r+\") as f:\n+        config_text = f.readlines()\n+        line = \"\".join(config_text).strip()\n+        if line == \"\":\n+            overrides = {}\n+        else:\n+            overrides = json.loads(line)\n+    override(overrides)\n+\n+\n+def override(obj):\n+    \"\"\"Given a dictionary representing the overrided defaults for this\n+    configuration, initialize the global configuration.\"\"\"\n+    global _overrides\n+    _overrides = obj\n+\n+\n+def _override(f):\n+    \"\"\"A decorator which first initializes the overrided configurations,","path":"remotespark/utils/configuration.py","position":48,"original_position":48,"commit_id":"37c0280083ca0c3b46908604a2bcc47d5a51b12d","original_commit_id":"37c0280083ca0c3b46908604a2bcc47d5a51b12d","user":{"login":"aggFTW","id":654204,"avatar_url":"https://avatars.githubusercontent.com/u/654204?v=3","gravatar_id":"","url":"https://api.github.com/users/aggFTW","html_url":"https://github.com/aggFTW","followers_url":"https://api.github.com/users/aggFTW/followers","following_url":"https://api.github.com/users/aggFTW/following{/other_user}","gists_url":"https://api.github.com/users/aggFTW/gists{/gist_id}","starred_url":"https://api.github.com/users/aggFTW/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aggFTW/subscriptions","organizations_url":"https://api.github.com/users/aggFTW/orgs","repos_url":"https://api.github.com/users/aggFTW/repos","events_url":"https://api.github.com/users/aggFTW/events{/privacy}","received_events_url":"https://api.github.com/users/aggFTW/received_events","type":"User","site_admin":false},"body":"Please add unit test for the decorator","created_at":"2015-12-18T19:57:41Z","updated_at":"2015-12-18T19:57:41Z","html_url":"https://github.com/jupyter-incubator/sparkmagic/pull/77#discussion_r48063414","pull_request_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/pulls/77","_links":{"self":{"href":"https://api.github.com/repos/jupyter-incubator/sparkmagic/pulls/comments/48063414"},"html":{"href":"https://github.com/jupyter-incubator/sparkmagic/pull/77#discussion_r48063414"},"pull_request":{"href":"https://api.github.com/repos/jupyter-incubator/sparkmagic/pulls/77"}}},"pull_request":{"url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/pulls/77","id":54137625,"html_url":"https://github.com/jupyter-incubator/sparkmagic/pull/77","diff_url":"https://github.com/jupyter-incubator/sparkmagic/pull/77.diff","patch_url":"https://github.com/jupyter-incubator/sparkmagic/pull/77.patch","issue_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/issues/77","number":77,"state":"open","locked":false,"title":"Rethink configuration","user":{"login":"msftristew","id":13972471,"avatar_url":"https://avatars.githubusercontent.com/u/13972471?v=3","gravatar_id":"","url":"https://api.github.com/users/msftristew","html_url":"https://github.com/msftristew","followers_url":"https://api.github.com/users/msftristew/followers","following_url":"https://api.github.com/users/msftristew/following{/other_user}","gists_url":"https://api.github.com/users/msftristew/gists{/gist_id}","starred_url":"https://api.github.com/users/msftristew/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/msftristew/subscriptions","organizations_url":"https://api.github.com/users/msftristew/orgs","repos_url":"https://api.github.com/users/msftristew/repos","events_url":"https://api.github.com/users/msftristew/events{/privacy}","received_events_url":"https://api.github.com/users/msftristew/received_events","type":"User","site_admin":false},"body":"This change improves the extensibility and functionality of the configuration module. The following changes have been made:\r\n\r\n* Begin reorganizing the repository by creating a new `remotespark.utils` subdirectory, under which some common utilities are stored.\r\n\r\n* Change the API of the configuration module. The module now supports the following methods:\r\n\r\n        # loads a set of configuration overrides from the config file on disk\r\n        configuration.load() \r\n        # initializes the set of configuration overrides, loading iff the module has not already been initialized\r\n        configuration.initialize()\r\n        # given a dict which is a set of configuration overrides, override those and only those configurations\r\n        # this method subsumes the old hacky _t_config_hook function\r\n        configuration.override(object)\r\n\r\n* The configuration module now supports a function for each configuration. e.g. there is a function `configuration.default_chart_type()`, a function `configuration.serialize()`, a function `configuration.status_sleep_seconds()`, etc.  These all come with a default value if they were not overridden by user settings.\r\n\r\n* Some slight cleanup regarding the schema of configuration files (`default_config.json`)\r\n\r\n* Appropriate changes to the rest of the codebase and to the Constants class to accomodate these changes\r\n\r\nThese tasks remain:\r\n\r\n* Configuration is not split between magics/kernels/Livy client lib.  That refactoring can be done in a later pull request.\r\n\r\n* Some further changes to the directory structure of the package are probably still needed.  This is simply an incremental step in the correct direction.","created_at":"2015-12-18T19:04:04Z","updated_at":"2015-12-18T19:57:41Z","closed_at":null,"merged_at":null,"merge_commit_sha":"4c0ff652a50dee8cbe833d1cf97e3abb5ced4c90","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/pulls/77/commits","review_comments_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/pulls/77/comments","review_comment_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/pulls/comments{/number}","comments_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/issues/77/comments","statuses_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/statuses/37c0280083ca0c3b46908604a2bcc47d5a51b12d","head":{"label":"msftristew:rethinkconfig","ref":"rethinkconfig","sha":"37c0280083ca0c3b46908604a2bcc47d5a51b12d","user":{"login":"msftristew","id":13972471,"avatar_url":"https://avatars.githubusercontent.com/u/13972471?v=3","gravatar_id":"","url":"https://api.github.com/users/msftristew","html_url":"https://github.com/msftristew","followers_url":"https://api.github.com/users/msftristew/followers","following_url":"https://api.github.com/users/msftristew/following{/other_user}","gists_url":"https://api.github.com/users/msftristew/gists{/gist_id}","starred_url":"https://api.github.com/users/msftristew/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/msftristew/subscriptions","organizations_url":"https://api.github.com/users/msftristew/orgs","repos_url":"https://api.github.com/users/msftristew/repos","events_url":"https://api.github.com/users/msftristew/events{/privacy}","received_events_url":"https://api.github.com/users/msftristew/received_events","type":"User","site_admin":false},"repo":{"id":46371305,"name":"sparkmagic","full_name":"msftristew/sparkmagic","owner":{"login":"msftristew","id":13972471,"avatar_url":"https://avatars.githubusercontent.com/u/13972471?v=3","gravatar_id":"","url":"https://api.github.com/users/msftristew","html_url":"https://github.com/msftristew","followers_url":"https://api.github.com/users/msftristew/followers","following_url":"https://api.github.com/users/msftristew/following{/other_user}","gists_url":"https://api.github.com/users/msftristew/gists{/gist_id}","starred_url":"https://api.github.com/users/msftristew/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/msftristew/subscriptions","organizations_url":"https://api.github.com/users/msftristew/orgs","repos_url":"https://api.github.com/users/msftristew/repos","events_url":"https://api.github.com/users/msftristew/events{/privacy}","received_events_url":"https://api.github.com/users/msftristew/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/msftristew/sparkmagic","description":"IPython magics for working with remote Spark clusters","fork":true,"url":"https://api.github.com/repos/msftristew/sparkmagic","forks_url":"https://api.github.com/repos/msftristew/sparkmagic/forks","keys_url":"https://api.github.com/repos/msftristew/sparkmagic/keys{/key_id}","collaborators_url":"https://api.github.com/repos/msftristew/sparkmagic/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/msftristew/sparkmagic/teams","hooks_url":"https://api.github.com/repos/msftristew/sparkmagic/hooks","issue_events_url":"https://api.github.com/repos/msftristew/sparkmagic/issues/events{/number}","events_url":"https://api.github.com/repos/msftristew/sparkmagic/events","assignees_url":"https://api.github.com/repos/msftristew/sparkmagic/assignees{/user}","branches_url":"https://api.github.com/repos/msftristew/sparkmagic/branches{/branch}","tags_url":"https://api.github.com/repos/msftristew/sparkmagic/tags","blobs_url":"https://api.github.com/repos/msftristew/sparkmagic/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/msftristew/sparkmagic/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/msftristew/sparkmagic/git/refs{/sha}","trees_url":"https://api.github.com/repos/msftristew/sparkmagic/git/trees{/sha}","statuses_url":"https://api.github.com/repos/msftristew/sparkmagic/statuses/{sha}","languages_url":"https://api.github.com/repos/msftristew/sparkmagic/languages","stargazers_url":"https://api.github.com/repos/msftristew/sparkmagic/stargazers","contributors_url":"https://api.github.com/repos/msftristew/sparkmagic/contributors","subscribers_url":"https://api.github.com/repos/msftristew/sparkmagic/subscribers","subscription_url":"https://api.github.com/repos/msftristew/sparkmagic/subscription","commits_url":"https://api.github.com/repos/msftristew/sparkmagic/commits{/sha}","git_commits_url":"https://api.github.com/repos/msftristew/sparkmagic/git/commits{/sha}","comments_url":"https://api.github.com/repos/msftristew/sparkmagic/comments{/number}","issue_comment_url":"https://api.github.com/repos/msftristew/sparkmagic/issues/comments{/number}","contents_url":"https://api.github.com/repos/msftristew/sparkmagic/contents/{+path}","compare_url":"https://api.github.com/repos/msftristew/sparkmagic/compare/{base}...{head}","merges_url":"https://api.github.com/repos/msftristew/sparkmagic/merges","archive_url":"https://api.github.com/repos/msftristew/sparkmagic/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/msftristew/sparkmagic/downloads","issues_url":"https://api.github.com/repos/msftristew/sparkmagic/issues{/number}","pulls_url":"https://api.github.com/repos/msftristew/sparkmagic/pulls{/number}","milestones_url":"https://api.github.com/repos/msftristew/sparkmagic/milestones{/number}","notifications_url":"https://api.github.com/repos/msftristew/sparkmagic/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/msftristew/sparkmagic/labels{/name}","releases_url":"https://api.github.com/repos/msftristew/sparkmagic/releases{/id}","created_at":"2015-11-17T19:49:34Z","updated_at":"2015-11-17T19:49:35Z","pushed_at":"2015-12-18T19:27:47Z","git_url":"git://github.com/msftristew/sparkmagic.git","ssh_url":"git@github.com:msftristew/sparkmagic.git","clone_url":"https://github.com/msftristew/sparkmagic.git","svn_url":"https://github.com/msftristew/sparkmagic","homepage":null,"size":273,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"jupyter-incubator:master","ref":"master","sha":"8bf7642d3278719bed5fed1945e8b20a7d068b11","user":{"login":"jupyter-incubator","id":14002361,"avatar_url":"https://avatars.githubusercontent.com/u/14002361?v=3","gravatar_id":"","url":"https://api.github.com/users/jupyter-incubator","html_url":"https://github.com/jupyter-incubator","followers_url":"https://api.github.com/users/jupyter-incubator/followers","following_url":"https://api.github.com/users/jupyter-incubator/following{/other_user}","gists_url":"https://api.github.com/users/jupyter-incubator/gists{/gist_id}","starred_url":"https://api.github.com/users/jupyter-incubator/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jupyter-incubator/subscriptions","organizations_url":"https://api.github.com/users/jupyter-incubator/orgs","repos_url":"https://api.github.com/users/jupyter-incubator/repos","events_url":"https://api.github.com/users/jupyter-incubator/events{/privacy}","received_events_url":"https://api.github.com/users/jupyter-incubator/received_events","type":"Organization","site_admin":false},"repo":{"id":42875585,"name":"sparkmagic","full_name":"jupyter-incubator/sparkmagic","owner":{"login":"jupyter-incubator","id":14002361,"avatar_url":"https://avatars.githubusercontent.com/u/14002361?v=3","gravatar_id":"","url":"https://api.github.com/users/jupyter-incubator","html_url":"https://github.com/jupyter-incubator","followers_url":"https://api.github.com/users/jupyter-incubator/followers","following_url":"https://api.github.com/users/jupyter-incubator/following{/other_user}","gists_url":"https://api.github.com/users/jupyter-incubator/gists{/gist_id}","starred_url":"https://api.github.com/users/jupyter-incubator/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jupyter-incubator/subscriptions","organizations_url":"https://api.github.com/users/jupyter-incubator/orgs","repos_url":"https://api.github.com/users/jupyter-incubator/repos","events_url":"https://api.github.com/users/jupyter-incubator/events{/privacy}","received_events_url":"https://api.github.com/users/jupyter-incubator/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/jupyter-incubator/sparkmagic","description":"IPython magics for working with remote Spark clusters","fork":false,"url":"https://api.github.com/repos/jupyter-incubator/sparkmagic","forks_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/forks","keys_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/keys{/key_id}","collaborators_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/teams","hooks_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/hooks","issue_events_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/issues/events{/number}","events_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/events","assignees_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/assignees{/user}","branches_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/branches{/branch}","tags_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/tags","blobs_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/git/refs{/sha}","trees_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/git/trees{/sha}","statuses_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/statuses/{sha}","languages_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/languages","stargazers_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/stargazers","contributors_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/contributors","subscribers_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/subscribers","subscription_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/subscription","commits_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/commits{/sha}","git_commits_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/git/commits{/sha}","comments_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/comments{/number}","issue_comment_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/issues/comments{/number}","contents_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/contents/{+path}","compare_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/compare/{base}...{head}","merges_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/merges","archive_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/downloads","issues_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/issues{/number}","pulls_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/pulls{/number}","milestones_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/milestones{/number}","notifications_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/labels{/name}","releases_url":"https://api.github.com/repos/jupyter-incubator/sparkmagic/releases{/id}","created_at":"2015-09-21T15:35:25Z","updated_at":"2015-12-07T02:43:32Z","pushed_at":"2015-12-18T19:27:47Z","git_url":"git://github.com/jupyter-incubator/sparkmagic.git","ssh_url":"git@github.com:jupyter-incubator/sparkmagic.git","clone_url":"https://github.com/jupyter-incubator/sparkmagic.git","svn_url":"https://github.com/jupyter-incubator/sparkmagic","homepage":null,"size":273,"stargazers_count":15,"watchers_count":15,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":6,"mirror_url":null,"open_issues_count":17,"forks":6,"open_issues":17,"watchers":15,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/jupyter-incubator/sparkmagic/pulls/77"},"html":{"href":"https://github.com/jupyter-incubator/sparkmagic/pull/77"},"issue":{"href":"https://api.github.com/repos/jupyter-incubator/sparkmagic/issues/77"},"comments":{"href":"https://api.github.com/repos/jupyter-incubator/sparkmagic/issues/77/comments"},"review_comments":{"href":"https://api.github.com/repos/jupyter-incubator/sparkmagic/pulls/77/comments"},"review_comment":{"href":"https://api.github.com/repos/jupyter-incubator/sparkmagic/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/jupyter-incubator/sparkmagic/pulls/77/commits"},"statuses":{"href":"https://api.github.com/repos/jupyter-incubator/sparkmagic/statuses/37c0280083ca0c3b46908604a2bcc47d5a51b12d"}}}}