{"action":"created","issue":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4475","labels_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4475/labels{/name}","comments_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4475/comments","events_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4475/events","html_url":"https://github.com/scikit-learn/scikit-learn/issues/4475","id":65445257,"number":4475,"title":"TSNE with correlation metric: ValueError: Distance matrix 'X' must be symmetric","user":{"login":"cel4","id":3819470,"avatar_url":"https://avatars.githubusercontent.com/u/3819470?v=3","gravatar_id":"","url":"https://api.github.com/users/cel4","html_url":"https://github.com/cel4","followers_url":"https://api.github.com/users/cel4/followers","following_url":"https://api.github.com/users/cel4/following{/other_user}","gists_url":"https://api.github.com/users/cel4/gists{/gist_id}","starred_url":"https://api.github.com/users/cel4/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cel4/subscriptions","organizations_url":"https://api.github.com/users/cel4/orgs","repos_url":"https://api.github.com/users/cel4/repos","events_url":"https://api.github.com/users/cel4/events{/privacy}","received_events_url":"https://api.github.com/users/cel4/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/labels/Bug","name":"Bug","color":"e10c02"},{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/labels/Easy","name":"Easy","color":"02d7e1"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/milestones/15","html_url":"https://github.com/scikit-learn/scikit-learn/milestones/0.17","labels_url":"https://api.github.com/repos/scikit-learn/scikit-learn/milestones/15/labels","id":995666,"number":15,"title":"0.17","description":"","creator":{"login":"amueller","id":449558,"avatar_url":"https://avatars.githubusercontent.com/u/449558?v=3","gravatar_id":"","url":"https://api.github.com/users/amueller","html_url":"https://github.com/amueller","followers_url":"https://api.github.com/users/amueller/followers","following_url":"https://api.github.com/users/amueller/following{/other_user}","gists_url":"https://api.github.com/users/amueller/gists{/gist_id}","starred_url":"https://api.github.com/users/amueller/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amueller/subscriptions","organizations_url":"https://api.github.com/users/amueller/orgs","repos_url":"https://api.github.com/users/amueller/repos","events_url":"https://api.github.com/users/amueller/events{/privacy}","received_events_url":"https://api.github.com/users/amueller/received_events","type":"User","site_admin":false},"open_issues":27,"closed_issues":46,"state":"open","created_at":"2015-02-25T18:59:07Z","updated_at":"2015-09-29T13:43:23Z","due_on":"2015-09-20T04:00:00Z","closed_at":null},"comments":5,"created_at":"2015-03-31T13:04:10Z","updated_at":"2015-09-30T14:27:32Z","closed_at":null,"body":"```\r\nfrom sklearn.manifold import TSNE\r\nimport numpy as np\r\nnp.random.seed(42)\r\n\r\ndata = np.random.rand(10, 3)\r\ndata[-1, :] = 0\r\n\r\nmodel = TSNE(metric=\"correlation\")\r\nmodel.fit_transform(data)\r\n```\r\n\r\nTSNE raises an obscure error, when the data set contains rows with a standard deviation `0` and therefore undefined correlations:\r\n\r\n```\r\n---------------------------------------------------------------------------\r\nValueError                                Traceback (most recent call last)\r\n<ipython-input-20-658142c1e315> in <module>()\r\n      1 model = TSNE(metric=\"correlation\")\r\n----> 2 res = model.fit_transform(data)\r\n      3 ran = model.fit_transform(ran_data)\r\n\r\n/Users/ch/miniconda/envs/sci34/lib/python3.4/site-packages/sklearn/manifold/t_sne.py in fit_transform(self, X, y)\r\n    522             Embedding of the training data in low-dimensional space.\r\n    523         \"\"\"\r\n--> 524         self.fit(X)\r\n    525         return self.embedding_\r\n\r\n/Users/ch/miniconda/envs/sci34/lib/python3.4/site-packages/sklearn/manifold/t_sne.py in fit(self, X, y)\r\n    447         self.training_data_ = X\r\n    448 \r\n--> 449         P = _joint_probabilities(distances, self.perplexity, self.verbose)\r\n    450         if self.init == 'pca':\r\n    451             pca = RandomizedPCA(n_components=self.n_components,\r\n\r\n/Users/ch/miniconda/envs/sci34/lib/python3.4/site-packages/sklearn/manifold/t_sne.py in _joint_probabilities(distances, desired_perplexity, verbose)\r\n     52     P = conditional_P + conditional_P.T\r\n     53     sum_P = np.maximum(np.sum(P), MACHINE_EPSILON)\r\n---> 54     P = np.maximum(squareform(P) / sum_P, MACHINE_EPSILON)\r\n     55     return P\r\n     56 \r\n\r\n/Users/ch/miniconda/envs/sci34/lib/python3.4/site-packages/scipy/spatial/distance.py in squareform(X, force, checks)\r\n   1479             raise ValueError('The matrix argument must be square.')\r\n   1480         if checks:\r\n-> 1481             is_valid_dm(X, throw=True, name='X')\r\n   1482 \r\n   1483         # One-side of the dimensions is set here.\r\n\r\n/Users/ch/miniconda/envs/sci34/lib/python3.4/site-packages/scipy/spatial/distance.py in is_valid_dm(D, tol, throw, name, warning)\r\n   1562                 if name:\r\n   1563                     raise ValueError(('Distance matrix \\'%s\\' must be '\r\n-> 1564                                      'symmetric.') % name)\r\n   1565                 else:\r\n   1566                     raise ValueError('Distance matrix must be symmetric.')\r\n\r\nValueError: Distance matrix 'X' must be symmetric\r\n```"},"comment":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/comments/144429365","html_url":"https://github.com/scikit-learn/scikit-learn/issues/4475#issuecomment-144429365","issue_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4475","id":144429365,"user":{"login":"ogrisel","id":89061,"avatar_url":"https://avatars.githubusercontent.com/u/89061?v=3","gravatar_id":"","url":"https://api.github.com/users/ogrisel","html_url":"https://github.com/ogrisel","followers_url":"https://api.github.com/users/ogrisel/followers","following_url":"https://api.github.com/users/ogrisel/following{/other_user}","gists_url":"https://api.github.com/users/ogrisel/gists{/gist_id}","starred_url":"https://api.github.com/users/ogrisel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ogrisel/subscriptions","organizations_url":"https://api.github.com/users/ogrisel/orgs","repos_url":"https://api.github.com/users/ogrisel/repos","events_url":"https://api.github.com/users/ogrisel/events{/privacy}","received_events_url":"https://api.github.com/users/ogrisel/received_events","type":"User","site_admin":false},"created_at":"2015-09-30T14:27:32Z","updated_at":"2015-09-30T14:27:32Z","body":"For the cosine metric it should be fixed a posteriori by changings negative numbers with small absolute value (e.g. smaller than `10 * np.finfo(X.dtype).eps`) to be set to zero.\r\n\r\nFor the correlation distance, it's looks like a problem in scipy's `pdist`."}}