{"action":"created","issue":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4475","labels_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4475/labels{/name}","comments_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4475/comments","events_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4475/events","html_url":"https://github.com/scikit-learn/scikit-learn/issues/4475","id":65445257,"number":4475,"title":"TSNE with correlation metric: ValueError: Distance matrix 'X' must be symmetric","user":{"login":"cel4","id":3819470,"avatar_url":"https://avatars.githubusercontent.com/u/3819470?v=3","gravatar_id":"","url":"https://api.github.com/users/cel4","html_url":"https://github.com/cel4","followers_url":"https://api.github.com/users/cel4/followers","following_url":"https://api.github.com/users/cel4/following{/other_user}","gists_url":"https://api.github.com/users/cel4/gists{/gist_id}","starred_url":"https://api.github.com/users/cel4/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cel4/subscriptions","organizations_url":"https://api.github.com/users/cel4/orgs","repos_url":"https://api.github.com/users/cel4/repos","events_url":"https://api.github.com/users/cel4/events{/privacy}","received_events_url":"https://api.github.com/users/cel4/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/labels/Bug","name":"Bug","color":"e10c02"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-03-31T13:04:10Z","updated_at":"2015-03-31T23:31:13Z","closed_at":null,"body":"```\r\nfrom sklearn.manifold import TSNE\r\nimport numpy as np\r\nnp.random.seed(42)\r\n\r\ndata = np.random.rand(10, 3)\r\ndata[-1, :] = 0\r\n\r\nmodel = TSNE(metric=\"correlation\")\r\nmodel.fit_transform(data)\r\n```\r\n\r\nTSNE raises an obscure error, when the data set contains rows with a standard deviation `0` and therefore undefined correlations:\r\n\r\n```\r\n---------------------------------------------------------------------------\r\nValueError                                Traceback (most recent call last)\r\n<ipython-input-20-658142c1e315> in <module>()\r\n      1 model = TSNE(metric=\"correlation\")\r\n----> 2 res = model.fit_transform(data)\r\n      3 ran = model.fit_transform(ran_data)\r\n\r\n/Users/ch/miniconda/envs/sci34/lib/python3.4/site-packages/sklearn/manifold/t_sne.py in fit_transform(self, X, y)\r\n    522             Embedding of the training data in low-dimensional space.\r\n    523         \"\"\"\r\n--> 524         self.fit(X)\r\n    525         return self.embedding_\r\n\r\n/Users/ch/miniconda/envs/sci34/lib/python3.4/site-packages/sklearn/manifold/t_sne.py in fit(self, X, y)\r\n    447         self.training_data_ = X\r\n    448 \r\n--> 449         P = _joint_probabilities(distances, self.perplexity, self.verbose)\r\n    450         if self.init == 'pca':\r\n    451             pca = RandomizedPCA(n_components=self.n_components,\r\n\r\n/Users/ch/miniconda/envs/sci34/lib/python3.4/site-packages/sklearn/manifold/t_sne.py in _joint_probabilities(distances, desired_perplexity, verbose)\r\n     52     P = conditional_P + conditional_P.T\r\n     53     sum_P = np.maximum(np.sum(P), MACHINE_EPSILON)\r\n---> 54     P = np.maximum(squareform(P) / sum_P, MACHINE_EPSILON)\r\n     55     return P\r\n     56 \r\n\r\n/Users/ch/miniconda/envs/sci34/lib/python3.4/site-packages/scipy/spatial/distance.py in squareform(X, force, checks)\r\n   1479             raise ValueError('The matrix argument must be square.')\r\n   1480         if checks:\r\n-> 1481             is_valid_dm(X, throw=True, name='X')\r\n   1482 \r\n   1483         # One-side of the dimensions is set here.\r\n\r\n/Users/ch/miniconda/envs/sci34/lib/python3.4/site-packages/scipy/spatial/distance.py in is_valid_dm(D, tol, throw, name, warning)\r\n   1562                 if name:\r\n   1563                     raise ValueError(('Distance matrix \\'%s\\' must be '\r\n-> 1564                                      'symmetric.') % name)\r\n   1565                 else:\r\n   1566                     raise ValueError('Distance matrix must be symmetric.')\r\n\r\nValueError: Distance matrix 'X' must be symmetric\r\n```"},"comment":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/comments/88283240","html_url":"https://github.com/scikit-learn/scikit-learn/issues/4475#issuecomment-88283240","issue_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4475","id":88283240,"user":{"login":"amueller","id":449558,"avatar_url":"https://avatars.githubusercontent.com/u/449558?v=3","gravatar_id":"","url":"https://api.github.com/users/amueller","html_url":"https://github.com/amueller","followers_url":"https://api.github.com/users/amueller/followers","following_url":"https://api.github.com/users/amueller/following{/other_user}","gists_url":"https://api.github.com/users/amueller/gists{/gist_id}","starred_url":"https://api.github.com/users/amueller/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amueller/subscriptions","organizations_url":"https://api.github.com/users/amueller/orgs","repos_url":"https://api.github.com/users/amueller/repos","events_url":"https://api.github.com/users/amueller/events{/privacy}","received_events_url":"https://api.github.com/users/amueller/received_events","type":"User","site_admin":false},"created_at":"2015-03-31T23:31:13Z","updated_at":"2015-03-31T23:31:13Z","body":"Thanks for the report. Maybe the best fix would be to not make the metric produce NaNs and instead produce zeros in metrics. That is how we usually deal with zero variance cases."}}