{"action":"closed","issue":{"url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4898","labels_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4898/labels{/name}","comments_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4898/comments","events_url":"https://api.github.com/repos/scikit-learn/scikit-learn/issues/4898/events","html_url":"https://github.com/scikit-learn/scikit-learn/issues/4898","id":90997709,"number":4898,"title":"Profiling RandomForestClassifier for bottlenecks - _set_oob_score","user":{"login":"dtsmith2001","id":3189861,"avatar_url":"https://avatars.githubusercontent.com/u/3189861?v=3","gravatar_id":"","url":"https://api.github.com/users/dtsmith2001","html_url":"https://github.com/dtsmith2001","followers_url":"https://api.github.com/users/dtsmith2001/followers","following_url":"https://api.github.com/users/dtsmith2001/following{/other_user}","gists_url":"https://api.github.com/users/dtsmith2001/gists{/gist_id}","starred_url":"https://api.github.com/users/dtsmith2001/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dtsmith2001/subscriptions","organizations_url":"https://api.github.com/users/dtsmith2001/orgs","repos_url":"https://api.github.com/users/dtsmith2001/repos","events_url":"https://api.github.com/users/dtsmith2001/events{/privacy}","received_events_url":"https://api.github.com/users/dtsmith2001/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":10,"created_at":"2015-06-25T15:34:50Z","updated_at":"2015-07-30T16:36:08Z","closed_at":"2015-07-30T16:36:08Z","body":"I observed a long delay between fitting trees after constructing a **RandomForestClassifier** with n_jobs=20. I fired up iPython and used\r\n\r\n```python\r\n%runit -l forest.py forest.fit(X, y)\r\n```\r\n\r\n(where the forest object is the constructed classifier - see below). I found that the function **_set_oob_score** was taking up most of the time. This function is in the class **ForestClassifier**.\r\n\r\nI am using scikit-learn version 0.16.1 with Anaconda 3 on Windows Server 2008 R2 Enterprise x64.\r\n\r\nHere is the code I’m using to create the forest object I use with %runit:\r\n\r\n```python\r\nimport sklearn as sk\r\nfrom sklearn.ensemble import RandomForestClassifier\r\nfrom sklearn.datasets import make_classification\r\n\r\nprint(\"scikit-learn version {0}\".format(sk.__version__))\r\n\r\nX, y = make_classification(n_samples=5000,\r\n                     n_features=30, n_informative=10, n_redundant=0, n_repeated=0,\r\n                     n_classes=2, random_state=0, weights=[0.9, 0.1],  # unbalanced\r\n                     shuffle=False)\r\n\r\nprint(\"Number of cases {0} number of features {1}\".format(X.shape[0], X.shape[1]))\r\nforest = RandomForestClassifier(n_estimators=50,\r\n                     random_state=100, oob_score=True,\r\n                     bootstrap=True, verbose=2)\r\n```\r\n\r\nAt this point we are ready to use %runit as I illustrated above.\r\n\r\nMy own use case corresponds to using n_samples=400000 and n_features=3500 (n_informative is not relevant in this example).\r\n\r\nThe issue here is that **_set_oob_score** uses vastly more time than actually creating the trees, particularly with my use case of 400,000 samples and 3,500 features.\r\n\r\nHere is a completely reproducible example of the code I'm running. I can see in Task Manager that the processors are in use generating the trees, but after the number of processors in use goes back to one, there is a long delay before fit finishes. Note that this code uses 20 processors, over 17 gb memory, and takes a lot of time on my own computer. YMMV.\r\n\r\n```python\r\nimport sklearn as sk\r\nfrom sklearn.ensemble import RandomForestClassifier\r\nfrom sklearn.datasets import make_classification\r\n\r\nprint(\"scikit-learn version {0}\".format(sk.__version__))\r\n\r\nX, y = make_classification(n_samples=400000, n_features=3500,\r\n\t\t\t\t\t\t   n_informative=10, n_redundant=0, n_repeated=0,\r\n\t\t\t\t\t\t   n_classes=2, random_state=0, weights=[0.9, 0.1],  # unbalanced\r\n\t\t\t\t\t\t   shuffle=False)\r\n\r\nprint(\"Number of cases {0} number of features {1}\".format(X.shape[0], X.shape[1]))\r\nforest = RandomForestClassifier(n_estimators=50, n_jobs=20,\r\n\t\t\t\t\t\t\t\trandom_state=100, oob_score=True, bootstrap=True, verbose=2)\r\nforest.fit(X, y)\r\nprint(\"Finished\")\r\n```\r\n\r\nPlease rewrite **_set_oob_score** to improve performance.\r\n"}}