{"action":"created","issue":{"url":"https://api.github.com/repos/yjxiong/caffe/issues/2","labels_url":"https://api.github.com/repos/yjxiong/caffe/issues/2/labels{/name}","comments_url":"https://api.github.com/repos/yjxiong/caffe/issues/2/comments","events_url":"https://api.github.com/repos/yjxiong/caffe/issues/2/events","html_url":"https://github.com/yjxiong/caffe/issues/2","id":101181491,"number":2,"title":"loss does not decrease","user":{"login":"whjxnyzh","id":6215715,"avatar_url":"https://avatars.githubusercontent.com/u/6215715?v=3","gravatar_id":"","url":"https://api.github.com/users/whjxnyzh","html_url":"https://github.com/whjxnyzh","followers_url":"https://api.github.com/users/whjxnyzh/followers","following_url":"https://api.github.com/users/whjxnyzh/following{/other_user}","gists_url":"https://api.github.com/users/whjxnyzh/gists{/gist_id}","starred_url":"https://api.github.com/users/whjxnyzh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/whjxnyzh/subscriptions","organizations_url":"https://api.github.com/users/whjxnyzh/orgs","repos_url":"https://api.github.com/users/whjxnyzh/repos","events_url":"https://api.github.com/users/whjxnyzh/events{/privacy}","received_events_url":"https://api.github.com/users/whjxnyzh/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-08-15T14:32:41Z","updated_at":"2015-08-16T13:35:36Z","closed_at":null,"body":"Hi, I trained the model using your script.\r\nThe loss looks like this.\r\n```\r\nI0815 18:56:21.700711 52129 solver.cpp:631] Iteration 920, lr = 0.001\r\nI0815 18:59:11.323932 52129 solver.cpp:232] Iteration 940, loss = 2.33851\r\nI0815 18:59:11.324184 52129 solver.cpp:631] Iteration 940, lr = 0.001\r\nI0815 19:02:00.977653 52129 solver.cpp:232] Iteration 960, loss = 2.43626\r\nI0815 19:02:00.977931 52129 solver.cpp:631] Iteration 960, lr = 0.001\r\nI0815 19:04:50.639415 52129 solver.cpp:232] Iteration 980, loss = 2.29515\r\nI0815 19:04:50.639688 52129 solver.cpp:631] Iteration 980, lr = 0.001\r\nI0815 19:07:32.759773 52129 solver.cpp:506] Snapshotting to cuhk_action_recognition_vgg_16_split1_rgb_iter_1000.caffemodel\r\nI0815 19:07:36.681205 52129 solver.cpp:514] Snapshotting solver state to cuhk_action_recognition_vgg_16_split1_rgb_iter_1000.solverstate\r\nI0815 19:07:40.538120 52129 solver.cpp:432] Iteration 1000, Testing net (#0)\r\nI0815 19:07:58.990095 52129 solver.cpp:488]     Test net output #0: accuracy = 0.685882\r\nI0815 19:07:58.990149 52129 solver.cpp:488]     Test net output #1: loss = 1.75962 (* 1 = 1.75962 loss)\r\nI0815 19:08:07.426625 52129 solver.cpp:232] Iteration 1000, loss = 4.31525\r\nI0815 19:08:07.426946 52129 solver.cpp:631] Iteration 1000, lr = 0.001\r\nI0815 19:10:57.163290 52129 solver.cpp:232] Iteration 1020, loss = 2.93576\r\nI0815 19:10:57.163581 52129 solver.cpp:631] Iteration 1020, lr = 0.001\r\nI0815 19:13:46.859581 52129 solver.cpp:232] Iteration 1040, loss = 2.40981\r\nI0815 19:13:46.859861 52129 solver.cpp:631] Iteration 1040, lr = 0.001\r\nI0815 19:16:36.546717 52129 solver.cpp:232] Iteration 1060, loss = 2.49907\r\nI0815 19:16:36.546993 52129 solver.cpp:631] Iteration 1060, lr = 0.001\r\nI0815 19:19:26.209902 52129 solver.cpp:232] Iteration 1080, loss = 2.27296\r\nI0815 19:19:26.210175 52129 solver.cpp:631] Iteration 1080, lr = 0.001\r\nI0815 19:22:15.858909 52129 solver.cpp:232] Iteration 1100, loss = 2.36718\r\nI0815 19:22:15.859191 52129 solver.cpp:631] Iteration 1100, lr = 0.001\r\nI0815 19:25:05.558161 52129 solver.cpp:232] Iteration 1120, loss = 2.32493\r\nI0815 19:25:05.558434 52129 solver.cpp:631] Iteration 1120, lr = 0.001\r\nI0815 19:27:55.255880 52129 solver.cpp:232] Iteration 1140, loss = 2.39129\r\nI0815 19:27:55.256165 52129 solver.cpp:631] Iteration 1140, lr = 0.001\r\nI0815 19:30:44.819021 52129 solver.cpp:232] Iteration 1160, loss = 2.36149\r\nI0815 19:30:44.819293 52129 solver.cpp:631] Iteration 1160, lr = 0.001\r\nI0815 19:33:34.412870 52129 solver.cpp:232] Iteration 1180, loss = 2.16151\r\nI0815 19:33:34.413147 52129 solver.cpp:631] Iteration 1180, lr = 0.001\r\nI0815 19:36:24.095429 52129 solver.cpp:232] Iteration 1200, loss = 2.36173\r\nI0815 19:36:24.095723 52129 solver.cpp:631] Iteration 1200, lr = 0.001\r\nI0815 19:39:13.749619 52129 solver.cpp:232] Iteration 1220, loss = 2.45683\r\nI0815 19:39:13.749883 52129 solver.cpp:631] Iteration 1220, lr = 0.001\r\nI0815 19:42:03.514300 52129 solver.cpp:232] Iteration 1240, loss = 2.41362\r\nI0815 19:42:03.514590 52129 solver.cpp:631] Iteration 1240, lr = 0.001\r\nI0815 19:44:53.258905 52129 solver.cpp:232] Iteration 1260, loss = 2.29926\r\nI0815 19:44:53.259179 52129 solver.cpp:631] Iteration 1260, lr = 0.001\r\nI0815 19:47:42.936760 52129 solver.cpp:232] Iteration 1280, loss = 2.29143\r\nI0815 19:47:42.937052 52129 solver.cpp:631] Iteration 1280, lr = 0.001\r\nI0815 19:50:32.691732 52129 solver.cpp:232] Iteration 1300, loss = 2.21673\r\nI0815 19:50:32.692003 52129 solver.cpp:631] Iteration 1300, lr = 0.001\r\nI0815 19:53:22.276563 52129 solver.cpp:232] Iteration 1320, loss = 2.30811\r\nI0815 19:53:22.276844 52129 solver.cpp:631] Iteration 1320, lr = 0.001\r\nI0815 19:56:11.834009 52129 solver.cpp:232] Iteration 1340, loss = 2.33596\r\nI0815 19:56:11.834276 52129 solver.cpp:631] Iteration 1340, lr = 0.001\r\nI0815 19:59:01.388342 52129 solver.cpp:232] Iteration 1360, loss = 2.1507\r\nI0815 19:59:01.388628 52129 solver.cpp:631] Iteration 1360, lr = 0.001\r\nI0815 20:01:50.989980 52129 solver.cpp:232] Iteration 1380, loss = 2.25501\r\nI0815 20:01:50.990263 52129 solver.cpp:631] Iteration 1380, lr = 0.001\r\nI0815 20:04:40.654201 52129 solver.cpp:232] Iteration 1400, loss = 2.25313\r\nI0815 20:04:40.654469 52129 solver.cpp:631] Iteration 1400, lr = 0.001\r\nI0815 20:07:30.328219 52129 solver.cpp:232] Iteration 1420, loss = 2.20313\r\nI0815 20:07:30.328493 52129 solver.cpp:631] Iteration 1420, lr = 0.001\r\nI0815 20:10:19.975951 52129 solver.cpp:232] Iteration 1440, loss = 2.20481\r\nI0815 20:10:19.976227 52129 solver.cpp:631] Iteration 1440, lr = 0.001\r\nI0815 20:13:09.721353 52129 solver.cpp:232] Iteration 1460, loss = 2.29201\r\nI0815 20:13:09.721652 52129 solver.cpp:631] Iteration 1460, lr = 0.001\r\nI0815 20:15:59.386811 52129 solver.cpp:232] Iteration 1480, loss = 2.25177\r\nI0815 20:15:59.387084 52129 solver.cpp:631] Iteration 1480, lr = 0.001\r\nI0815 20:18:49.073662 52129 solver.cpp:232] Iteration 1500, loss = 2.22122\r\nI0815 20:18:49.073942 52129 solver.cpp:631] Iteration 1500, lr = 0.001\r\nI0815 20:21:38.805486 52129 solver.cpp:232] Iteration 1520, loss = 2.19175\r\nI0815 20:21:38.805810 52129 solver.cpp:631] Iteration 1520, lr = 0.001\r\nI0815 20:24:28.513808 52129 solver.cpp:232] Iteration 1540, loss = 2.15029\r\nI0815 20:24:28.514101 52129 solver.cpp:631] Iteration 1540, lr = 0.001\r\nI0815 20:27:18.180346 52129 solver.cpp:232] Iteration 1560, loss = 2.21155\r\nI0815 20:27:18.180622 52129 solver.cpp:631] Iteration 1560, lr = 0.001\r\nI0815 20:30:07.847918 52129 solver.cpp:232] Iteration 1580, loss = 2.17659\r\nI0815 20:30:07.848196 52129 solver.cpp:631] Iteration 1580, lr = 0.001\r\nI0815 20:32:57.541263 52129 solver.cpp:232] Iteration 1600, loss = 2.09776\r\nI0815 20:32:57.541563 52129 solver.cpp:631] Iteration 1600, lr = 0.001\r\nI0815 20:35:47.284319 52129 solver.cpp:232] Iteration 1620, loss = 1.99478\r\nI0815 20:35:47.284600 52129 solver.cpp:631] Iteration 1620, lr = 0.001\r\nI0815 20:38:37.013903 52129 solver.cpp:232] Iteration 1640, loss = 2.15796\r\nI0815 20:38:37.014174 52129 solver.cpp:631] Iteration 1640, lr = 0.001\r\nI0815 20:41:26.810891 52129 solver.cpp:232] Iteration 1660, loss = 2.19062\r\nI0815 20:41:26.811189 52129 solver.cpp:631] Iteration 1660, lr = 0.001\r\nI0815 20:44:16.574542 52129 solver.cpp:232] Iteration 1680, loss = 2.07286\r\nI0815 20:44:16.574836 52129 solver.cpp:631] Iteration 1680, lr = 0.001\r\nI0815 20:47:06.222946 52129 solver.cpp:232] Iteration 1700, loss = 2.07381\r\nI0815 20:47:06.223240 52129 solver.cpp:631] Iteration 1700, lr = 0.001\r\nI0815 20:49:55.951117 52129 solver.cpp:232] Iteration 1720, loss = 2.21451\r\nI0815 20:49:55.951405 52129 solver.cpp:631] Iteration 1720, lr = 0.001\r\nI0815 20:52:45.639834 52129 solver.cpp:232] Iteration 1740, loss = 2.07811\r\nI0815 20:52:45.640107 52129 solver.cpp:631] Iteration 1740, lr = 0.001\r\nI0815 20:55:35.355908 52129 solver.cpp:232] Iteration 1760, loss = 2.14357\r\nI0815 20:55:35.356190 52129 solver.cpp:631] Iteration 1760, lr = 0.001\r\nI0815 20:58:25.062911 52129 solver.cpp:232] Iteration 1780, loss = 2.19717\r\nI0815 20:58:25.063179 52129 solver.cpp:631] Iteration 1780, lr = 0.001\r\nI0815 21:01:14.744174 52129 solver.cpp:232] Iteration 1800, loss = 2.07986\r\nI0815 21:01:14.744451 52129 solver.cpp:631] Iteration 1800, lr = 0.001\r\nI0815 21:04:04.339823 52129 solver.cpp:232] Iteration 1820, loss = 2.11781\r\nI0815 21:04:04.340122 52129 solver.cpp:631] Iteration 1820, lr = 0.001\r\nI0815 21:06:53.922601 52129 solver.cpp:232] Iteration 1840, loss = 2.10838\r\nI0815 21:06:53.922893 52129 solver.cpp:631] Iteration 1840, lr = 0.001\r\nI0815 21:09:43.493850 52129 solver.cpp:232] Iteration 1860, loss = 2.08746\r\nI0815 21:09:43.494134 52129 solver.cpp:631] Iteration 1860, lr = 0.001\r\nI0815 21:12:33.116439 52129 solver.cpp:232] Iteration 1880, loss = 1.96505\r\nI0815 21:12:33.116732 52129 solver.cpp:631] Iteration 1880, lr = 0.001\r\nI0815 21:15:22.650295 52129 solver.cpp:232] Iteration 1900, loss = 2.25801\r\nI0815 21:15:22.650579 52129 solver.cpp:631] Iteration 1900, lr = 0.001\r\nI0815 21:18:12.180665 52129 solver.cpp:232] Iteration 1920, loss = 2.0309\r\nI0815 21:18:12.180948 52129 solver.cpp:631] Iteration 1920, lr = 0.001\r\nI0815 21:21:01.732786 52129 solver.cpp:232] Iteration 1940, loss = 1.99836\r\nI0815 21:21:01.733085 52129 solver.cpp:631] Iteration 1940, lr = 0.001\r\nI0815 21:23:51.258694 52129 solver.cpp:232] Iteration 1960, loss = 2.14637\r\nI0815 21:23:51.258970 52129 solver.cpp:631] Iteration 1960, lr = 0.001\r\nI0815 21:26:40.873399 52129 solver.cpp:232] Iteration 1980, loss = 2.11196\r\nI0815 21:26:40.873687 52129 solver.cpp:631] Iteration 1980, lr = 0.001\r\nI0815 21:29:22.737175 52129 solver.cpp:506] Snapshotting to cuhk_action_recognition_vgg_16_split1_rgb_iter_2000.caffemodel\r\nI0815 21:29:24.311609 52129 solver.cpp:514] Snapshotting solver state to cuhk_action_recognition_vgg_16_split1_rgb_iter_2000.solverstate\r\nI0815 21:29:27.109315 52129 solver.cpp:432] Iteration 2000, Testing net (#0)\r\nI0815 21:29:45.471135 52129 solver.cpp:488]     Test net output #0: accuracy = 0.69\r\nI0815 21:29:45.471182 52129 solver.cpp:488]     Test net output #1: loss = 2.12226 (* 1 = 2.12226 loss)\r\nI0815 21:29:53.924059 52129 solver.cpp:232] Iteration 2000, loss = 4.92101\r\nI0815 21:29:53.924412 52129 solver.cpp:631] Iteration 2000, lr = 0.001\r\nI0815 21:32:43.561449 52129 solver.cpp:232] Iteration 2020, loss = 3.19778\r\nI0815 21:32:43.561743 52129 solver.cpp:631] Iteration 2020, lr = 0.001\r\nI0815 21:35:33.365213 52129 solver.cpp:232] Iteration 2040, loss = 2.46677\r\nI0815 21:35:33.365488 52129 solver.cpp:631] Iteration 2040, lr = 0.001\r\nI0815 21:38:23.062963 52129 solver.cpp:232] Iteration 2060, loss = 2.20488\r\nI0815 21:38:23.063457 52129 solver.cpp:631] Iteration 2060, lr = 0.001\r\nI0815 21:41:12.869400 52129 solver.cpp:232] Iteration 2080, loss = 2.15901\r\nI0815 21:41:12.869673 52129 solver.cpp:631] Iteration 2080, lr = 0.001\r\nI0815 21:44:02.653378 52129 solver.cpp:232] Iteration 2100, loss = 2.18091\r\nI0815 21:44:02.653652 52129 solver.cpp:631] Iteration 2100, lr = 0.001\r\nI0815 21:46:52.403762 52129 solver.cpp:232] Iteration 2120, loss = 2.2991\r\nI0815 21:46:52.404038 52129 solver.cpp:631] Iteration 2120, lr = 0.001\r\nI0815 21:49:42.154250 52129 solver.cpp:232] Iteration 2140, loss = 2.05538\r\nI0815 21:49:42.154541 52129 solver.cpp:631] Iteration 2140, lr = 0.001\r\nI0815 21:52:31.944736 52129 solver.cpp:232] Iteration 2160, loss = 2.1253\r\nI0815 21:52:31.945013 52129 solver.cpp:631] Iteration 2160, lr = 0.001\r\nI0815 21:55:21.718902 52129 solver.cpp:232] Iteration 2180, loss = 2.0968\r\nI0815 21:55:21.719180 52129 solver.cpp:631] Iteration 2180, lr = 0.001\r\nI0815 21:58:11.463587 52129 solver.cpp:232] Iteration 2200, loss = 2.07307\r\nI0815 21:58:11.463840 52129 solver.cpp:631] Iteration 2200, lr = 0.001\r\nI0815 22:01:01.200736 52129 solver.cpp:232] Iteration 2220, loss = 2.05853\r\nI0815 22:01:01.201009 52129 solver.cpp:631] Iteration 2220, lr = 0.001\r\nI0815 22:03:50.985465 52129 solver.cpp:232] Iteration 2240, loss = 2.06146\r\nI0815 22:03:50.985741 52129 solver.cpp:631] Iteration 2240, lr = 0.001\r\nI0815 22:06:40.745090 52129 solver.cpp:232] Iteration 2260, loss = 2.02932\r\nI0815 22:06:40.745376 52129 solver.cpp:631] Iteration 2260, lr = 0.001\r\nI0815 22:09:30.424286 52129 solver.cpp:232] Iteration 2280, loss = 2.10739\r\nI0815 22:09:30.424578 52129 solver.cpp:631] Iteration 2280, lr = 0.001\r\nI0815 22:12:20.215556 52129 solver.cpp:232] Iteration 2300, loss = 1.98266\r\nI0815 22:12:20.215840 52129 solver.cpp:631] Iteration 2300, lr = 0.001\r\nI0815 22:15:10.013236 52129 solver.cpp:232] Iteration 2320, loss = 2.05712\r\nI0815 22:15:10.013514 52129 solver.cpp:631] Iteration 2320, lr = 0.001\r\nI0815 22:17:59.798974 52129 solver.cpp:232] Iteration 2340, loss = 2.04119\r\nI0815 22:17:59.799223 52129 solver.cpp:631] Iteration 2340, lr = 0.001\r\nI0815 22:20:49.522065 52129 solver.cpp:232] Iteration 2360, loss = 2.10465\r\nI0815 22:20:49.522343 52129 solver.cpp:631] Iteration 2360, lr = 0.001\r\nI0815 22:23:39.197793 52129 solver.cpp:232] Iteration 2380, loss = 2.04238\r\nI0815 22:23:39.198057 52129 solver.cpp:631] Iteration 2380, lr = 0.001\r\nI0815 22:26:28.971088 52129 solver.cpp:232] Iteration 2400, loss = 2.00524\r\nI0815 22:26:28.971369 52129 solver.cpp:631] Iteration 2400, lr = 0.001\r\n\r\n```\r\nIt seems that the two 1000 iterations are the same.\r\n"},"comment":{"url":"https://api.github.com/repos/yjxiong/caffe/issues/comments/131552651","html_url":"https://github.com/yjxiong/caffe/issues/2#issuecomment-131552651","issue_url":"https://api.github.com/repos/yjxiong/caffe/issues/2","id":131552651,"user":{"login":"whjxnyzh","id":6215715,"avatar_url":"https://avatars.githubusercontent.com/u/6215715?v=3","gravatar_id":"","url":"https://api.github.com/users/whjxnyzh","html_url":"https://github.com/whjxnyzh","followers_url":"https://api.github.com/users/whjxnyzh/followers","following_url":"https://api.github.com/users/whjxnyzh/following{/other_user}","gists_url":"https://api.github.com/users/whjxnyzh/gists{/gist_id}","starred_url":"https://api.github.com/users/whjxnyzh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/whjxnyzh/subscriptions","organizations_url":"https://api.github.com/users/whjxnyzh/orgs","repos_url":"https://api.github.com/users/whjxnyzh/repos","events_url":"https://api.github.com/users/whjxnyzh/events{/privacy}","received_events_url":"https://api.github.com/users/whjxnyzh/received_events","type":"User","site_admin":false},"created_at":"2015-08-16T13:35:36Z","updated_at":"2015-08-16T13:35:36Z","body":"Hi, @yjxiong \r\nAt last \r\nthe loss looks like this\r\n```\r\nI0816 11:40:31.063642 52129 solver.cpp:432] Iteration 8000, Testing net (#0)\r\nI0816 11:40:49.557504 52129 solver.cpp:488]     Test net output #0: accuracy = 0.746471\r\nI0816 11:40:49.557560 52129 solver.cpp:488]     Test net output #1: loss = 2.46194 (* 1 = 2.46194 loss)\r\nI0816 11:40:58.005064 52129 solver.cpp:232] Iteration 8000, loss = 7.19096\r\nI0816 11:40:58.005342 52129 solver.cpp:631] Iteration 8000, lr = 1e-05\r\nI0816 11:43:47.659518 52129 solver.cpp:232] Iteration 8020, loss = 7.6423\r\nI0816 11:43:47.659813 52129 solver.cpp:631] Iteration 8020, lr = 1e-05\r\nI0816 11:46:37.380287 52129 solver.cpp:232] Iteration 8040, loss = 7.24899\r\nI0816 11:46:37.380571 52129 solver.cpp:631] Iteration 8040, lr = 1e-05\r\nI0816 11:49:27.106161 52129 solver.cpp:232] Iteration 8060, loss = 6.82677\r\nI0816 11:49:27.106447 52129 solver.cpp:631] Iteration 8060, lr = 1e-05\r\nI0816 11:52:16.780643 52129 solver.cpp:232] Iteration 8080, loss = 6.52308\r\nI0816 11:52:16.780917 52129 solver.cpp:631] Iteration 8080, lr = 1e-05\r\nI0816 11:55:06.489707 52129 solver.cpp:232] Iteration 8100, loss = 7.16052\r\nI0816 11:55:06.489995 52129 solver.cpp:631] Iteration 8100, lr = 1e-05\r\nI0816 11:57:56.226898 52129 solver.cpp:232] Iteration 8120, loss = 7.34915\r\nI0816 11:57:56.227174 52129 solver.cpp:631] Iteration 8120, lr = 1e-05\r\nI0816 12:00:46.069665 52129 solver.cpp:232] Iteration 8140, loss = 6.77685\r\nI0816 12:00:46.069948 52129 solver.cpp:631] Iteration 8140, lr = 1e-05\r\nI0816 12:03:35.744577 52129 solver.cpp:232] Iteration 8160, loss = 6.14444\r\nI0816 12:03:35.744856 52129 solver.cpp:631] Iteration 8160, lr = 1e-05\r\nI0816 12:06:25.556581 52129 solver.cpp:232] Iteration 8180, loss = 6.55798\r\nI0816 12:06:25.556856 52129 solver.cpp:631] Iteration 8180, lr = 1e-05\r\nI0816 12:09:15.170249 52129 solver.cpp:232] Iteration 8200, loss = 6.77813\r\nI0816 12:09:15.170527 52129 solver.cpp:631] Iteration 8200, lr = 1e-05\r\nI0816 12:12:04.958855 52129 solver.cpp:232] Iteration 8220, loss = 6.33232\r\nI0816 12:12:04.959137 52129 solver.cpp:631] Iteration 8220, lr = 1e-05\r\nI0816 12:14:54.668884 52129 solver.cpp:232] Iteration 8240, loss = 6.05957\r\nI0816 12:14:54.669155 52129 solver.cpp:631] Iteration 8240, lr = 1e-05\r\nI0816 12:17:44.386068 52129 solver.cpp:232] Iteration 8260, loss = 6.26374\r\nI0816 12:17:44.386370 52129 solver.cpp:631] Iteration 8260, lr = 1e-05\r\nI0816 12:20:33.956786 52129 solver.cpp:232] Iteration 8280, loss = 5.9378\r\nI0816 12:20:33.957121 52129 solver.cpp:631] Iteration 8280, lr = 1e-05\r\nI0816 12:23:23.640388 52129 solver.cpp:232] Iteration 8300, loss = 6.53556\r\nI0816 12:23:23.640674 52129 solver.cpp:631] Iteration 8300, lr = 1e-05\r\nI0816 12:26:13.239284 52129 solver.cpp:232] Iteration 8320, loss = 6.17501\r\nI0816 12:26:13.239559 52129 solver.cpp:631] Iteration 8320, lr = 1e-05\r\nI0816 12:29:02.992275 52129 solver.cpp:232] Iteration 8340, loss = 5.90202\r\nI0816 12:29:02.992542 52129 solver.cpp:631] Iteration 8340, lr = 1e-05\r\nI0816 12:31:52.680603 52129 solver.cpp:232] Iteration 8360, loss = 5.9879\r\nI0816 12:31:52.680886 52129 solver.cpp:631] Iteration 8360, lr = 1e-05\r\nI0816 12:34:42.196454 52129 solver.cpp:232] Iteration 8380, loss = 5.86749\r\nI0816 12:34:42.196724 52129 solver.cpp:631] Iteration 8380, lr = 1e-05\r\nI0816 12:37:31.827867 52129 solver.cpp:232] Iteration 8400, loss = 5.70989\r\nI0816 12:37:31.828158 52129 solver.cpp:631] Iteration 8400, lr = 1e-05\r\nI0816 12:40:21.419359 52129 solver.cpp:232] Iteration 8420, loss = 5.31888\r\nI0816 12:40:21.419633 52129 solver.cpp:631] Iteration 8420, lr = 1e-05\r\nI0816 12:43:11.049187 52129 solver.cpp:232] Iteration 8440, loss = 5.24846\r\nI0816 12:43:11.049460 52129 solver.cpp:631] Iteration 8440, lr = 1e-05\r\nI0816 12:46:00.751379 52129 solver.cpp:232] Iteration 8460, loss = 5.5015\r\nI0816 12:46:00.751674 52129 solver.cpp:631] Iteration 8460, lr = 1e-05\r\nI0816 12:48:50.313210 52129 solver.cpp:232] Iteration 8480, loss = 5.56008\r\nI0816 12:48:50.313493 52129 solver.cpp:631] Iteration 8480, lr = 1e-05\r\nI0816 12:51:39.991730 52129 solver.cpp:232] Iteration 8500, loss = 5.38535\r\nI0816 12:51:39.992009 52129 solver.cpp:631] Iteration 8500, lr = 1e-05\r\nI0816 12:54:29.609454 52129 solver.cpp:232] Iteration 8520, loss = 5.69843\r\nI0816 12:54:29.609730 52129 solver.cpp:631] Iteration 8520, lr = 1e-05\r\nI0816 12:57:19.228298 52129 solver.cpp:232] Iteration 8540, loss = 5.79265\r\nI0816 12:57:19.228585 52129 solver.cpp:631] Iteration 8540, lr = 1e-05\r\nI0816 13:00:08.825079 52129 solver.cpp:232] Iteration 8560, loss = 5.10108\r\nI0816 13:00:08.825356 52129 solver.cpp:631] Iteration 8560, lr = 1e-05\r\nI0816 13:02:58.393731 52129 solver.cpp:232] Iteration 8580, loss = 5.22401\r\nI0816 13:02:58.394018 52129 solver.cpp:631] Iteration 8580, lr = 1e-05\r\nI0816 13:05:47.962023 52129 solver.cpp:232] Iteration 8600, loss = 5.31076\r\nI0816 13:05:47.962316 52129 solver.cpp:631] Iteration 8600, lr = 1e-05\r\nI0816 13:08:37.575601 52129 solver.cpp:232] Iteration 8620, loss = 5.26973\r\nI0816 13:08:37.575892 52129 solver.cpp:631] Iteration 8620, lr = 1e-05\r\nI0816 13:11:27.092717 52129 solver.cpp:232] Iteration 8640, loss = 4.71255\r\nI0816 13:11:27.093001 52129 solver.cpp:631] Iteration 8640, lr = 1e-05\r\nI0816 13:14:16.729068 52129 solver.cpp:232] Iteration 8660, loss = 4.90248\r\nI0816 13:14:16.729358 52129 solver.cpp:631] Iteration 8660, lr = 1e-05\r\nI0816 13:17:06.256260 52129 solver.cpp:232] Iteration 8680, loss = 4.94517\r\nI0816 13:17:06.256559 52129 solver.cpp:631] Iteration 8680, lr = 1e-05\r\nI0816 13:19:55.810688 52129 solver.cpp:232] Iteration 8700, loss = 5.48554\r\nI0816 13:19:55.810973 52129 solver.cpp:631] Iteration 8700, lr = 1e-05\r\nI0816 13:22:45.465327 52129 solver.cpp:232] Iteration 8720, loss = 5.50927\r\nI0816 13:22:45.465610 52129 solver.cpp:631] Iteration 8720, lr = 1e-05\r\nI0816 13:25:34.891710 52129 solver.cpp:232] Iteration 8740, loss = 5.12494\r\nI0816 13:25:34.891984 52129 solver.cpp:631] Iteration 8740, lr = 1e-05\r\nI0816 13:28:24.414628 52129 solver.cpp:232] Iteration 8760, loss = 4.7183\r\nI0816 13:28:24.414917 52129 solver.cpp:631] Iteration 8760, lr = 1e-05\r\nI0816 13:31:13.983726 52129 solver.cpp:232] Iteration 8780, loss = 4.79555\r\nI0816 13:31:13.983996 52129 solver.cpp:631] Iteration 8780, lr = 1e-05\r\nI0816 13:34:03.476532 52129 solver.cpp:232] Iteration 8800, loss = 5.03278\r\nI0816 13:34:03.476806 52129 solver.cpp:631] Iteration 8800, lr = 1e-05\r\nI0816 13:36:53.036485 52129 solver.cpp:232] Iteration 8820, loss = 4.32562\r\nI0816 13:36:53.036774 52129 solver.cpp:631] Iteration 8820, lr = 1e-05\r\nI0816 13:39:42.588925 52129 solver.cpp:232] Iteration 8840, loss = 4.79604\r\nI0816 13:39:42.589242 52129 solver.cpp:631] Iteration 8840, lr = 1e-05\r\nI0816 13:42:32.083003 52129 solver.cpp:232] Iteration 8860, loss = 4.38464\r\nI0816 13:42:32.083291 52129 solver.cpp:631] Iteration 8860, lr = 1e-05\r\nI0816 13:45:21.678282 52129 solver.cpp:232] Iteration 8880, loss = 4.31263\r\nI0816 13:45:21.678563 52129 solver.cpp:631] Iteration 8880, lr = 1e-05\r\nI0816 13:48:11.243085 52129 solver.cpp:232] Iteration 8900, loss = 4.37252\r\nI0816 13:48:11.243376 52129 solver.cpp:631] Iteration 8900, lr = 1e-05\r\nI0816 13:51:00.879232 52129 solver.cpp:232] Iteration 8920, loss = 4.49006\r\nI0816 13:51:00.879511 52129 solver.cpp:631] Iteration 8920, lr = 1e-05\r\nI0816 13:53:50.444349 52129 solver.cpp:232] Iteration 8940, loss = 4.73392\r\nI0816 13:53:50.444629 52129 solver.cpp:631] Iteration 8940, lr = 1e-05\r\nI0816 13:56:40.107027 52129 solver.cpp:232] Iteration 8960, loss = 4.4215\r\nI0816 13:56:40.107323 52129 solver.cpp:631] Iteration 8960, lr = 1e-05\r\nI0816 13:59:29.661310 52129 solver.cpp:232] Iteration 8980, loss = 4.15559\r\nI0816 13:59:29.661584 52129 solver.cpp:631] Iteration 8980, lr = 1e-05\r\nI0816 14:02:11.586153 52129 solver.cpp:506] Snapshotting to cuhk_action_recognition_vgg_16_split1_rgb_iter_9000.caffemodel\r\nI0816 14:02:15.386831 52129 solver.cpp:514] Snapshotting solver state to cuhk_action_recognition_vgg_16_split1_rgb_iter_9000.solverstate\r\nI0816 14:02:18.430192 52129 solver.cpp:432] Iteration 9000, Testing net (#0)\r\nI0816 14:02:36.847875 52129 solver.cpp:488]     Test net output #0: accuracy = 0.694117\r\nI0816 14:02:36.847923 52129 solver.cpp:488]     Test net output #1: loss = 2.76251 (* 1 = 2.76251 loss)\r\nI0816 14:02:45.288152 52129 solver.cpp:232] Iteration 9000, loss = 7.11311\r\nI0816 14:02:45.288441 52129 solver.cpp:631] Iteration 9000, lr = 1e-05\r\nI0816 14:05:35.010887 52129 solver.cpp:232] Iteration 9020, loss = 7.06957\r\nI0816 14:05:35.011155 52129 solver.cpp:631] Iteration 9020, lr = 1e-05\r\nI0816 14:08:24.704478 52129 solver.cpp:232] Iteration 9040, loss = 6.96292\r\nI0816 14:08:24.704766 52129 solver.cpp:631] Iteration 9040, lr = 1e-05\r\nI0816 14:11:14.341878 52129 solver.cpp:232] Iteration 9060, loss = 7.25751\r\nI0816 14:11:14.342172 52129 solver.cpp:631] Iteration 9060, lr = 1e-05\r\nI0816 14:14:04.162201 52129 solver.cpp:232] Iteration 9080, loss = 7.02462\r\nI0816 14:14:04.162482 52129 solver.cpp:631] Iteration 9080, lr = 1e-05\r\nI0816 14:16:54.008741 52129 solver.cpp:232] Iteration 9100, loss = 6.97052\r\nI0816 14:16:54.009028 52129 solver.cpp:631] Iteration 9100, lr = 1e-05\r\nI0816 14:19:43.903246 52129 solver.cpp:232] Iteration 9120, loss = 6.98235\r\nI0816 14:19:43.903530 52129 solver.cpp:631] Iteration 9120, lr = 1e-05\r\nI0816 14:22:33.685515 52129 solver.cpp:232] Iteration 9140, loss = 6.72721\r\nI0816 14:22:33.685824 52129 solver.cpp:631] Iteration 9140, lr = 1e-05\r\nI0816 14:25:23.406747 52129 solver.cpp:232] Iteration 9160, loss = 5.9254\r\nI0816 14:25:23.407026 52129 solver.cpp:631] Iteration 9160, lr = 1e-05\r\nI0816 14:28:13.144767 52129 solver.cpp:232] Iteration 9180, loss = 7.08832\r\nI0816 14:28:13.145053 52129 solver.cpp:631] Iteration 9180, lr = 1e-05\r\nI0816 14:31:02.803170 52129 solver.cpp:232] Iteration 9200, loss = 6.97289\r\nI0816 14:31:02.803449 52129 solver.cpp:631] Iteration 9200, lr = 1e-05\r\nI0816 14:33:52.448231 52129 solver.cpp:232] Iteration 9220, loss = 6.3758\r\nI0816 14:33:52.448549 52129 solver.cpp:631] Iteration 9220, lr = 1e-05\r\nI0816 14:36:42.164077 52129 solver.cpp:232] Iteration 9240, loss = 6.82073\r\nI0816 14:36:42.164350 52129 solver.cpp:631] Iteration 9240, lr = 1e-05\r\nI0816 14:39:31.818503 52129 solver.cpp:232] Iteration 9260, loss = 5.89686\r\nI0816 14:39:31.818804 52129 solver.cpp:631] Iteration 9260, lr = 1e-05\r\nI0816 14:42:21.394268 52129 solver.cpp:232] Iteration 9280, loss = 6.23381\r\nI0816 14:42:21.394589 52129 solver.cpp:631] Iteration 9280, lr = 1e-05\r\nI0816 14:45:11.114181 52129 solver.cpp:232] Iteration 9300, loss = 6.42245\r\nI0816 14:45:11.114462 52129 solver.cpp:631] Iteration 9300, lr = 1e-05\r\nI0816 14:48:00.865546 52129 solver.cpp:232] Iteration 9320, loss = 5.56103\r\nI0816 14:48:00.865872 52129 solver.cpp:631] Iteration 9320, lr = 1e-05\r\nI0816 14:50:50.602396 52129 solver.cpp:232] Iteration 9340, loss = 6.37945\r\nI0816 14:50:50.602692 52129 solver.cpp:631] Iteration 9340, lr = 1e-05\r\nI0816 14:53:40.341265 52129 solver.cpp:232] Iteration 9360, loss = 6.39877\r\nI0816 14:53:40.341572 52129 solver.cpp:631] Iteration 9360, lr = 1e-05\r\nI0816 14:56:30.004266 52129 solver.cpp:232] Iteration 9380, loss = 5.59915\r\nI0816 14:56:30.004551 52129 solver.cpp:631] Iteration 9380, lr = 1e-05\r\nI0816 14:59:19.630707 52129 solver.cpp:232] Iteration 9400, loss = 6.1192\r\nI0816 14:59:19.630988 52129 solver.cpp:631] Iteration 9400, lr = 1e-05\r\nI0816 15:02:09.238507 52129 solver.cpp:232] Iteration 9420, loss = 5.9329\r\nI0816 15:02:09.238783 52129 solver.cpp:631] Iteration 9420, lr = 1e-05\r\nI0816 15:04:58.855051 52129 solver.cpp:232] Iteration 9440, loss = 5.35295\r\nI0816 15:04:58.855331 52129 solver.cpp:631] Iteration 9440, lr = 1e-05\r\nI0816 15:07:48.589382 52129 solver.cpp:232] Iteration 9460, loss = 6.05228\r\nI0816 15:07:48.589670 52129 solver.cpp:631] Iteration 9460, lr = 1e-05\r\nI0816 15:10:38.318300 52129 solver.cpp:232] Iteration 9480, loss = 5.66635\r\nI0816 15:10:38.318593 52129 solver.cpp:631] Iteration 9480, lr = 1e-05\r\nI0816 15:13:28.015619 52129 solver.cpp:232] Iteration 9500, loss = 5.63941\r\nI0816 15:13:28.015908 52129 solver.cpp:631] Iteration 9500, lr = 1e-05\r\nI0816 15:16:17.842252 52129 solver.cpp:232] Iteration 9520, loss = 5.70803\r\nI0816 15:16:17.842540 52129 solver.cpp:631] Iteration 9520, lr = 1e-05\r\nI0816 15:19:07.583556 52129 solver.cpp:232] Iteration 9540, loss = 5.29195\r\nI0816 15:19:07.583839 52129 solver.cpp:631] Iteration 9540, lr = 1e-05\r\nI0816 15:21:57.306762 52129 solver.cpp:232] Iteration 9560, loss = 5.43421\r\nI0816 15:21:57.307034 52129 solver.cpp:631] Iteration 9560, lr = 1e-05\r\nI0816 15:24:47.071312 52129 solver.cpp:232] Iteration 9580, loss = 5.31714\r\nI0816 15:24:47.071600 52129 solver.cpp:631] Iteration 9580, lr = 1e-05\r\nI0816 15:27:36.835381 52129 solver.cpp:232] Iteration 9600, loss = 5.30388\r\nI0816 15:27:36.835656 52129 solver.cpp:631] Iteration 9600, lr = 1e-05\r\nI0816 15:30:26.539476 52129 solver.cpp:232] Iteration 9620, loss = 5.29028\r\nI0816 15:30:26.539774 52129 solver.cpp:631] Iteration 9620, lr = 1e-05\r\nI0816 15:33:16.262919 52129 solver.cpp:232] Iteration 9640, loss = 4.89042\r\nI0816 15:33:16.263203 52129 solver.cpp:631] Iteration 9640, lr = 1e-05\r\nI0816 15:36:06.037367 52129 solver.cpp:232] Iteration 9660, loss = 5.39254\r\nI0816 15:36:06.037636 52129 solver.cpp:631] Iteration 9660, lr = 1e-05\r\nI0816 15:38:55.871461 52129 solver.cpp:232] Iteration 9680, loss = 5.15788\r\nI0816 15:38:55.871724 52129 solver.cpp:631] Iteration 9680, lr = 1e-05\r\nI0816 15:41:45.693110 52129 solver.cpp:232] Iteration 9700, loss = 4.97606\r\nI0816 15:41:45.693403 52129 solver.cpp:631] Iteration 9700, lr = 1e-05\r\nI0816 15:44:35.474977 52129 solver.cpp:232] Iteration 9720, loss = 4.6912\r\nI0816 15:44:35.475281 52129 solver.cpp:631] Iteration 9720, lr = 1e-05\r\nI0816 15:47:25.190691 52129 solver.cpp:232] Iteration 9740, loss = 4.94748\r\nI0816 15:47:25.190971 52129 solver.cpp:631] Iteration 9740, lr = 1e-05\r\nI0816 15:50:15.008975 52129 solver.cpp:232] Iteration 9760, loss = 5.09008\r\nI0816 15:50:15.009266 52129 solver.cpp:631] Iteration 9760, lr = 1e-05\r\nI0816 15:53:04.767146 52129 solver.cpp:232] Iteration 9780, loss = 4.639\r\nI0816 15:53:04.767431 52129 solver.cpp:631] Iteration 9780, lr = 1e-05\r\nI0816 15:55:54.423077 52129 solver.cpp:232] Iteration 9800, loss = 4.80646\r\nI0816 15:55:54.423355 52129 solver.cpp:631] Iteration 9800, lr = 1e-05\r\nI0816 15:58:44.061983 52129 solver.cpp:232] Iteration 9820, loss = 4.62463\r\nI0816 15:58:44.062345 52129 solver.cpp:631] Iteration 9820, lr = 1e-05\r\nI0816 16:01:33.765540 52129 solver.cpp:232] Iteration 9840, loss = 4.97929\r\nI0816 16:01:33.765830 52129 solver.cpp:631] Iteration 9840, lr = 1e-05\r\nI0816 16:04:23.415734 52129 solver.cpp:232] Iteration 9860, loss = 4.22811\r\nI0816 16:04:23.416013 52129 solver.cpp:631] Iteration 9860, lr = 1e-05\r\nI0816 16:07:13.065361 52129 solver.cpp:232] Iteration 9880, loss = 4.43138\r\nI0816 16:07:13.065664 52129 solver.cpp:631] Iteration 9880, lr = 1e-05\r\nI0816 16:10:02.831210 52129 solver.cpp:232] Iteration 9900, loss = 4.44416\r\nI0816 16:10:02.831507 52129 solver.cpp:631] Iteration 9900, lr = 1e-05\r\nI0816 16:12:52.533488 52129 solver.cpp:232] Iteration 9920, loss = 4.50714\r\nI0816 16:12:52.533778 52129 solver.cpp:631] Iteration 9920, lr = 1e-05\r\nI0816 16:15:42.258718 52129 solver.cpp:232] Iteration 9940, loss = 4.39002\r\nI0816 16:15:42.258997 52129 solver.cpp:631] Iteration 9940, lr = 1e-05\r\nI0816 16:18:31.892333 52129 solver.cpp:232] Iteration 9960, loss = 4.06315\r\nI0816 16:18:31.892607 52129 solver.cpp:631] Iteration 9960, lr = 1e-05\r\nI0816 16:21:21.583122 52129 solver.cpp:232] Iteration 9980, loss = 4.32309\r\nI0816 16:21:21.583412 52129 solver.cpp:631] Iteration 9980, lr = 1e-05\r\nI0816 16:24:03.584899 52129 solver.cpp:506] Snapshotting to cuhk_action_recognition_vgg_16_split1_rgb_iter_10000.caffemodel\r\nI0816 16:24:07.015681 52129 solver.cpp:514] Snapshotting solver state to cuhk_action_recognition_vgg_16_split1_rgb_iter_10000.solverstate\r\nI0816 16:24:11.039654 52129 solver.cpp:414] Iteration 10000, loss = 0.240114\r\nI0816 16:24:11.039693 52129 solver.cpp:432] Iteration 10000, Testing net (#0)\r\nI0816 16:24:29.430523 52129 solver.cpp:488]     Test net output #0: accuracy = 0.717059\r\nI0816 16:24:29.430578 52129 solver.cpp:488]     Test net output #1: loss = 1.99005 (* 1 = 1.99005 loss)\r\nI0816 16:24:29.430594 52129 solver.cpp:419] Optimization Done.\r\nI0816 16:24:29.430603 52129 caffe.cpp:187] Optimization Done.\r\n\r\n```\r\nwhy the loss became large at last?"}}