{"action":"created","issue":{"url":"https://api.github.com/repos/BVLC/caffe/issues/2907","labels_url":"https://api.github.com/repos/BVLC/caffe/issues/2907/labels{/name}","comments_url":"https://api.github.com/repos/BVLC/caffe/issues/2907/comments","events_url":"https://api.github.com/repos/BVLC/caffe/issues/2907/events","html_url":"https://github.com/BVLC/caffe/issues/2907","id":100318502,"number":2907,"title":"CIFAR-10 tutorial train_quick issue - solver state filename","user":{"login":"dymczykm","id":6044558,"avatar_url":"https://avatars.githubusercontent.com/u/6044558?v=3","gravatar_id":"","url":"https://api.github.com/users/dymczykm","html_url":"https://github.com/dymczykm","followers_url":"https://api.github.com/users/dymczykm/followers","following_url":"https://api.github.com/users/dymczykm/following{/other_user}","gists_url":"https://api.github.com/users/dymczykm/gists{/gist_id}","starred_url":"https://api.github.com/users/dymczykm/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dymczykm/subscriptions","organizations_url":"https://api.github.com/users/dymczykm/orgs","repos_url":"https://api.github.com/users/dymczykm/repos","events_url":"https://api.github.com/users/dymczykm/events{/privacy}","received_events_url":"https://api.github.com/users/dymczykm/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-08-11T14:02:48Z","updated_at":"2015-09-02T09:22:30Z","closed_at":null,"body":"Hi,\r\n\r\nI'm new to Caffe so I might be doing something very wrong, but I think I precisely followed the tutorial available here:\r\nhttp://caffe.berkeleyvision.org/gathered/examples/cifar10.html\r\nI have a clean master caffe installation on OS X 10.10.4.\r\n\r\nThe problem is that when running ``train_quick.sh`` at some point the training crashes as it expects to see the solver state (after 4k iters) file with ``.h5`` extension (seems related to the recent commit: https://github.com/BVLC/caffe/commit/c9b333e06212365e3ea166bacbd4b560158316f0):\r\nhttps://github.com/BVLC/caffe/blob/master/examples/cifar10/train_quick.sh#L11\r\n\r\nBacktrace:\r\n```\r\nI0811 15:05:17.383101 2041144064 solver.cpp:241] Restoring previous solver status from examples/cifar10/cifar10_quick_iter_4000.solverstate.h5\r\nHDF5-DIAG: Error detected in HDF5 (1.8.14) thread 0:\r\n  #000: H5F.c line 604 in H5Fopen(): unable to open file\r\n    major: File accessibilty\r\n    minor: Unable to open file\r\n  #001: H5Fint.c line 990 in H5F_open(): unable to open file: time = Tue Aug 11 15:05:17 2015\r\n, name = 'examples/cifar10/cifar10_quick_iter_4000.solverstate.h5', tent_flags = 0\r\n    major: File accessibilty\r\n    minor: Unable to open file\r\n  #002: H5FD.c line 992 in H5FD_open(): open failed\r\n    major: Virtual File Layer\r\n    minor: Unable to initialize object\r\n  #003: H5FDsec2.c line 343 in H5FD_sec2_open(): unable to open file: name = 'examples/cifar10/cifar10_quick_iter_4000.solverstate.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0\r\n    major: File accessibilty\r\n    minor: Unable to open file\r\nF0811 15:05:17.399014 2041144064 solver.cpp:702] Check failed: file_hid >= 0 (-1 vs. 0) Couldn't open solver state file examples/cifar10/cifar10_quick_iter_4000.solverstate.h5\r\n*** Check failure stack trace: ***\r\n    @        0x1054e0664  google::LogMessage::SendToLog()\r\n    @        0x1054e0c97  google::LogMessage::Flush()\r\n    @        0x1054e46af  google::LogMessageFatal::~LogMessageFatal()\r\n    @        0x1054e1489  google::LogMessageFatal::~LogMessageFatal()\r\n    @        0x10523b624  caffe::SGDSolver<>::RestoreSolverStateFromHDF5()\r\n    @        0x1052333c1  caffe::Solver<>::Restore()\r\n    @        0x105233166  caffe::Solver<>::Solve()\r\n    @        0x10516ceba  train()\r\n    @        0x10516f22f  main\r\n    @     0x7fff8c0f25c9  start\r\n./examples/cifar10/train_quick.sh: line 11: 41703 Abort trap: 6           $TOOLS/caffe train --solver=examples/cifar10/cifar10_quick_solver_lr1.prototxt --snapshot=examples/cifar10/cifar10_quick_iter_4000.solverstate.h5\r\n```\r\n\r\nBut apparently, the training procedure snapshots the state to a file without ``.h5`` extension. Quickly hacking the ``train_quick.sh`` file to expect just ``examples/cifar10/cifar10_quick_iter_4000.solverstate`` resolved the issue.\r\n\r\nCorrect me if I'm wrong, but it looks like there's some incompatibility between the training procedure and snapshot loading, \r\n\r\nCheers,\r\nMarcin"},"comment":{"url":"https://api.github.com/repos/BVLC/caffe/issues/comments/136994204","html_url":"https://github.com/BVLC/caffe/issues/2907#issuecomment-136994204","issue_url":"https://api.github.com/repos/BVLC/caffe/issues/2907","id":136994204,"user":{"login":"SerhanGul","id":4326033,"avatar_url":"https://avatars.githubusercontent.com/u/4326033?v=3","gravatar_id":"","url":"https://api.github.com/users/SerhanGul","html_url":"https://github.com/SerhanGul","followers_url":"https://api.github.com/users/SerhanGul/followers","following_url":"https://api.github.com/users/SerhanGul/following{/other_user}","gists_url":"https://api.github.com/users/SerhanGul/gists{/gist_id}","starred_url":"https://api.github.com/users/SerhanGul/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SerhanGul/subscriptions","organizations_url":"https://api.github.com/users/SerhanGul/orgs","repos_url":"https://api.github.com/users/SerhanGul/repos","events_url":"https://api.github.com/users/SerhanGul/events{/privacy}","received_events_url":"https://api.github.com/users/SerhanGul/received_events","type":"User","site_admin":false},"created_at":"2015-09-02T09:22:30Z","updated_at":"2015-09-02T09:22:30Z","body":"I experienced the same problem and had to fix `train_quick.sh` . Can someone explain what the extension `.h5` means and why it was in the script originally?"}}