{"action":"created","issue":{"url":"https://api.github.com/repos/BVLC/caffe/issues/2334","labels_url":"https://api.github.com/repos/BVLC/caffe/issues/2334/labels{/name}","comments_url":"https://api.github.com/repos/BVLC/caffe/issues/2334/comments","events_url":"https://api.github.com/repos/BVLC/caffe/issues/2334/events","html_url":"https://github.com/BVLC/caffe/issues/2334","id":69255045,"number":2334,"title":"MemoryDataLayer Problem","user":{"login":"TJKlein","id":7634373,"avatar_url":"https://avatars.githubusercontent.com/u/7634373?v=3","gravatar_id":"","url":"https://api.github.com/users/TJKlein","html_url":"https://github.com/TJKlein","followers_url":"https://api.github.com/users/TJKlein/followers","following_url":"https://api.github.com/users/TJKlein/following{/other_user}","gists_url":"https://api.github.com/users/TJKlein/gists{/gist_id}","starred_url":"https://api.github.com/users/TJKlein/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TJKlein/subscriptions","organizations_url":"https://api.github.com/users/TJKlein/orgs","repos_url":"https://api.github.com/users/TJKlein/repos","events_url":"https://api.github.com/users/TJKlein/events{/privacy}","received_events_url":"https://api.github.com/users/TJKlein/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/BVLC/caffe/labels/bug","name":"bug","color":"fc2929"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":26,"created_at":"2015-04-18T02:08:17Z","updated_at":"2015-05-13T12:44:13Z","closed_at":null,"body":"Hi,\r\n\r\nwhen choosing MemoryDataLayer as input in Python I get segmentation faults and Check failed: status == CUBLAS_STATUS_SUCCESS (14 vs. 0)  CUBLAS_STATUS_INTERNAL_ERROR.\r\nFor small data sets it runs for a while until the Python kernel crashes with the above error. For large data sets this error occurs immediately. Tested in on both OSX and Linux, with the latest master branch. Same issue\r\n\r\nHowever, when using HDF5 layer input for the exactly same data / network, it works perfectly. I used the networks as defined in the tutorial - with the adaption of MemoryDataLayer: \r\n\r\nhttp://nbviewer.ipython.org/github/BVLC/caffe/blob/tutorial/examples/01-learning-lenet.ipynb\r\n\r\nI am also training / testing on MNIST data set\r\n\r\n\r\n```\r\nlayer {\r\n  name: \"data\"\r\n  type: \"MemoryData\"\r\n  top: \"data\"\r\n  top: \"label\"\r\n  memory_data_param {\r\n    batch_size: 50\r\n    channels: 1\r\n    height: 28\r\n    width: 28\r\n  }\r\n}\r\nlayer {\r\n  name: \"conv1\"\r\n  type: \"Convolution\"\r\n  bottom: \"data\"\r\n  top: \"conv1\"\r\n  convolution_param {\r\n    num_output: 20\r\n    kernel_size: 5\r\n    weight_filler {\r\n      type: \"xavier\"\r\n    }\r\n  }\r\n}\r\nlayer {\r\n  name: \"pool1\"\r\n  type: \"Pooling\"\r\n  bottom: \"conv1\"\r\n  top: \"pool1\"\r\n  pooling_param {\r\n    pool: MAX\r\n    kernel_size: 2\r\n    stride: 2\r\n  }\r\n}\r\nlayer {\r\n  name: \"conv2\"\r\n  type: \"Convolution\"\r\n  bottom: \"pool1\"\r\n  top: \"conv2\"\r\n  convolution_param {\r\n    num_output: 50\r\n    kernel_size: 5\r\n    weight_filler {\r\n      type: \"xavier\"\r\n    }\r\n  }\r\n}\r\nlayer {\r\n  name: \"pool2\"\r\n  type: \"Pooling\"\r\n  bottom: \"conv2\"\r\n  top: \"pool2\"\r\n  pooling_param {\r\n    pool: MAX\r\n    kernel_size: 2\r\n    stride: 2\r\n  }\r\n}\r\nlayer {\r\n  name: \"ip1\"\r\n  type: \"InnerProduct\"\r\n  bottom: \"pool2\"\r\n  top: \"ip1\"\r\n  inner_product_param {\r\n    num_output: 500\r\n    weight_filler {\r\n      type: \"xavier\"\r\n    }\r\n  }\r\n}\r\nlayer {\r\n  name: \"relu1\"\r\n  type: \"ReLU\"\r\n  bottom: \"ip1\"\r\n  top: \"ip1\"\r\n}\r\nlayer {\r\n  name: \"ip2\"\r\n  type: \"InnerProduct\"\r\n  bottom: \"ip1\"\r\n  top: \"ip2\"\r\n  inner_product_param {\r\n    num_output: 10\r\n    weight_filler {\r\n      type: \"xavier\"\r\n    }\r\n  }\r\n}\r\nlayer {\r\n  name: \"loss\"\r\n  type: \"SoftmaxWithLoss\"\r\n  bottom: \"ip2\"\r\n  bottom: \"label\"\r\n  top: \"loss\"\r\n}\r\n```\r\nSetting up the data and the solver\r\n```\r\nnum_elements_train = 50000\r\nnum_elements_test = 2000\r\n\r\ntrain_set_x = train_set_x[0:num_elements_train,:].eval()\r\ntrain_set_y = train_set_y[0:num_elements_train].eval()\r\ntest_set_x = test_set_x[0:num_elements_test,:].eval()\r\ntest_set_y = test_set_y[0:num_elements_test].eval()\r\n\r\ntrain_set_x = train_set_x.reshape((num_elements_train,1,28,28))\r\ntrain_set_y = train_set_y.reshape((num_elements_train))\r\n\r\ntest_set_x = test_set_x.reshape((num_elements_test,1,28,28))\r\ntest_set_y = test_set_y.reshape((num_elements_test))\r\n\r\n\r\nsolver.net.set_input_arrays(train_set_x.astype(np.float32),train_set_y.astype(np.float32))\r\nsolver.test_nets[0].set_input_arrays(test_set_x.astype(np.float32),test_set_y.astype(np.float32))\r\n```\r\n\r\nSame as in the example code:\r\n\r\n```\r\nniter = 1000\r\ntest_interval = 25\r\n# losses will also be stored in the log\r\ntrain_loss = zeros(niter)\r\ntest_acc = zeros(int(np.ceil(niter / test_interval)))\r\noutput = zeros((niter, 8, 10))\r\n\r\n# the main solver loop\r\nfor it in range(niter):\r\n    solver.step(1)  # SGD by Caffe\r\n\r\n    # store the train loss\r\n    train_loss[it] = solver.net.blobs['loss'].data\r\n\r\n    # store the output on the first test batch\r\n    # (start the forward pass at conv1 to avoid loading new data)\r\n    solver.test_nets[0].forward(start='conv1')\r\n    output[it] = solver.test_nets[0].blobs['ip2'].data[:8]\r\n\r\n    # run a full test every so often\r\n    # (Caffe can also do this for us and write to a log, but we show here\r\n    #  how to do it directly in Python, where more complicated things are easier.)\r\n    if it % test_interval == 0:\r\n        print 'Iteration', it, 'testing...'\r\n        correct = 0\r\n        for test_it in range(100):\r\n            solver.test_nets[0].forward()\r\n            correct += sum(solver.test_nets[0].blobs['ip2'].data.argmax(1)\r\n                           == solver.test_nets[0].blobs['label'].data)\r\n        test_acc[it // test_interval] = correct / 1e4\r\n```"},"comment":{"url":"https://api.github.com/repos/BVLC/caffe/issues/comments/101652194","html_url":"https://github.com/BVLC/caffe/issues/2334#issuecomment-101652194","issue_url":"https://api.github.com/repos/BVLC/caffe/issues/2334","id":101652194,"user":{"login":"TJKlein","id":7634373,"avatar_url":"https://avatars.githubusercontent.com/u/7634373?v=3","gravatar_id":"","url":"https://api.github.com/users/TJKlein","html_url":"https://github.com/TJKlein","followers_url":"https://api.github.com/users/TJKlein/followers","following_url":"https://api.github.com/users/TJKlein/following{/other_user}","gists_url":"https://api.github.com/users/TJKlein/gists{/gist_id}","starred_url":"https://api.github.com/users/TJKlein/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TJKlein/subscriptions","organizations_url":"https://api.github.com/users/TJKlein/orgs","repos_url":"https://api.github.com/users/TJKlein/repos","events_url":"https://api.github.com/users/TJKlein/events{/privacy}","received_events_url":"https://api.github.com/users/TJKlein/received_events","type":"User","site_admin":false},"created_at":"2015-05-13T12:44:13Z","updated_at":"2015-05-13T12:44:13Z","body":"That's weird.  For me it works well (on three different machines) and for me it passes all the tests. Technically, I am working on a different branch. But I don't see why this should have any effect on it."}}