{"action":"opened","issue":{"url":"https://api.github.com/repos/deeplearning4j/deeplearning4j/issues/429","labels_url":"https://api.github.com/repos/deeplearning4j/deeplearning4j/issues/429/labels{/name}","comments_url":"https://api.github.com/repos/deeplearning4j/deeplearning4j/issues/429/comments","events_url":"https://api.github.com/repos/deeplearning4j/deeplearning4j/issues/429/events","html_url":"https://github.com/deeplearning4j/deeplearning4j/issues/429","id":94554182,"number":429,"title":"CUBLAS_STATUS_INTERNAL_ERROR when running ml.JavaIrisClassification example","user":{"login":"mikeorzel2","id":8913986,"avatar_url":"https://avatars.githubusercontent.com/u/8913986?v=3","gravatar_id":"","url":"https://api.github.com/users/mikeorzel2","html_url":"https://github.com/mikeorzel2","followers_url":"https://api.github.com/users/mikeorzel2/followers","following_url":"https://api.github.com/users/mikeorzel2/following{/other_user}","gists_url":"https://api.github.com/users/mikeorzel2/gists{/gist_id}","starred_url":"https://api.github.com/users/mikeorzel2/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mikeorzel2/subscriptions","organizations_url":"https://api.github.com/users/mikeorzel2/orgs","repos_url":"https://api.github.com/users/mikeorzel2/repos","events_url":"https://api.github.com/users/mikeorzel2/events{/privacy}","received_events_url":"https://api.github.com/users/mikeorzel2/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-07-12T10:33:02Z","updated_at":"2015-07-12T10:33:02Z","closed_at":null,"body":"Hi,\r\n\r\nI am currently getting an error while trying to run the ml.JavaIrisClassification example against a GPU.\r\n\r\nStack trace of error is below\r\n15/07/12 09:42:44 INFO MultiLayerNetwork: Training on layer 1 with 181 examples\r\n15/07/12 09:42:44 ERROR Executor: Exception in task 0.0 in stage 6.0 (TID 6)\r\njcuda.CudaException: CUBLAS_STATUS_INTERNAL_ERROR\r\n\tat jcuda.jcublas.JCublas2.checkResult(JCublas2.java:127)\r\n\tat jcuda.jcublas.JCublas2.cublasSdot(JCublas2.java:642)\r\n\tat org.nd4j.linalg.jcublas.blas.JcublasLevel1.sdot(JcublasLevel1.java:42)\r\n\tat org.nd4j.linalg.api.blas.impl.BaseLevel1.dot(BaseLevel1.java:32)\r\n\tat org.nd4j.linalg.factory.BaseBlasWrapper.dot(BaseBlasWrapper.java:185)\r\n\tat org.deeplearning4j.optimize.solvers.BackTrackLineSearch.optimize(BackTrackLineSearch.java:152)\r\n\tat org.deeplearning4j.optimize.solvers.BaseOptimizer.optimize(BaseOptimizer.java:173)\r\n\tat org.deeplearning4j.optimize.Solver.optimize(Solver.java:52)\r\n\tat org.deeplearning4j.nn.layers.BaseLayer.fit(BaseLayer.java:444)\r\n\tat org.deeplearning4j.nn.layers.feedforward.rbm.RBM.fit(RBM.java:399)\r\n\tat org.deeplearning4j.nn.multilayer.MultiLayerNetwork.pretrain(MultiLayerNetwork.java:216)\r\n\tat org.deeplearning4j.nn.multilayer.MultiLayerNetwork.fit(MultiLayerNetwork.java:1242)\r\n\tat org.deeplearning4j.spark.ml.classification.NeuralNetworkClassification$$anonfun$1.apply(MultiLayerNetworkClassification.scala:121)\r\n\tat org.deeplearning4j.spark.ml.classification.NeuralNetworkClassification$$anonfun$1.apply(MultiLayerNetworkClassification.scala:108)\r\n\tat org.deeplearning4j.spark.ml.nn.ParameterAveragingTrainingStrategy$$anonfun$train$1$$anonfun$apply$mcVI$sp$1.apply(TrainingStrategy.scala:89)\r\n\tat org.deeplearning4j.spark.ml.nn.ParameterAveragingTrainingStrategy$$anonfun$train$1$$anonfun$apply$mcVI$sp$1.apply(TrainingStrategy.scala:81)\r\n\tat org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:878)\r\n\tat org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:878)\r\n\tat org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1765)\r\n\tat org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1765)\r\n\tat org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:63)\r\n\tat org.apache.spark.scheduler.Task.run(Task.scala:70)\r\n\tat org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)\r\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\r\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\r\n\tat java.lang.Thread.run(Thread.java:745)\r\n\r\nI have run the nd4j test cases and they pass for jcublas-7.0\r\n\r\nEnvironment is a single node, Titan X GPU, cuda 7.0.28 nvidia 346.46 drivers running in a docker container with the following params\r\n\r\ndocker run -it --privileged=true --device=/dev/nvidia0:/dev/nvidia0 --device=/dev/nvidia1:/dev/nvidia1 --device=/dev/nvidiactl:/dev/nvidiactl --device=/dev/nvidia-uvm:/dev/nvidia-uvm deeplearning4j bash\r\n\r\nThe container is based on ubuntu14.04 and includes all the items for deeplearning4j with it running fine against cpu and jblas."}}