{"action":"created","issue":{"url":"https://api.github.com/repos/SkymindIO/deeplearning4j/issues/123","labels_url":"https://api.github.com/repos/SkymindIO/deeplearning4j/issues/123/labels{/name}","comments_url":"https://api.github.com/repos/SkymindIO/deeplearning4j/issues/123/comments","events_url":"https://api.github.com/repos/SkymindIO/deeplearning4j/issues/123/events","html_url":"https://github.com/SkymindIO/deeplearning4j/issues/123","id":54671069,"number":123,"title":"batch training produce poor results","user":{"login":"winstonquock","id":89413,"avatar_url":"https://avatars.githubusercontent.com/u/89413?v=3","gravatar_id":"","url":"https://api.github.com/users/winstonquock","html_url":"https://github.com/winstonquock","followers_url":"https://api.github.com/users/winstonquock/followers","following_url":"https://api.github.com/users/winstonquock/following{/other_user}","gists_url":"https://api.github.com/users/winstonquock/gists{/gist_id}","starred_url":"https://api.github.com/users/winstonquock/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/winstonquock/subscriptions","organizations_url":"https://api.github.com/users/winstonquock/orgs","repos_url":"https://api.github.com/users/winstonquock/repos","events_url":"https://api.github.com/users/winstonquock/events{/privacy}","received_events_url":"https://api.github.com/users/winstonquock/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-01-17T18:51:12Z","updated_at":"2015-02-01T00:13:55Z","closed_at":null,"body":"In this multi-layer config, full set training produce reasonable result of F1 but batching the training produce very poor results\r\n\r\n```\r\n    trainingSetFetcher.fetch(nSamples)\r\n\r\n    val trainingSet = trainingSetFetcher.next()!!\r\n    val col3 = columns(trainingSet.getFeatures(), 7, 8)\r\n\r\n    trainingSet.normalizeZeroMeanZeroUnitVariance()\r\n    val layerSizes = array(500, 100, 10) // 500, 100, 10\r\n    val gen = MersenneTwister(123)\r\n\r\n    val iteration = 5\r\n    val layerFactory = LayerFactories.getFactory(javaClass<RBM>())\r\n    val listBuilder = NeuralNetConfiguration.Builder()\r\n            .iterations(iteration)!!\r\n            .weightInit(WeightInit.DISTRIBUTION)!!\r\n            .dist(Distributions.normal(gen, 1e-2))!!\r\n            .activationFunction(Activations.tanh())!!\r\n            .hiddenUnit(RBM.HiddenUnit.BINARY)!!\r\n            .visibleUnit(RBM.VisibleUnit.BINARY)!!\r\n            .layerFactory(layerFactory)!!\r\n            .constrainGradientToUnitNorm(true )!!\r\n            .optimizationAlgo(OptimizationAlgorithm.ITERATION_GRADIENT_DESCENT)!!\r\n            .rng(gen)!!\r\n            .learningRate(5e-2.toDouble())!!\r\n            .nIn(trainingSet.numInputs())!!\r\n            .nOut(trainingSet.numOutcomes())!!\r\n            .list(layerSizes.size() + 1)!!\r\n            .hiddenLayerSizes(*layerSizes.toIntArray())\r\n\r\n    listBuilder.override({ i, builder ->\r\n                if (i >= layerSizes.size() ) { // 0 based, last one is output\r\n                    builder.layerFactory(DefaultLayerFactory(javaClass<OutputLayer>()))\r\n                    builder.activationFunction(Activations.tanh())\r\n                    builder.lossFunction(LossFunctions.LossFunction.MCXENT)\r\n                    builder.iterations(iteration)\r\n                }\r\n            })\r\n    val conf = listBuilder.build()!!\r\n    val nn = MultiLayerNetwork(conf)\r\n    if (true) { // batched\r\n        var batch = 1\r\n        val iter = ListDataSetIterator(trainingSet.asList(), 500)\r\n        while (iter.hasNext()) {\r\n            println(\"================== train batch ${batch++} ==================\")\r\n            nn.fit(iter.next())\r\n        }\r\n    } else { // full set\r\n        nn.fit(trainingSet)\r\n    }\r\n```\r\n(data set provided separately)"},"comment":{"url":"https://api.github.com/repos/SkymindIO/deeplearning4j/issues/comments/72344112","html_url":"https://github.com/SkymindIO/deeplearning4j/issues/123#issuecomment-72344112","issue_url":"https://api.github.com/repos/SkymindIO/deeplearning4j/issues/123","id":72344112,"user":{"login":"agibsonccc","id":1144306,"avatar_url":"https://avatars.githubusercontent.com/u/1144306?v=3","gravatar_id":"","url":"https://api.github.com/users/agibsonccc","html_url":"https://github.com/agibsonccc","followers_url":"https://api.github.com/users/agibsonccc/followers","following_url":"https://api.github.com/users/agibsonccc/following{/other_user}","gists_url":"https://api.github.com/users/agibsonccc/gists{/gist_id}","starred_url":"https://api.github.com/users/agibsonccc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/agibsonccc/subscriptions","organizations_url":"https://api.github.com/users/agibsonccc/orgs","repos_url":"https://api.github.com/users/agibsonccc/repos","events_url":"https://api.github.com/users/agibsonccc/events{/privacy}","received_events_url":"https://api.github.com/users/agibsonccc/received_events","type":"User","site_admin":false},"created_at":"2015-02-01T00:13:55Z","updated_at":"2015-02-01T00:13:55Z","body":"So I'm ready to look in to this. How big was your dataset? You're pre processing it right. How are you doing with your test set though? Are you normalizing the training there as well? What batch sizes did you try?"}}