{"action":"reopened","issue":{"url":"https://api.github.com/repos/SkymindIO/deeplearning4j/issues/123","labels_url":"https://api.github.com/repos/SkymindIO/deeplearning4j/issues/123/labels{/name}","comments_url":"https://api.github.com/repos/SkymindIO/deeplearning4j/issues/123/comments","events_url":"https://api.github.com/repos/SkymindIO/deeplearning4j/issues/123/events","html_url":"https://github.com/SkymindIO/deeplearning4j/issues/123","id":54671069,"number":123,"title":"batch training produce poor results","user":{"login":"winstonquock","id":89413,"avatar_url":"https://avatars.githubusercontent.com/u/89413?v=3","gravatar_id":"","url":"https://api.github.com/users/winstonquock","html_url":"https://github.com/winstonquock","followers_url":"https://api.github.com/users/winstonquock/followers","following_url":"https://api.github.com/users/winstonquock/following{/other_user}","gists_url":"https://api.github.com/users/winstonquock/gists{/gist_id}","starred_url":"https://api.github.com/users/winstonquock/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/winstonquock/subscriptions","organizations_url":"https://api.github.com/users/winstonquock/orgs","repos_url":"https://api.github.com/users/winstonquock/repos","events_url":"https://api.github.com/users/winstonquock/events{/privacy}","received_events_url":"https://api.github.com/users/winstonquock/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2015-01-17T18:51:12Z","updated_at":"2015-02-05T20:04:54Z","closed_at":null,"body":"In this multi-layer config, full set training produce reasonable result of F1 but batching the training produce very poor results\r\n\r\n```\r\n    trainingSetFetcher.fetch(nSamples)\r\n\r\n    val trainingSet = trainingSetFetcher.next()!!\r\n    val col3 = columns(trainingSet.getFeatures(), 7, 8)\r\n\r\n    trainingSet.normalizeZeroMeanZeroUnitVariance()\r\n    val layerSizes = array(500, 100, 10) // 500, 100, 10\r\n    val gen = MersenneTwister(123)\r\n\r\n    val iteration = 5\r\n    val layerFactory = LayerFactories.getFactory(javaClass<RBM>())\r\n    val listBuilder = NeuralNetConfiguration.Builder()\r\n            .iterations(iteration)!!\r\n            .weightInit(WeightInit.DISTRIBUTION)!!\r\n            .dist(Distributions.normal(gen, 1e-2))!!\r\n            .activationFunction(Activations.tanh())!!\r\n            .hiddenUnit(RBM.HiddenUnit.BINARY)!!\r\n            .visibleUnit(RBM.VisibleUnit.BINARY)!!\r\n            .layerFactory(layerFactory)!!\r\n            .constrainGradientToUnitNorm(true )!!\r\n            .optimizationAlgo(OptimizationAlgorithm.ITERATION_GRADIENT_DESCENT)!!\r\n            .rng(gen)!!\r\n            .learningRate(5e-2.toDouble())!!\r\n            .nIn(trainingSet.numInputs())!!\r\n            .nOut(trainingSet.numOutcomes())!!\r\n            .list(layerSizes.size() + 1)!!\r\n            .hiddenLayerSizes(*layerSizes.toIntArray())\r\n\r\n    listBuilder.override({ i, builder ->\r\n                if (i >= layerSizes.size() ) { // 0 based, last one is output\r\n                    builder.layerFactory(DefaultLayerFactory(javaClass<OutputLayer>()))\r\n                    builder.activationFunction(Activations.tanh())\r\n                    builder.lossFunction(LossFunctions.LossFunction.MCXENT)\r\n                    builder.iterations(iteration)\r\n                }\r\n            })\r\n    val conf = listBuilder.build()!!\r\n    val nn = MultiLayerNetwork(conf)\r\n    if (true) { // batched\r\n        var batch = 1\r\n        val iter = ListDataSetIterator(trainingSet.asList(), 500)\r\n        while (iter.hasNext()) {\r\n            println(\"================== train batch ${batch++} ==================\")\r\n            nn.fit(iter.next())\r\n        }\r\n    } else { // full set\r\n        nn.fit(trainingSet)\r\n    }\r\n```\r\n(data set provided separately)"}}