{"action":"opened","issue":{"url":"https://api.github.com/repos/deeplearning4j/deeplearning4j/issues/839","labels_url":"https://api.github.com/repos/deeplearning4j/deeplearning4j/issues/839/labels{/name}","comments_url":"https://api.github.com/repos/deeplearning4j/deeplearning4j/issues/839/comments","events_url":"https://api.github.com/repos/deeplearning4j/deeplearning4j/issues/839/events","html_url":"https://github.com/deeplearning4j/deeplearning4j/issues/839","id":115026177,"number":839,"title":"redistributeParams cannot be modified if NeuralNetConfiguration.Builder() is used to construct a configuration","user":{"login":"DennisChen0307","id":13622753,"avatar_url":"https://avatars.githubusercontent.com/u/13622753?v=3","gravatar_id":"","url":"https://api.github.com/users/DennisChen0307","html_url":"https://github.com/DennisChen0307","followers_url":"https://api.github.com/users/DennisChen0307/followers","following_url":"https://api.github.com/users/DennisChen0307/following{/other_user}","gists_url":"https://api.github.com/users/DennisChen0307/gists{/gist_id}","starred_url":"https://api.github.com/users/DennisChen0307/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DennisChen0307/subscriptions","organizations_url":"https://api.github.com/users/DennisChen0307/orgs","repos_url":"https://api.github.com/users/DennisChen0307/repos","events_url":"https://api.github.com/users/DennisChen0307/events{/privacy}","received_events_url":"https://api.github.com/users/DennisChen0307/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-11-04T10:53:26Z","updated_at":"2015-11-04T10:53:26Z","closed_at":null,"body":"I was searching for methods to save and load model of the network and I got  this article:\r\n\r\n    http://deeplearning4j.org/modelpersistence.html\r\n\r\nThe code seemed to be worked for me, but at last I got wrong parameters output from the newly loaded model: all the values are zero. \r\n\r\nOriginal network params [ 47.32, -47.21, -47.23, -47.27, -47.15, 47.18, 47.34, 47.17, 47.15.....\r\n\r\nSaved network params [ 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, 0.00, ....\r\n\r\nHere I use NeuralNetConfiguration.Builder() to build  MultiLayerConfiguration.\r\nI went throught the source code and I believe it is because of this:\r\n\r\npublic class NeuralNetConfiguration implements Serializable,Cloneable {\r\n    public static class ListBuilder extends MultiLayerConfiguration.Builder {\r\n......\r\n        public MultiLayerConfiguration build() {\r\n            List<NeuralNetConfiguration> list = new ArrayList<>();\r\n            for(int i = 0; i < layerwise.size(); i++) {\r\n                list.add(layerwise.get(i).build());\r\n            }\r\n            return new MultiLayerConfiguration.Builder().backprop(backprop).inputPreProcessors(inputPreProcessors).\r\n                    pretrain(pretrain).backpropType(backpropType).tBPTTForwardLength(tbpttFwdLength)\r\n                    .tBPTTBackwardLength(tbpttBackLength)\r\n                    .redistributeParams(redistributeParams)\r\n                    .confs(list).build();\r\n        }\r\n......\r\n\r\nWe don't have a interface to modify redistributeParams for a MultiLayerConfiguration in NeuralNetConfiguration so it can only be false by default.\r\n\r\nI found there is a very interesting point of this methodology: the model can only be trained once and redistributed many times. But I still believe this is not what a network should really behave. We cannot tell whether it is loaded properly\r\n\r\nI paste my code here:\r\n\r\n\r\n\r\n/*\r\n * To change this license header, choose License Headers in Project Properties.\r\n * To change this template file, choose Tools | Templates\r\n * and open the template in the editor.\r\n */\r\npackage com.test;\r\n\r\nimport java.io.DataInputStream;\r\nimport java.io.DataOutputStream;\r\nimport java.io.File;\r\nimport java.io.FileInputStream;\r\nimport java.io.OutputStream;\r\nimport java.nio.file.Files;\r\nimport java.nio.file.Paths;\r\nimport org.apache.commons.io.FileUtils;\r\nimport org.deeplearning4j.eval.Evaluation;\r\nimport org.deeplearning4j.nn.api.OptimizationAlgorithm;\r\nimport org.deeplearning4j.nn.conf.MultiLayerConfiguration;\r\nimport org.deeplearning4j.nn.conf.NeuralNetConfiguration;\r\nimport org.deeplearning4j.nn.conf.layers.DenseLayer;\r\nimport org.deeplearning4j.nn.conf.layers.OutputLayer;\r\nimport org.deeplearning4j.nn.multilayer.MultiLayerNetwork;\r\nimport org.deeplearning4j.nn.weights.WeightInit;\r\nimport org.deeplearning4j.optimize.listeners.ScoreIterationListener;\r\nimport org.nd4j.linalg.api.ndarray.INDArray;\r\nimport org.nd4j.linalg.dataset.DataSet;\r\nimport org.nd4j.linalg.dataset.SplitTestAndTrain;\r\nimport org.nd4j.linalg.factory.Nd4j;\r\nimport org.nd4j.linalg.lossfunctions.LossFunctions;\r\nimport org.springframework.core.io.ClassPathResource;\r\nimport weka.core.Instances;\r\nimport weka.core.converters.ConverterUtils;\r\nimport org.nd4j.linalg.api.ops.impl.transforms.*;\r\n\r\n/**\r\n * Created by agibsonccc on 9/11/14.\r\n *\r\n * Diff from small single layer\r\n */\r\npublic class DeepLearner {\r\n\r\n    public static void main(String[] args) throws Exception {\r\n        \r\n        /*\r\n        Predictor p = new Predictor(12);\r\n\r\n        Instances data1=ConverterUtils.DataSource.read(new ClassPathResource(\"abc.arff\").getPath());\r\n\r\n        Instances samples = p.assembleSample(data1);\r\n        \r\n        DataSet ds=Utils.wekaInstanceToDLDataSet(samples);\r\n        */\r\n        \r\n        \r\n        int nSamples = 50;\r\n        INDArray x = Nd4j.linspace(-10, 10, nSamples).reshape(nSamples, 1);\r\n        INDArray y = x.div(2);\r\n        DataSet ds = new DataSet(x, y);\r\n        \r\n        \r\n        ds.normalize();\r\n        \r\n        int seed = 10000;\r\n        int iterations = 100;\r\n        int inputNum = ds.numInputs();\r\n        int hiddenNum = 100;\r\n        double learningRate = 5000;\r\n\r\n        MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder()\r\n                .seed(seed) // Seed to lock in weight initia-lization for tuning\r\n                .iterations(iterations) // # training iterations predict/classify & backprop\r\n                .optimizationAlgo(OptimizationAlgorithm.LINE_GRADIENT_DESCENT)\r\n                .learningRate(learningRate) // Optimization step size\r\n                .constrainGradientToUnitNorm(false)\r\n                \r\n                .list(4) // # NN layers (does not count input layer)\r\n                .layer(0, new DenseLayer.Builder()\r\n                                .nIn(inputNum) // # input nodes\r\n                                .nOut(hiddenNum) // # output nodes\r\n                                .activation(\"tanh\")\r\n                                .weightInit(WeightInit.XAVIER)\r\n                                .build()\r\n                )\r\n                .layer(1, new DenseLayer.Builder()\r\n                                .nIn(hiddenNum) // # input nodes\r\n                                .nOut(hiddenNum*2) // # output nodes\r\n                                .activation(\"tanh\")\r\n                                .weightInit(WeightInit.XAVIER)\r\n                                .build()\r\n                )\r\n                .layer(2, new DenseLayer.Builder()\r\n                                .nIn(hiddenNum*2) // # input nodes\r\n                                .nOut(hiddenNum) // # output nodes\r\n                                .activation(\"tanh\")\r\n                                .weightInit(WeightInit.XAVIER)\r\n                                .build()\r\n                )\r\n                .layer(3, new OutputLayer.Builder(LossFunctions.LossFunction.MSE)\r\n                                .nIn(hiddenNum) // # input nodes\r\n                                .nOut(1) // # output nodes\r\n                                .activation(\"identity\")\r\n                                .weightInit(WeightInit.XAVIER)\r\n                                .build()\r\n                )// NN layer type\r\n                .backprop(true)\r\n                .build();\r\n       \r\n        \r\n        System.out.println(ds.numInputs());\r\n        System.out.println(ds.numExamples());\r\n        System.out.println(ds.getLabels());\r\n        \r\n        \r\n        MultiLayerNetwork network = new MultiLayerNetwork(conf);\r\n        network.init();\r\n        //network.setListeners(new ScoreIterationListener(1));\r\n        network.fit(ds);\r\n        SplitTestAndTrain testAndTrain = ds.splitTestAndTrain(0.9);\r\n        DataSet test = testAndTrain.getTest();\r\n        INDArray testFeature = test.getFeatureMatrix();\r\n        INDArray testLabel = test.getLabels();\r\n        INDArray testPredictedLabel = network.output(testFeature);\r\n        // Fitting model\r\n        // Get predictions from test feature\r\n        /*\r\n        \r\n        \r\n        INDArray yEst = network.output(x);\r\n        INDArray error = Nd4j.std(testPredictedLabel.sub(yEst));\r\n\r\n        System.out.println(\"X\" + x);\r\n        System.out.println(\"Z\" + yEst);\r\n        System.out.println(\"Y\" + y);\r\n        System.out.println(error);\r\n        */\r\n\r\n        // Eval without class number\r\n        //Evaluation eval2 = new Evaluation(); //// No class num\r\n        \r\n        System.out.println(testLabel);\r\n        System.out.println(testPredictedLabel);\r\n        System.out.println(network.params());\r\n        //eval2.eval(testLabel, testPredictedLabel);\r\n        \r\n        //System.out.println(eval2.stats());\r\n        \r\n        String coef_file = \"coefficients.bin\";\r\n        String conf_file = \"conf.json\";\r\n        OutputStream fos = Files.newOutputStream(Paths.get(coef_file));\r\n        DataOutputStream dos = new DataOutputStream(fos);\r\n        Nd4j.write(network.params(), dos);\r\n        dos.flush();\r\n        dos.close();\r\n        FileUtils.write(new File(conf_file), network.getLayerWiseConfigurations().toJson());\r\n\r\n        MultiLayerConfiguration confFromJson = MultiLayerConfiguration.fromJson(\r\n                            FileUtils.readFileToString(new File(conf_file)));\r\n        DataInputStream dis = new DataInputStream(new FileInputStream(coef_file));\r\n        INDArray newParams = Nd4j.read(dis);\r\n        System.out.println(newParams);\r\n        dis.close();\r\n        MultiLayerNetwork savedNetwork = new MultiLayerNetwork(confFromJson,newParams);\r\n        savedNetwork.reDistributeParams();\r\n        //savedNetwork.setParameters(newParams);\r\n        //savedNetwork.init();\r\n        System.out.println(\"Original network params \" + network);\r\n        System.out.println(\"Saved network params \" + savedNetwork);\r\n    }\r\n\r\n}\r\n"}}