{"action":"created","issue":{"url":"https://api.github.com/repos/dmlc/mxnet/issues/895","labels_url":"https://api.github.com/repos/dmlc/mxnet/issues/895/labels{/name}","comments_url":"https://api.github.com/repos/dmlc/mxnet/issues/895/comments","events_url":"https://api.github.com/repos/dmlc/mxnet/issues/895/events","html_url":"https://github.com/dmlc/mxnet/issues/895","id":121473746,"number":895,"title":"Help for Adversial example to apply it to a single image with a pre-trained network with Batch Normalization?","user":{"login":"erogol","id":1402048,"avatar_url":"https://avatars.githubusercontent.com/u/1402048?v=3","gravatar_id":"","url":"https://api.github.com/users/erogol","html_url":"https://github.com/erogol","followers_url":"https://api.github.com/users/erogol/followers","following_url":"https://api.github.com/users/erogol/following{/other_user}","gists_url":"https://api.github.com/users/erogol/gists{/gist_id}","starred_url":"https://api.github.com/users/erogol/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/erogol/subscriptions","organizations_url":"https://api.github.com/users/erogol/orgs","repos_url":"https://api.github.com/users/erogol/repos","events_url":"https://api.github.com/users/erogol/events{/privacy}","received_events_url":"https://api.github.com/users/erogol/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-12-10T12:35:30Z","updated_at":"2015-12-11T11:12:23Z","closed_at":null,"body":"My real purpose is to load a pretrained network to the work-flow proposed in example/adversary and obtain a gradient image for a single image.\r\n\r\nI load a pre-trained model by the following code and pass all its parameters to newly created model.\r\n```\r\n# Create new model graph\r\nfc = alexnet_grad_img_model(21)\r\ndev = mx.gpu(0)\r\n\r\n# Load pretrained net\r\nepoch = 25\r\nmodel_prefix = \"models/AlexNet\"\r\ntmp_model = mx.model.FeedForward.load(model_prefix, epoch, ctx=dev)\r\n\r\n# Pass params of trained net to new model \r\nmodel_params = tmp_model.get_params()\r\n\r\ndata_shape = (batch_size, 3, 224, 224)\r\nmodel_params['arg_params']['data'] = mx.nd.zeros(data_shape, ctx=dev)\r\n\r\narg_names = model_params['arg_params'].keys()\r\ngrad_arrays = [(name, mx.nd.zeros(param.shape, ctx=dev)) for name, param in model_params['arg_params'].items()]\r\ngrad_map = dict(grad_arrays)\r\nmodel_params['args_grad'] = grad_map\r\nmodel_params['grad_req'] = dict([(name,\"write\") for name in arg_names])\r\n\r\n# Bind the new model\r\nmodel = fc.bind(ctx=dev, args=model_params['arg_params'], args_grad= model_params['args_grad'], aux_states=model_params['aux_params'], grad_req= model_params['grad_req'])\r\n\r\ndata_grad = model_params[\"args_grad\"]['data']\r\nout_grad = mx.nd.zeros(model.outputs[0].shape, ctx=dev)\r\n```\r\n\r\nAfter this stage, even in everywhere I defined the context as mx.gpu(0), still model stays in cpu memory as the output of ```print model.debug_str()``` says. My first question is how can I keep the new model inside GPU?\r\n\r\nThe second problem is I use BN with AlexNet and when I reduce the batch_size to 1 due to BN (I believe), I got always skewed outputs and always same class prediction. However, if I increase the batch_size to 64 (which is my training batch size), prediction go smoothly. My second question is how can I handle this problem for single image prediction? Below is my stupid code I use for single image prediction with iterators created by mx.io.ImageRecordIter.\r\n\r\n```\r\nnum_round = 4\r\ntrain_acc = 0.\r\nnbatch = 0\r\nfor i in range(num_round):\r\n    train_loss = 0.\r\n    train_acc = 0.\r\n    nbatch = 0\r\n    train_iter.reset()\r\n    for data, label in train_iter:\r\n        arg_map[\"data\"][:] = data\r\n        model.forward(is_train=True)\r\n        theta = model.outputs[0].asnumpy()\r\n        alpha = Softmax(theta)\r\n        train_acc += CalAcc(alpha, label.asnumpy()) / batch_size\r\n        train_loss += CalLoss(alpha, label.asnumpy()) / batch_size\r\n        losGrad_theta = LogLossGrad(alpha, label.asnumpy())\r\n        out_grad[:] = losGrad_theta\r\n        model.backward([out_grad])\r\n        # data_grad[:] = grad_map[\"data\"]\r\n        for name in arg_names:\r\n            if name != \"data\":\r\n                SGD(arg_map[name], grad_map[name])\r\n        \r\n        nbatch += 1\r\n    #print(np.linalg.norm(data_grad.asnumpy(), 2))\r\n    train_acc /= nbatch\r\n    train_loss /= nbatch\r\n    print(\"Train Accuracy: %.2f\\t Train Loss: %.5f\" % (train_acc, train_loss))\r\n````"},"comment":{"url":"https://api.github.com/repos/dmlc/mxnet/issues/comments/163912666","html_url":"https://github.com/dmlc/mxnet/issues/895#issuecomment-163912666","issue_url":"https://api.github.com/repos/dmlc/mxnet/issues/895","id":163912666,"user":{"login":"erogol","id":1402048,"avatar_url":"https://avatars.githubusercontent.com/u/1402048?v=3","gravatar_id":"","url":"https://api.github.com/users/erogol","html_url":"https://github.com/erogol","followers_url":"https://api.github.com/users/erogol/followers","following_url":"https://api.github.com/users/erogol/following{/other_user}","gists_url":"https://api.github.com/users/erogol/gists{/gist_id}","starred_url":"https://api.github.com/users/erogol/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/erogol/subscriptions","organizations_url":"https://api.github.com/users/erogol/orgs","repos_url":"https://api.github.com/users/erogol/repos","events_url":"https://api.github.com/users/erogol/events{/privacy}","received_events_url":"https://api.github.com/users/erogol/received_events","type":"User","site_admin":false},"created_at":"2015-12-11T11:12:23Z","updated_at":"2015-12-11T11:12:23Z","body":"You mean setting an parameter accordingly or coding this function to that layer. But how does indeption net predict on a single image in predict-with-pretrained-model.ipynb ?"}}