{"action":"created","issue":{"url":"https://api.github.com/repos/dnouri/nolearn/issues/80","labels_url":"https://api.github.com/repos/dnouri/nolearn/issues/80/labels{/name}","comments_url":"https://api.github.com/repos/dnouri/nolearn/issues/80/comments","events_url":"https://api.github.com/repos/dnouri/nolearn/issues/80/events","html_url":"https://github.com/dnouri/nolearn/issues/80","id":71769461,"number":80,"title":"Non-mutually exclusive outcomes","user":{"login":"BrianMiner","id":8807554,"avatar_url":"https://avatars.githubusercontent.com/u/8807554?v=3","gravatar_id":"","url":"https://api.github.com/users/BrianMiner","html_url":"https://github.com/BrianMiner","followers_url":"https://api.github.com/users/BrianMiner/followers","following_url":"https://api.github.com/users/BrianMiner/following{/other_user}","gists_url":"https://api.github.com/users/BrianMiner/gists{/gist_id}","starred_url":"https://api.github.com/users/BrianMiner/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/BrianMiner/subscriptions","organizations_url":"https://api.github.com/users/BrianMiner/orgs","repos_url":"https://api.github.com/users/BrianMiner/repos","events_url":"https://api.github.com/users/BrianMiner/events{/privacy}","received_events_url":"https://api.github.com/users/BrianMiner/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-04-29T03:40:17Z","updated_at":"2015-05-20T15:01:02Z","closed_at":null,"body":"Is it possible to have non mutually exclusive outcomes in nolearn / lasagne? Specifically where there is a vector of outcomes for each training observation? Something along these lines? What would be a suitable loss function?\r\n\r\nimport numpy as np\r\n\r\nX=np.random.random((50, 10))\r\ny=np.random.randint(2, size=(50,3))\r\ny=np.cast['int32'](y)\r\n\r\nnum_classes = 3\r\nnum_features = X.shape[1]\r\n\r\nlayers0 = [('input', InputLayer),\r\n           ('dense0', DenseLayer),\r\n           ('output', DenseLayer)]\r\n\r\n\r\nnet0 = NeuralNet(layers=layers0,\r\n                 \r\n                 input_shape=(None, num_features),  \r\n                 dense0_num_units=3, \r\n                 \r\n                 output_num_units=num_classes,\r\n                 output_nonlinearity=sigmoid,\r\n                 \r\n                 objective_loss_function =categorical_crossentropy,\r\n                 \r\n                 update=nesterov_momentum,\r\n                 update_learning_rate=0.5,\r\n                 update_momentum=0.9,\r\n                 \r\n                 eval_size=0,\r\n                 verbose=1,\r\n                 max_epochs=50)\r\n\r\nnet0.fit(X, y)"},"comment":{"url":"https://api.github.com/repos/dnouri/nolearn/issues/comments/103919108","html_url":"https://github.com/dnouri/nolearn/issues/80#issuecomment-103919108","issue_url":"https://api.github.com/repos/dnouri/nolearn/issues/80","id":103919108,"user":{"login":"swarbrickjones","id":7874482,"avatar_url":"https://avatars.githubusercontent.com/u/7874482?v=3","gravatar_id":"","url":"https://api.github.com/users/swarbrickjones","html_url":"https://github.com/swarbrickjones","followers_url":"https://api.github.com/users/swarbrickjones/followers","following_url":"https://api.github.com/users/swarbrickjones/following{/other_user}","gists_url":"https://api.github.com/users/swarbrickjones/gists{/gist_id}","starred_url":"https://api.github.com/users/swarbrickjones/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/swarbrickjones/subscriptions","organizations_url":"https://api.github.com/users/swarbrickjones/orgs","repos_url":"https://api.github.com/users/swarbrickjones/repos","events_url":"https://api.github.com/users/swarbrickjones/events{/privacy}","received_events_url":"https://api.github.com/users/swarbrickjones/received_events","type":"User","site_admin":false},"created_at":"2015-05-20T15:01:02Z","updated_at":"2015-05-20T15:01:02Z","body":"I've just done this, it seems to work OK.\r\n\r\nN.B. you need to leave the regression flag = True, otherwise the stratified crossvalidation nolearn uses will throw an error.  Also remember to put a nonlinearity on the output to fix it to (0,1) , NOT softmax, I used sigmoid.\r\n\r\nHere's the net I used - \r\n\r\n```python\r\nae = NeuralNet(\r\n    layers=[\r\n        ('input', layers.InputLayer),\r\n        ('conv10', layers.Conv2DLayer),\r\n        #('conv11', layers.Conv2DLayer),\r\n        ('pool1', layers.MaxPool2DLayer),\r\n        ('conv20', layers.Conv2DLayer),\r\n        ('conv21', layers.Conv2DLayer),\r\n        ('pool2', layers.MaxPool2DLayer),\r\n        ('hidden1', layers.DenseLayer),\r\n        ('hidden2', layers.DenseLayer),\r\n        #('hidden3', layers.DenseLayer),\r\n        ('output', layers.DenseLayer),\r\n        ],\r\n    input_shape=(None, 3, 80, 60),\r\n    conv10_num_filters=16, \r\n    conv10_filter_size=(3, 3), \r\n    conv10_border_mode=\"valid\",\r\n    #conv11_num_filters=32, \r\n    #conv11_filter_size=(3, 3), \r\n    #conv11_border_mode=\"valid\",\r\n    pool1_ds=(2, 2),\r\n    conv20_num_filters=64, \r\n    conv20_filter_size=(5, 5), \r\n    conv20_border_mode=\"valid\",\r\n    pool2_ds=(2, 2),\r\n    conv21_num_filters=64, \r\n    conv21_filter_size=(5, 5), \r\n    conv21_border_mode=\"valid\",\r\n    hidden1_num_units = 1048,\r\n    hidden2_num_units = 1048,\r\n    #hidden3_num_units=512,\r\n    output_num_units = y_train.shape[1],\r\n    output_nonlinearity = lasagne.nonlinearities.sigmoid,\r\n    update_learning_rate=0.01,\r\n    update_momentum=0.975,\r\n    regression = True,\r\n    batch_iterator_train=FlipBatchIterator(batch_size=128),\r\n    objective_loss_function=lasagne.objectives.binary_crossentropy,\r\n    max_epochs= 100,\r\n    verbose=1,\r\n    )\r\n```"}}