<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
Step1:  This example illustrates how simple it is to train a classifier using
side information.
It illustrates the exemplary use of the <b>multi-view</b> pattern; for more info
on how to use other patterns, check out the other examples.
Step2:  Data generation
Step3:  Now let's define some side information
Step4:  Let's make it harder to find S in X and Z by applying a random rotations to both data sets
Step5:  Finally, split our data into training, test, and validation data
Step6:  Purely supervised learning
Let's check how hard the problem is for supervised learning alone.
Step7:  Learning with side information
Step8:  Now define the functions - we choose linear functions.
concarne internally relies on lasagne which encodes functions as (sets of) layers. Additionally, concarne supports nolearn style initialization of lasagne layers as follows
Step9:  For the variable of your layer that denotes the output of the network you should use the markers PHI_OUTPUT_SHAPE,
PSI_OUTPUT_SHAPE and BETA_OUTPUT_SHAPE, so that the pattern can automatically infer the correct shape.
Step10:  Training
To train a pattern, you can use the PatternTrainer which trains the pattern via stochastic gradient descent.
It also supports different procedures to train the pattern.
Step11:  <b>Let's train!</b>
Step12:  Some statistics
Step13:  We can also compute a test score for the side loss
Step14:  You can then also query the prediction output, similar to the scikit-learn API
<ASSISTANT_TASK:>
Python Code:
from __future__ import print_function
import concarne
import concarne.patterns
import concarne.training
import lasagne
import theano.tensor as T
%pylab inline
try:
    import sklearn.linear_model as sklm
except:
    print (
You don't have scikit-learn installed; install it to compare
learning with side information to simple supervised learning)
    sklm = None
import numpy as np
Explanation: This example illustrates how simple it is to train a classifier using
side information.
It illustrates the exemplary use of the <b>multi-view</b> pattern; for more info
on how to use other patterns, check out the other examples.
End of explanation
num_samples = 300
input_dim = 50
side_dim = 50
# generate some random data with 100 samples 
# and 5 dimensions
X = np.random.randn(num_samples, input_dim)
# select the third dimension as the relevant 
# for our classification task
S = X[:, 2:3]
# The labels are simply the sign of S 
# (note the downcast to int32 - this is required 
# by theano)
y = np.asarray(S > 0, dtype='int32').reshape( (-1,) )
# This means we have 2 classes - we will use 
# that later for building the pattern
num_classes = 2
plt.plot(S)
plt.plot(y)
Explanation: Data generation
End of explanation
Z = np.random.randn(num_samples, side_dim)
# set second dimension of Z to correspond to S
Z[:, 1] = S[:,0]
Explanation: Now let's define some side information: we simulate an additional sensorwhich contains S, but embedded into a different space.
End of explanation
# random rotation 1
R = np.linalg.qr(np.random.randn(input_dim, input_dim))[0]
X = X.dot(R)
# random rotation 2
Q = np.linalg.qr(np.random.randn(side_dim, side_dim))[0]
Z = Z.dot(Q)
Explanation: Let's make it harder to find S in X and Z by applying a random rotations to both data sets
End of explanation
split = num_samples/3
X_train = X[:split]
X_val = X[split:2*split]
X_test = X[2*split:]
y_train = y[:split]
y_val = y[split:2*split]
y_test = y[2*split:]
Z_train = Z[:split]
Z_val = Z[split:2*split]
Z_test = Z[2*split:]
Explanation: Finally, split our data into training, test, and validation data
End of explanation
if sklm is not None:
    # let's try different regularizations
    for c in [1e-5, 1e-1, 1, 10, 100, 1e5]:
        lr = sklm.LogisticRegression(C=c)
        lr.fit(X_train, y_train)
        print ("Logistic Regression (C=%f)\n accuracy = %.3f %%" % (c, 100*lr.score(X_test, y_test)))
Explanation: Purely supervised learning
Let's check how hard the problem is for supervised learning alone.
End of explanation
# Let's first define the theano variables which will represent our data
input_var = T.matrix('inputs')  # for X
target_var = T.ivector('targets')  # for Y
side_var = T.matrix('sideinfo')  # for Z
# Size of the intermediate representation phi(X); 
# since S is 1-dim, phi(X) can also map to a 
# 1-dim vector
representation_dim = 1 
Explanation: Learning with side information: building the pattern
End of explanation
phi = [ (lasagne.layers.DenseLayer, 
         { 'num_units': concarne.patterns.Pattern.PHI_OUTPUT_SHAPE,
         'nonlinearity':None, 'b':None })]
psi = [(lasagne.layers.DenseLayer, 
        { 'num_units': concarne.patterns.Pattern.PSI_OUTPUT_SHAPE, 
        'nonlinearity':lasagne.nonlinearities.softmax, 'b':None })]
beta = [(lasagne.layers.DenseLayer, 
        { 'num_units': concarne.patterns.Pattern.BETA_OUTPUT_SHAPE, 
        'nonlinearity':None, 'b':None })]
Explanation: Now define the functions - we choose linear functions.
concarne internally relies on lasagne which encodes functions as (sets of) layers. Additionally, concarne supports nolearn style initialization of lasagne layers as follows:
End of explanation
pattern = concarne.patterns.MultiViewPattern(
     phi=phi, psi=psi, beta=beta,
     # the following parameters are required to
     # build the functions and the losses
     input_var=input_var, 
     target_var=target_var, 
     side_var=side_var,
     input_shape=input_dim,
     target_shape=num_classes,
     side_shape=side_dim,
     representation_shape=representation_dim,
     # we have to define two loss functions: 
     # 1) the target loss deals with 
     #    optimizing psi and phi wrt. X & Y
     target_loss=lasagne.objectives.categorical_crossentropy,
     # 2) the side loss deals with 
     #    optimizing beta and phi wrt. X & Z,
     #    for multi-view it is beta(Z)~phi(X)
     side_loss=lasagne.objectives.squared_error)
Explanation: For the variable of your layer that denotes the output of the network you should use the markers PHI_OUTPUT_SHAPE,
PSI_OUTPUT_SHAPE and BETA_OUTPUT_SHAPE, so that the pattern can automatically infer the correct shape.
End of explanation
trainer = concarne.training.PatternTrainer(
   pattern,
   procedure='simultaneous',
   num_epochs=500,
   batch_size=10,
   update=lasagne.updates.nesterov_momentum,
   update_learning_rate=0.01,
   update_momentum=0.9,
   )
Explanation: Training
To train a pattern, you can use the PatternTrainer which trains the pattern via stochastic gradient descent.
It also supports different procedures to train the pattern.
End of explanation
trainer.fit_XYZ(X_train, y_train, [Z_train], 
                X_val=X_val, y_val=y_val, 
                side_val=[X_val, Z_val], 
                verbose=True)
pass
Explanation: <b>Let's train!</b>
End of explanation
trainer.score(X_test, y_test, verbose=True)    
pass
Explanation: Some statistics: Test score.
End of explanation
trainer.score_side([X_test, Z_test], verbose=True)    
pass
Explanation: We can also compute a test score for the side loss:
End of explanation
trainer.predict(X_test)
Explanation: You can then also query the prediction output, similar to the scikit-learn API:
End of explanation
<END_TASK>