<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Deep Belief Nets
Examples covered in the notebook touches on
Step1:  ----------------------------------------------------------------------------------------------------------------------------------------------------------
1. Pretraining
Fitting the data to the deep belief model
----------------------------------------------------------------------------------------------------------------------------------------------------------
Step2:  Visualizing the changes in likelihood for each iteration
Step3:  ----------------------------------------------------------------------------------------------------------------------------------------------------------
2. Extracting Features
Retrieve features created by the unsupervised pretrained DBN outputs.
----------------------------------------------------------------------------------------------------------------------------------------------------------
Step4:  ----------------------------------------------------------------------------------------------------------------------------------------------------------
3. Fine-Tuning
Demonstrates the conversion of a DBN into an MLP and training it w.r.t. to targets
----------------------------------------------------------------------------------------------------------------------------------------------------------
Step5:  Loading the trained MLP
Step6:  Calculating the accuracy of the trained model on the full dataset
Step7:  Demonstrating how the predict function can be used to calculated the classes
<ASSISTANT_TASK:>
Python Code:
import os
os.chdir('~/Codes/DL - Topic Modelling')
from __future__ import print_function, division
import sys
import timeit
from six.moves import cPickle as pickle
import numpy as np
import pandas as pd
import theano
import theano.tensor as T
from lib.deeplearning import deepbeliefnet
# loading the data and transforming it into theano compatible variables
dat_x = np.genfromtxt('data/dtm_20news.csv', dtype='float32', delimiter=',', skip_header = 1)
dat_y = dat_x[:,0]
dat_x = dat_x[:,1:]
vocab =  np.genfromtxt('data/dtm_20news.csv', dtype=str, delimiter=',', max_rows = 1)[1:]
x = theano.shared(dat_x)
y = T.cast(dat_y, dtype='int32')
Explanation: Deep Belief Nets
Examples covered in the notebook touches on:
Pre-training, 
feature extraction,
fine-tuning and 
classification
End of explanation
model = deepbeliefnet(architecture = [2756, 500, 500, 128])
model.pretrain(input = x, pretraining_epochs = 10, output_path = 'params/to_delete')
Explanation: ----------------------------------------------------------------------------------------------------------------------------------------------------------
1. Pretraining
Fitting the data to the deep belief model
----------------------------------------------------------------------------------------------------------------------------------------------------------
End of explanation
import matplotlib.pyplot as plt
%matplotlib inline
plt_dat = np.genfromtxt('params_2000/dbn_params_pretrain/lproxy_layer_2.csv', delimiter=',', names = True)[:20]
plt.plot(plt_dat)
plt.show()
Explanation: Visualizing the changes in likelihood for each iteration
End of explanation
model = deepbeliefnet(architecture = [2756, 500, 500, 128], opt_epochs = [900,5,10],
                      predefined_weights = 'params/dbn_params')
output = model.score(input = x)
colnames = ['bit'] * 128
colnames = [colnames[i] + str(i) for i in range(128)]
colnames.insert(0,'_label_')
pd.DataFrame(data = np.c_[dat_y, output], 
             columns = colnames). \
             to_csv( 'data/dbn_features.csv', index = False)
Explanation: ----------------------------------------------------------------------------------------------------------------------------------------------------------
2. Extracting Features
Retrieve features created by the unsupervised pretrained DBN outputs.
----------------------------------------------------------------------------------------------------------------------------------------------------------
End of explanation
model = deepbeliefnet(architecture = [2756, 500, 500, 128], opt_epochs = [900,5,10], n_outs = 20, predefined_weights = 'params/dbn_params')
#model.train(x=x, y=y,batch_size = 70, training_epochs = 10, output_path = 'params/to_delete')
model.train(x=x, y=y, training_epochs = 10000, learning_rate = (1/70)/2, batch_size = 120,
            drop_out = [0.2, .5, .5, .5], output_path = 'params/to_delete')
Explanation: ----------------------------------------------------------------------------------------------------------------------------------------------------------
3. Fine-Tuning
Demonstrates the conversion of a DBN into an MLP and training it w.r.t. to targets
----------------------------------------------------------------------------------------------------------------------------------------------------------
End of explanation
model = deepbeliefnet(architecture = [2756, 500, 500, 128], n_outs = 20, predefined_weights = 'params/to_delete/trained_dbn.pkl')
Explanation: Loading the trained MLP
End of explanation
sum([1 for i, j in zip(model.predict(x), dat_y) if i == j])/len(dat_y)
Explanation: Calculating the accuracy of the trained model on the full dataset
End of explanation
model.predict(x,prob=True)
Explanation: Demonstrating how the predict function can be used to calculated the classes
End of explanation
<END_TASK>