<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Study of connectivity distribution
This notebook is to study the connectivity distrubtion
Step1:  Dependency in how many states are added
Here we see whether the distribution of synaptic influences depends on the state of the vector.
First we start with only two
Step2:  We generate two random patterns and test whether the field (h) is dependent on the initial state. We see very similar normal behavior for both of them
Step3:  We now then try this with 10 patterns to see how the distributions behave
Step4:  We see that the normal distribution is mainted, then we calculate the field h (result of the np.dot(w, s) calculation) for a bunch of different initial random states and concatenate the results to see how the whole distribution looks like
Step5:  Dependence on network size
Now we test test how the histogram looks for different sizes
Step6:  Now we calculate the variance of the h vector as a function of the dimension
<ASSISTANT_TASK:>
Python Code:
from __future__ import print_function
import sys
sys.path.append('../')
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.gridspec as gridspec
import seaborn as sns
from hopfield import Hopfield
%matplotlib inline
sns.set(font_scale=2.0)
prng = np.random.RandomState(seed=100)
normalize = True
T = 1.0
n_store = 7
N_samples = 1000
Explanation: Study of connectivity distribution
This notebook is to study the connectivity distrubtion
End of explanation
n_dim = 400
nn = Hopfield(n_dim=n_dim, T=T, prng=prng)
list_of_patterns = nn.generate_random_patterns(n_dim)
nn.train(list_of_patterns, normalize=normalize)
Explanation: Dependency in how many states are added
Here we see whether the distribution of synaptic influences depends on the state of the vector.
First we start with only two
End of explanation
x = np.dot(nn.w, np.sign(prng.normal(size=n_dim)))
y = np.dot(nn.w, np.sign(prng.normal(size=n_dim)))
fig = plt.figure(figsize=(16, 12))
ax = fig.add_subplot(111)
ax.hist(x)
ax.hist(y)
print(np.std(x))
Explanation: We generate two random patterns and test whether the field (h) is dependent on the initial state. We see very similar normal behavior for both of them
End of explanation
fig = plt.figure(figsize=(16, 12))
ax = fig.add_subplot(111)
for i in range(10):
    x = np.dot(nn.w, np.sign(prng.normal(size=n_dim)))
    ax.hist(x, alpha=0.5)
Explanation: We now then try this with 10 patterns to see how the distributions behave
End of explanation
n_dim = 400
nn = Hopfield(n_dim=n_dim, T=T, prng=prng)
list_of_patterns = nn.generate_random_patterns(n_dim)
nn.train(list_of_patterns, normalize=normalize)
x = np.empty(n_dim)
for i in range(N_samples):
    h = np.dot(nn.w, np.sign(prng.normal(size=n_dim)))
    x = np.concatenate((x, h))
fig = plt.figure(figsize=(16, 12))
ax = fig.add_subplot(111)
n, bins, patches = ax.hist(x, bins=30)
print(np.var(x))
print(nn.sigma)
Explanation: We see that the normal distribution is mainted, then we calculate the field h (result of the np.dot(w, s) calculation) for a bunch of different initial random states and concatenate the results to see how the whole distribution looks like
End of explanation
n_dimensions = [200, 800, 2000, 5000]
fig = plt.figure(figsize=(16, 12))
gs = gridspec.GridSpec(2, 2)
for index, n_dim in enumerate(n_dimensions):
    nn = Hopfield(n_dim=n_dim, T=T, prng=prng)
    list_of_patterns = nn.generate_random_patterns(n_store)
    nn.train(list_of_patterns, normalize=normalize)
    x = np.empty(n_dim)
    for i in range(N_samples):
        h = np.dot(nn.w, np.sign(prng.normal(size=n_dim)))
        x = np.concatenate((x, h))
    
    ax = fig.add_subplot(gs[index//2, index%2])
    ax.set_xlim([-1, 1])
    ax.set_title('n_dim = ' + str(n_dim) + ' std = ' + str(np.std(x)))
    
    weights = np.ones_like(x)/float(len(x))
    n, bins, patches = ax.hist(x, bins=30, weights=weights, normed=False)
Explanation: Dependence on network size
Now we test test how the histogram looks for different sizes
End of explanation
n_dimensions = np.logspace(1, 4, num=20)
variances = []
standar_deviations = []
for index, n_dim in enumerate(n_dimensions):
    print('number', index, 'of', n_dimensions.size, '   n_dim =', n_dim)
    n_dim = int(n_dim)
  
    nn = Hopfield(n_dim=n_dim, T=T, prng=prng)
    list_of_patterns = nn.generate_random_patterns(n_store)
    nn.train(list_of_patterns, normalize=normalize)
    x = np.empty(n_dim)
    for i in range(N_samples):
        h = np.dot(nn.w, np.sign(prng.normal(size=n_dim)))
        x = np.concatenate((x, h))
        
    variances.append(np.var(x))
    standar_deviations.append(np.std(x))
fig = plt.figure(figsize=(16, 12))
ax = fig.add_subplot(111)
ax.semilogx(n_dimensions, variances,'*-', markersize=16, label='var')
ax.semilogx(n_dimensions, standar_deviations, '*-', markersize=16, label='std')
ax.axhline(y=nn.sigma, color='k', label='nn.sigma')
ax.legend()
Explanation: Now we calculate the variance of the h vector as a function of the dimension
End of explanation
<END_TASK>