<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Experiments with Toy (synthetic) dataset
Generate data
$E \sim N(0, \sigma_e^2 I)$
for each $k$
$R_k \sim MN(0, \sigma_r^2 I, I)$
$X_k \sim N(ER_kE^\top, \sigma_x^2)$
and fit the models with this dataset.
Step1:  Split data into training/test data
... but both models consider the input tensor as a full tensor.
Step2:  Fit
Fit $= 1 - A / B$
$A = \sum_{i,k,j}(x_{i,k,j} - \hat{x}_{i,k,j})^2 \quad\quad$ where $\hat{x}$ is a predicted value of triple $(i,k,j)$. This is the sum of squared error.
$B = \sum_{i,k,j}(x_{i,k,j}) \quad\quad\quad\quad$ = Sum of squared datapoints.
$A$ is the objective function of RESCAL
Step3:  Control variance of observed/unobserved data
Placing different variances (var_x) on observed/unobserved points will improve the factorisation performance. (Implicit feedback)
Variance of observed elements (obs_var) = 1
Variance of unboserved elements (unobs_var) = 100 (=almost do not care about unobserved data points)
<ASSISTANT_TASK:>
Python Code:
import numpy as np
import logging
import itertools
from scipy.sparse import csr_matrix
import rescal
from almc.bayesian_rescal import BayesianRescal
%matplotlib inline  
logger = logging.getLogger()
logger.setLevel(logging.INFO)
max_iter = 20
n_entity = 10
n_dim = 5
n_relation = 20
var_e = 1.
var_r = 1.
var_x = 1.
e_mean = np.zeros(n_dim)
r_mean = np.zeros(n_dim**2)
E = np.random.multivariate_normal(e_mean, np.identity(n_dim) * var_e, size = n_entity)
R = np.zeros([n_relation, n_dim, n_dim])
for k in range(n_relation):
    R[k] = np.random.multivariate_normal(r_mean, np.identity(n_dim**2)*var_r).reshape(n_dim,n_dim)
X = np.zeros([n_relation, n_entity, n_entity])
for k in range(n_relation):
    ERET = np.dot(np.dot(E, R[k]), E.T)
    for i,j in itertools.product(range(n_entity), repeat=2):
        X[k,i,j] = np.random.normal(ERET[i,j], var_x)
import itertools
trainX = np.zeros_like(X)
p = 1
for k in range(n_relation):
    for i,j in itertools.product(range(n_entity),repeat=2):
        if X[k, i, j] and np.random.binomial(1, p):
            trainX[k, i, j] = X[k, i, j]
model = BayesianRescal(n_dim, var_e=var_e, var_x=var_x, var_r=var_r)
model.fit(trainX, max_iter=max_iter)
csr_X = list()
for k in range(n_relation):
    csr_X.append(csr_matrix(trainX[k]))
    
E, R, f, itr, exectimes = rescal.rescal_als(csr_X, n_dim)
_X = model._reconstruct()
print("BayesRESCAL:Element-wise squared error: %.3f" %(np.sum((X-_X)**2)))
_X = np.zeros_like(X)
for k in range(n_relation):
    _X[k] = np.dot(np.dot(E, R[k]), E.T)
    
print("RESCAL:Element-wise squared error: %.3f" %(np.sum((X-_X)**2)))
Explanation: Experiments with Toy (synthetic) dataset
Generate data
$E \sim N(0, \sigma_e^2 I)$
for each $k$
$R_k \sim MN(0, \sigma_r^2 I, I)$
$X_k \sim N(ER_kE^\top, \sigma_x^2)$
and fit the models with this dataset.
End of explanation
import itertools
trainX = np.zeros_like(X)
p = 0.5 # proportion of training data points
for k in range(n_relation):
    for i,j in itertools.product(range(n_entity),repeat=2):
        if X[k, i, j] and np.random.binomial(1, p):
            trainX[k, i, j] = X[k, i, j]
            
model = BayesianRescal(n_dim, var_e=var_e, var_x=var_x, var_r=var_r)
model.fit(trainX, max_iter=max_iter)
csr_X = list()
for k in range(n_relation):
    csr_X.append(csr_matrix(trainX[k]))
    
E, R, f, itr, exectimes = rescal.rescal_als(csr_X, n_dim)
_bX = model._reconstruct()
print("BayesRESCAL:Element-wise squared error on training data: %.3f" %(np.sum((trainX-_bX)**2)))
print("BayesRESCAL:Element-wise squared error on test data: %.3f\n" %(np.sum((X-_bX)[trainX==0]**2)))
_X = np.zeros_like(X)
for k in range(n_relation):
    _X[k] = np.dot(np.dot(E, R[k]), E.T)
    
print("RESCAL:Element-wise squared error on training data: %.3f" %(np.sum((trainX-_X)**2)))
print("RESCAL:Element-wise squared error on test data: %.3f" %(np.sum((X-_X)[trainX==0]**2)))
Explanation: Split data into training/test data
... but both models consider the input tensor as a full tensor.
End of explanation
A = np.sum((trainX-_X)**2)
B = np.sum(trainX**2)
fit = 1.-A/B
print(fit)
Explanation: Fit
Fit $= 1 - A / B$
$A = \sum_{i,k,j}(x_{i,k,j} - \hat{x}_{i,k,j})^2 \quad\quad$ where $\hat{x}$ is a predicted value of triple $(i,k,j)$. This is the sum of squared error.
$B = \sum_{i,k,j}(x_{i,k,j}) \quad\quad\quad\quad$ = Sum of squared datapoints.
$A$ is the objective function of RESCAL
End of explanation
model = BayesianRescal(n_dim, var_e=var_e, var_x=var_x, var_r=var_r, controlled_var=True, obs_var=1., unobs_var=100.)
model.fit(trainX, max_iter=20)
_cX = model._reconstruct()
print("BayesRESCAL:Element-wise squared error on training data: %.3f" %(np.sum((trainX[trainX!=0]-_bX[trainX!=0])**2)))
print("BayesRESCAL:Element-wise squared error on test data: %.3f\n" %(np.sum((X-_bX)[trainX==0]**2)))
print("RESCAL:Element-wise squared error on training data: %.3f" %(np.sum((trainX[trainX!=0]-_X[trainX!=0])**2)))
print("RESCAL:Element-wise squared error on test data: %.3f\n" %(np.sum((X-_X)[trainX==0]**2)))
print("CV_BayesRESCAL:Element-wise squared error on training data: %.3f" %(np.sum((trainX[trainX!=0]-_cX[trainX!=0])**2)))
print("CV_BayesRESCAL:Element-wise squared error on test data: %.3f" %(np.sum((X-_cX)[trainX==0]**2)))
Explanation: Control variance of observed/unobserved data
Placing different variances (var_x) on observed/unobserved points will improve the factorisation performance. (Implicit feedback)
Variance of observed elements (obs_var) = 1
Variance of unboserved elements (unobs_var) = 100 (=almost do not care about unobserved data points)
End of explanation
<END_TASK>