<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Experiment
Step1:  Load and check data
Step2:  ## Analysis
Experiment Details
Step3:  Results
<ASSISTANT_TASK:>
Python Code:
%load_ext autoreload
%autoreload 2
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import os
import glob
import tabulate
import pprint
import click
import numpy as np
import pandas as pd
from ray.tune.commands import *
from nupic.research.frameworks.dynamic_sparse.common.browser import *
import matplotlib
import matplotlib.pyplot as plt
from matplotlib import rcParams
%config InlineBackend.figure_format = 'retina'
import seaborn as sns
sns.set(style="whitegrid")
sns.set_palette("colorblind")
Explanation: Experiment:
Replicate how so dense experiments using the new dynamic sparse framework. Compare results with published in the paper
Motivation.
Ensure our code has no known bugs before proceeding with further experimentation.
Ensure How so Dense experiments are replicable
Conclusion
End of explanation
# exps = ['replicate_hsd_test2']
# exps = ['replicate_hsd_debug1']
# exps = ['replicate_hsd_debug3_8x']
exps = ['replicate_hsd_debug6_8x']
paths = [os.path.expanduser("~/nta/results/{}".format(e)) for e in exps]
df = load_many(paths)
df.head(5)
# replace hebbian prine
df['hebbian_prune_perc'] = df['hebbian_prune_perc'].replace(np.nan, 0.0, regex=True)
df['weight_prune_perc'] = df['weight_prune_perc'].replace(np.nan, 0.0, regex=True)
df.columns
df.shape
df.iloc[1]
df.groupby('model')['model'].count()
Explanation: Load and check data
End of explanation
num_epochs = 25
# Did any  trials failed?
df[df["epochs"]<num_epochs]["epochs"].count()
# Removing failed or incomplete trials
df_origin = df.copy()
df = df_origin[df_origin["epochs"]>=num_epochs]
df.shape
# which ones failed?
# failed, or still ongoing?
df_origin['failed'] = df_origin["epochs"]<num_epochs
df_origin[df_origin['failed']]['epochs']
# helper functions
def mean_and_std(s):
    return "{:.2f} ± {:.2f}".format(s.mean()*100, s.std()*100)
def round_mean(s):
    return "{:.0f}".format(round(s.mean()))
stats = ['min', 'max', 'mean', 'std']
def agg(columns, filter=None, round=3):
    if filter is None:
        return (df.groupby(columns)
             .agg({'val_acc_max_epoch': round_mean,
                   'val_acc_max': stats,
                   'val_acc_last': stats,
                   'model': ['count']})).round(round)
    else:
        return (df[filter].groupby(columns)
             .agg({'val_acc_max_epoch': round_mean,
                   'val_acc_max': stats,                
                   'val_acc_last': stats,
                   'model': ['count']})).round(round)
    
    
def agg_paper(columns, filter=None, round=3):
    if filter is None:
        return (df.groupby(columns)
             .agg({'val_acc_max': mean_and_std,
                   'val_acc_last': mean_and_std,
                   'train_acc_last': mean_and_std,
                   'model': ['count']})).round(round)
    else:
        return (df[filter].groupby(columns)
             .agg({'val_acc_max': mean_and_std,
                   'val_acc_last': mean_and_std,
                   'train_acc_last': mean_and_std,
                   'model': ['count']})).round(round)
Explanation: ## Analysis
Experiment Details
End of explanation
agg(['model'])
agg_paper(['model'])
Explanation: Results
End of explanation
<END_TASK>