<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Goal
Step1:  Step 1) Load the data from bicorr1
Step2:  To remind ourselves what this file contains, the columns are
Step3:  I used a numpy array. That's kind of a shame. If I had used a pandas array, I could easily add new colums with energies, but oh well. Moving on.
Skipping step 2 to keep this notebook in line with build_bicorr_hist_master.ipynb.
Step 3) Preallocate bhm_e matrix
Follow the method in build_bicorr_hist_master.
Step4:  Interaction type bins
Step5:  Detector pair bins
Step6:  Preallocate matrix
Step7:  How large when stored to disk?
Step8:  This is pretty small. Good. I could even avoid converting it to and from sparse matrix at this size.
Functionalize this
Step9:  Step 4) Fill the histogram
Now I'm going to use the final method from build_bhm to fill bhm_e element by element.
I will need to add one more step, which is to retrieve the distance from each detector to the fission chamber for use in calculating the energy to each detector. Make it happen. 
Set up a dictionary for retrieving detector distance
I want to call dict_dist(det_num) and have it return the detector distance in m.
First, load the file with detector distances.
Step10:  Set up dictionaries for returning pair and type indices
Step11:  Calculate energy for one event
Step12:  These are pretty close together. Now convert those to energy using the time stamps. Only proceed when both time stamps are greater than 0.
Step13:  Set up info for filling the histogram
Step14:  Only proceed if both particles are neutrons AND both times are greater than 0. How do I implement this logic?
Step15:  The tricky thing here is that np.logical_and looks at elements 0 of both input arrays as one pair, then elements 1, etc. I had originally implemented it with the assumption that it looked at each input array as a pair. Thus, the split implementation.
Implement tqdm status bar.
Step16:  Functionalize it
Step17:  Skipping step 5. I am not going to convert to sparse matrix because the file size will be small anyway.
Step18:  Step 6) Save the histogram and related vectors to disk
What do I need to save? Mostly the same stuff but in energy units.
Step19:  Step 7) Reload from disk
Step20:  Functionalize for many folders
I need to pull the bicorr files from many folders and produce bhm_e along with bhm.
If I were going to reproduce bhm from the beginning, I would modify build_bhm to include another line of generating bhm_e. In this case, though, I am only going to produce bhm_e so I will write a separate function.
<ASSISTANT_TASK:>
Python Code:
import matplotlib.pyplot as plt
import matplotlib.colors
import numpy as np
import os
import scipy.io as sio
import sys
import time
import inspect
import pandas as pd
from tqdm import *
# Plot entire array
np.set_printoptions(threshold=np.nan)
import seaborn as sns
sns.set_palette('spectral')
%load_ext autoreload
%autoreload 2
sys.path.append('../scripts/')
import bicorr as bicorr
import bicorr_plot as bicorr_plot
import bicorr_e as bicorr_e
Explanation: Goal: Build bhm in energy space
The problem here is that I can't convert directly from bhm in time to energy space because then the energy bins are different (17% range of variation) due to different distances from the fission chamber. Thus, I need to go back to the original construction of the bicorr_hist_master and create two versions: one in time, and one in energy. 
This is probably pretty hasty, but I am going to only create this for $nn$ events. Why you might ask? Because.
I'm choosing to not modify the original notebook build_bicorr_hist_master because... just because. I will work with the same methods here. 
I will work with the data in the fnpc &gt; datar folder.
End of explanation
os.listdir('../datar/1')
with open('../datar/1/bicorr1_part') as f:
    print(f.read())
Explanation: Step 1) Load the data from bicorr1
End of explanation
bicorr_data = bicorr.load_bicorr(1, root_path = '../datar')
type(bicorr_data)
Explanation: To remind ourselves what this file contains, the columns are:
col 1) Event number
col 2) d1ch
col 3) d1 particle type
col 4) d1 $\Delta t_1$
col 5) d2ch
col 6) d2 particle type
col 7) d2 $\Delta t_2$
From this I need to calculate the energies. I don't really want to regenerate the bicorr file, or even the bhm file. I need a separate function that will take the bicorr file and generate a bhm_e distribution.
End of explanation
help(bicorr_e.build_energy_bin_edges)
e_bin_edges, num_e_bins = bicorr_e.build_energy_bin_edges()
print(e_bin_edges)
print(num_e_bins)
Explanation: I used a numpy array. That's kind of a shame. If I had used a pandas array, I could easily add new colums with energies, but oh well. Moving on.
Skipping step 2 to keep this notebook in line with build_bicorr_hist_master.ipynb.
Step 3) Preallocate bhm_e matrix
Follow the method in build_bicorr_hist_master.
End of explanation
# Number of bins in interaction type
num_intn_types = 1 #(0=nn, 1=np, 2=pn, 3=pp), only going to use nn
Explanation: Interaction type bins
End of explanation
# What are the unique detector numbers? Use same technique as in bicorr.py
chList, fcList, detList, num_dets, num_det_pairs = bicorr.build_ch_lists(print_flag=True)
Explanation: Detector pair bins
End of explanation
bhm_e = np.zeros((num_det_pairs,num_intn_types,num_e_bins,num_e_bins),dtype=np.uint32)
bhm_e.shape
Explanation: Preallocate matrix
End of explanation
bhm_e.nbytes/1e9
Explanation: How large when stored to disk?
End of explanation
help(bicorr_e.alloc_bhm_e)
bhm_e = bicorr_e.alloc_bhm_e(num_det_pairs, num_intn_types, num_e_bins)
bhm_e.shape
Explanation: This is pretty small. Good. I could even avoid converting it to and from sparse matrix at this size.
Functionalize this
End of explanation
dict_det_dist = bicorr_e.build_dict_det_dist()
dict_det_dist
dict_det_dist[45]
Explanation: Step 4) Fill the histogram
Now I'm going to use the final method from build_bhm to fill bhm_e element by element.
I will need to add one more step, which is to retrieve the distance from each detector to the fission chamber for use in calculating the energy to each detector. Make it happen. 
Set up a dictionary for retrieving detector distance
I want to call dict_dist(det_num) and have it return the detector distance in m.
First, load the file with detector distances.
End of explanation
# Set up dictionary for returning detector pair index
det_df = bicorr.load_det_df()
dict_pair_to_index, dict_index_to_pair, dict_pair_to_angle = bicorr.build_dict_det_pair(det_df)
print(det_df)
print(dict_pair_to_index)
print(dict_index_to_pair)
print(dict_pair_to_angle)
# Type index
dict_type_to_index = {11:0, 12:1, 21:2, 22:3}
Explanation: Set up dictionaries for returning pair and type indices
End of explanation
i = 3
bicorr_data[i]
det1dist = dict_det_dist[bicorr_data[i]['det1ch']]
det2dist = dict_det_dist[bicorr_data[i]['det2ch']]
print(det1dist,det2dist)
Explanation: Calculate energy for one event
End of explanation
det1t = bicorr_data[i]['det1t']
det2t = bicorr_data[i]['det2t']
print(det1t, det2t)
det1e = bicorr.convert_time_to_energy(det1t, det1dist)
det2e = bicorr.convert_time_to_energy(det2t, det2dist)
print(det1e,det2e)
Explanation: These are pretty close together. Now convert those to energy using the time stamps. Only proceed when both time stamps are greater than 0.
End of explanation
e_min = np.min(e_bin_edges); e_max = np.max(e_bin_edges)
e_step = e_bin_edges[1]-e_bin_edges[0]
Explanation: Set up info for filling the histogram
End of explanation
i = 16
event = bicorr_data[i]
det1t = event['det1t']; det2t = event['det2t'];
print(event, det1t, det2t, event['det1par'], event['det2par'])
np.logical_and([det1t > 0, event['det1par'] == 1], [det2t>0, event['det2par'] == 1])
Explanation: Only proceed if both particles are neutrons AND both times are greater than 0. How do I implement this logic?
End of explanation
for i in tqdm(np.arange(bicorr_data.shape[0]),ascii=True,disable=False):
    event = bicorr_data[i]
    det1t = event['det1t']; det2t = event['det2t'];
    
    logic = np.logical_and([det1t > 0, event['det1par'] == 1], [det2t>0, event['det2par'] == 1])
    
    if np.logical_and(logic[0],logic[1]): # nn with both t > 0
        det1dist = dict_det_dist[event['det1ch']]
        det2dist = dict_det_dist[event['det2ch']]
        
        det1e = bicorr.convert_time_to_energy(det1t, det1dist)
        det2e = bicorr.convert_time_to_energy(det2t, det2dist)
        
        # Check that they are in range of the histogram        
        if np.logical_and(e_min < det1e < e_max, e_min < det2e < e_max):
            # Determine index of detector pairs
            pair_i = dict_pair_to_index[event['det1ch']*100+event['det2ch']]
        
            # Determine indices of energy values
            e1_i = int(np.floor((det1e-e_min)/e_step))
            e2_i = int(np.floor((det2e-e_min)/e_step))
            
            # Increment bhm_e
            pair_i = dict_pair_to_index[event['det1ch']*100+event['det2ch']]
            bhm_e[pair_i,0,e1_i,e2_i] += 1
Explanation: The tricky thing here is that np.logical_and looks at elements 0 of both input arrays as one pair, then elements 1, etc. I had originally implemented it with the assumption that it looked at each input array as a pair. Thus, the split implementation.
Implement tqdm status bar.
End of explanation
import inspect
print(inspect.getsource(bicorr_e.fill_bhm_e))
bhm_e = bicorr_e.alloc_bhm_e(num_det_pairs, num_intn_types, num_e_bins)
bhm_e = bicorr_e.fill_bhm_e(bhm_e, bicorr_data, det_df, dict_det_dist, e_bin_edges, disable_tqdm = False)
Explanation: Functionalize it
End of explanation
bhm_e.shape
Explanation: Skipping step 5. I am not going to convert to sparse matrix because the file size will be small anyway.
End of explanation
save_filename = r'../datar/1/bhm_e'
note = 'Here is my note'
np.savez(save_filename, bhm_e = bhm_e, e_bin_edges=e_bin_edges, note = note)
bicorr_e.save_bhm_e(bhm_e, e_bin_edges, r'../datar/1/')
Explanation: Step 6) Save the histogram and related vectors to disk
What do I need to save? Mostly the same stuff but in energy units.
End of explanation
load_filename = r'../datar/1/bhm_e.npz'
bhm_e = np.load(load_filename)['bhm_e']
e_bin_edges = np.load(load_filename)['e_bin_edges']
note = np.load(load_filename)['note']
print(bhm_e.shape)
print(e_bin_edges.shape)
print(note)
bhm_e, e_bin_edges, note = bicorr_e.load_bhm_e(r'../datar/1/')
help(bicorr_e.save_bhm_e)
Explanation: Step 7) Reload from disk
End of explanation
help(bicorr_e.build_bhm_e)
bhm_e, e_bin_edges = bicorr_e.build_bhm_e(1,3,root_path = '../datar/')
help(bicorr_e.load_bhm_e)
bhm_e, e_bin_edges, note = bicorr_e.load_bhm_e()
note
Explanation: Functionalize for many folders
I need to pull the bicorr files from many folders and produce bhm_e along with bhm.
If I were going to reproduce bhm from the beginning, I would modify build_bhm to include another line of generating bhm_e. In this case, though, I am only going to produce bhm_e so I will write a separate function.
End of explanation
<END_TASK>