<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Demonstration of SMPS Calculations
The SMPS calculations require two main packages from atmPy - smps and dma.  dma contains the DMA class and its children.  The children of DMA simply contain the definition of the dimensions of the DMA used in the current SMPS instance.  In this case, we use the definition of the NOAA wide DMA, which has the dimensions $r_i = 0.0312$, $r_o = 0.03613$ and $l = 0.34054$ where all units are in meters.  The SMPS object provides a set of utilities for taking the scan data, applying a transfer function and correcting the distribution for multiple charges to produce a size distribution.
The import of the sizedistribution package allows us to manipulate the output of the DMA ($dN/d\log{D_p}$) such that we can pull out other representations of the size distribution.  The remaining packages are simply used for data manipulation.
Step1:  The first thing we do in the analysis is we create a new SMPS object with the DMA instance we wish to use.  Here, we also set the initial directory to search for SMPS data.  When a new SMPS object is created, an open file dialog window will be produced and the user may select one or many files to analyze.  The file names will be stored in the SMPS attribute files.
Step2:  Determining the Lag
In order to properly analyze a scan, we must first align the data such that the particle concentrations are consistent with the conditions in the DMA.  Although conditions such as voltages and flows adjust to changes almost immediately, there will be a lag in the particle response due to isntrument residence time.  We can determine the lag either through a correlation or heuristically.  The SMPS provides a function getLag which takes an integer indicating the file index to select in the attribute files containing the array of files selected by the user.  The optional input, delta, allows the user to offset the result of the correlation by some amount to provide a more reasonable estimate of the lag.
The SMPS
Step3:  Processing the Files
Once the user has pointed to the files they wish to use, they can begin processing of the files using the function SMPS
Step4:  Charge Correction
The SMPS scans mobilities which are a function of voltage.  In truth, at each voltage setpoint, the DMA allows a range of particle mobilities through the instrument.  This range is expressed by the DMA transfer function as 
\begin{equation}
\Omega = 1/q_a\max{\left[q_a,q_s,\left[\frac{1}{2}\left(q_a+q_s\right)-\right]\right]}
\end{equation}
\begin{equation}
Z=\frac{neCc(D_p)}{3\pi\mu(T)*D_p}
\end{equation}
In any charge correction, we must assume that there are no particles beyond the topmost bin.  This allows us to make the assumption that all particles in that bin are singly charged.  In order to determine the total number of particles in the current bin, we can simply use Wiedensohler's equation for the charging efficiency for singly charged particles.  Starting at the topmost bin, we can calculate the total number of particles as 
\begin{equation}
\frac{N_1(D_p,i)}{f_1}=N(D_p)
\end{equation}
where $N_1$ is the number of particles of size $D_p$ having 1 charge, $f_1$ is the charging efficiency for singly charged particles and $N(D_p)$ is the total number of particles diameter $D_p$.  Using the initial number of particles, we can then calculated the number of multiply charged particles in a similar fashion.  
Once these numbers have been calculated, we can determine the location of the multiply charged particles (i.e. the diameter bin with which they have been identified).  To do 
Finally, to get the total number of particles in the bin, we can apply the sum
\begin{equation}
N(D_p) = \frac{N_1(D_p)}{f_1}\sum_{i=0}^\inf{f_i}
\end{equation}
However, in each of these cases, only a finite number of particles may be available in each bin, so in the code, we will have to take the minimum of the following
Step5:  Use of the SizeDistr Object
The sizeditribution package contains some classes and routines for ready manipulation of the data.  But first, we will need to convert the data of interest to a PANDAS data frame with the time as index.
Step6:  In addition, we will need to convert the bin centers produced by the SMPS object to bin edges.  To do this, we will make the simple assumption that the bin edges are just the halfway points between the centers.  For the edge cases, we will simply take the difference between the smallest bin center and the halfway point between the first and second bin centers and subtract this value from the smallest diameter.  Similarly, for the largest diameter, we will take the difference between the halfway point between the largest and second largest bin centers and the largest bin center and add it to the largest bin center.
Step7:  Once we have the corresponding SizeDistr object, we can now change the current distribution which is in $dN/d\log D_p$ space and change this to a surface area distribution in log space.  This will produce a new object that we will call sfSD.
Step8:  To get an overall view, we can further manipulate the data to produce average distributions from the entire time series.
<ASSISTANT_TASK:>
Python Code:
from atmPy.instruments.DMA import smps
from atmPy.instruments.DMA import dma
from matplotlib import colors
import matplotlib.pyplot as plt
from numpy import meshgrid
import numpy as np
import pandas as pd
from matplotlib.dates import date2num
from matplotlib import dates
from atmPy import sizedistribution as sd
%matplotlib inline
Explanation: Demonstration of SMPS Calculations
The SMPS calculations require two main packages from atmPy - smps and dma.  dma contains the DMA class and its children.  The children of DMA simply contain the definition of the dimensions of the DMA used in the current SMPS instance.  In this case, we use the definition of the NOAA wide DMA, which has the dimensions $r_i = 0.0312$, $r_o = 0.03613$ and $l = 0.34054$ where all units are in meters.  The SMPS object provides a set of utilities for taking the scan data, applying a transfer function and correcting the distribution for multiple charges to produce a size distribution.
The import of the sizedistribution package allows us to manipulate the output of the DMA ($dN/d\log{D_p}$) such that we can pull out other representations of the size distribution.  The remaining packages are simply used for data manipulation.
End of explanation
hagis = smps.SMPS(dma.NoaaWide(),scan_folder="C:/Users/mrichardson/Documents/HAGIS/SMPS/Scans")
Explanation: The first thing we do in the analysis is we create a new SMPS object with the DMA instance we wish to use.  Here, we also set the initial directory to search for SMPS data.  When a new SMPS object is created, an open file dialog window will be produced and the user may select one or many files to analyze.  The file names will be stored in the SMPS attribute files.
End of explanation
hagis.getLag(10, delta=10)
Explanation: Determining the Lag
In order to properly analyze a scan, we must first align the data such that the particle concentrations are consistent with the conditions in the DMA.  Although conditions such as voltages and flows adjust to changes almost immediately, there will be a lag in the particle response due to isntrument residence time.  We can determine the lag either through a correlation or heuristically.  The SMPS provides a function getLag which takes an integer indicating the file index to select in the attribute files containing the array of files selected by the user.  The optional input, delta, allows the user to offset the result of the correlation by some amount to provide a more reasonable estimate of the lag.
The SMPS::getLag method will produce two plots.  The first is the results from the attempted correlation and the second shows how the two scans align with the lag estimate, both the smoothed and raw data. This method will set the lag attribute in the SMPS instance which will be used in future calculations.  This attribute is directly accessible if the user wishes to adjust it.
End of explanation
hagis.lag = 10
hagis.proc_files()
Explanation: Processing the Files
Once the user has pointed to the files they wish to use, they can begin processing of the files using the function SMPS::procFiles().  Each file is processed as follows:
The raw data concerning the conditions is truncated for both the up and down scans to the beginning and end of the respective scans.  The important parameters here are the values $t_{scan}$ and $t_{dwell}$ from the header of the files.  The range of the data from the upscan spans the indices 0 to $t_{scan}$ and the range for the down data is $t_{scan}+t_{dwell}$ to $2\times t_{scan}+t_{dwell}$.
The CN data is adjusted based on the lag.  This data is truncated for the up scan as $t_{lag}$ to $t_{lag} + t_{scan}$ where $t_{lag}$ is the lag time determined by the user (possibly with the function SMPS::getLag().  In the downward scan, the data array is reversed and the data is truncated to the range $t_{dwell}-t_{lag}$ to $t_{scan}+t_{dwell}-t_{lag}$.  In all cases, the CN concentration is calculated from the 1 second buffer and the CPC flow rate as $N_{1 s}/Q_{cpc}$.
The truncated [CN] is then smoothed using a Lowess smoothing function for both the up and down data.
Diameters for each of the corresponding [CN] are then calculated from the set point voltage (rather than the measured voltage).
The resulting diameters and smoothed [CN] are then run through a transfer function.  In this case, the transfer function is a simple full width half max based off of the mobility range of the current voltage.  This function allows us to produce a $d\log D_p$ for the ditribution.
The distribution is then corrected based on the algorithm described below.
The charge corrected distribution is then converted to a logarithmic distribution using the values from the FWHM function.
The resulting distribution is then interpolated onto a logarithmically distributed array that consists of bin ranging from 1 to 1000 nm.
End of explanation
hagis.date
index = []
for i,e in enumerate(hagis.date):
    if e is None:
        index.append(i)
        
print(index)
if index:
    hagis.date = np.delete(hagis.date, index)
    hagis.dn_interp = np.delete(hagis.dn_interp,index, axis=0)   
xfmt = dates.DateFormatter('%m/%d %H:%M')
xi = date2num(hagis.date)
XI, YI = meshgrid(xi, hagis.diam_interp)
#XI = dates.datetime.datetime.fromtimestamp(XI)
Z = hagis.dn_interp.transpose()
Z[np.where(Z <= 0)] = np.nan
pmax = 1e6 # 10**np.ceil(np.log10(np.amax(Z[np.where(Z > 0)])))
pmin = 1 #10**np.floor(np.log10(np.amin(Z[np.where(Z > 0)])))
fig, ax = plt.subplots()
pc = ax.pcolor(XI, YI, Z, cmap=plt.cm.jet, norm=colors.LogNorm(pmin, pmax, clip=False), alpha=0.8)
plt.colorbar(pc)
plt.yscale('log')
plt.ylim(5, 1000)
ax.xaxis.set_major_formatter(xfmt)
fig.autofmt_xdate()
fig.tight_layout()
Explanation: Charge Correction
The SMPS scans mobilities which are a function of voltage.  In truth, at each voltage setpoint, the DMA allows a range of particle mobilities through the instrument.  This range is expressed by the DMA transfer function as 
\begin{equation}
\Omega = 1/q_a\max{\left[q_a,q_s,\left[\frac{1}{2}\left(q_a+q_s\right)-\right]\right]}
\end{equation}
\begin{equation}
Z=\frac{neCc(D_p)}{3\pi\mu(T)*D_p}
\end{equation}
In any charge correction, we must assume that there are no particles beyond the topmost bin.  This allows us to make the assumption that all particles in that bin are singly charged.  In order to determine the total number of particles in the current bin, we can simply use Wiedensohler's equation for the charging efficiency for singly charged particles.  Starting at the topmost bin, we can calculate the total number of particles as 
\begin{equation}
\frac{N_1(D_p,i)}{f_1}=N(D_p)
\end{equation}
where $N_1$ is the number of particles of size $D_p$ having 1 charge, $f_1$ is the charging efficiency for singly charged particles and $N(D_p)$ is the total number of particles diameter $D_p$.  Using the initial number of particles, we can then calculated the number of multiply charged particles in a similar fashion.  
Once these numbers have been calculated, we can determine the location of the multiply charged particles (i.e. the diameter bin with which they have been identified).  To do 
Finally, to get the total number of particles in the bin, we can apply the sum
\begin{equation}
N(D_p) = \frac{N_1(D_p)}{f_1}\sum_{i=0}^\inf{f_i}
\end{equation}
However, in each of these cases, only a finite number of particles may be available in each bin, so in the code, we will have to take the minimum of the following:
\begin{equation}
\delta{N(k)}=\min{\left(\frac{f_iN_1}{f_1},N_k\right)}
\end{equation}
where $\delta{N(k)}$ is the number of particles to remove from bin $k$ and $N_k$ is the number of particles in bin $k$.
Output
In the following, we take the results from the SMPS::procFiles() method and produce a color map of size distributions in $dN/d\log D_p$ space as a function of time.  The attribute date from the instance of SMPS is a set of DateTime for each scan based on the start time of the file and the scan time collected from the header.
End of explanation
dataframe = pd.DataFrame(hagis.dn_interp)
dataframe.index = hagis.date
Explanation: Use of the SizeDistr Object
The sizeditribution package contains some classes and routines for ready manipulation of the data.  But first, we will need to convert the data of interest to a PANDAS data frame with the time as index.
End of explanation
binedges = (hagis.diam_interp[1:]+hagis.diam_interp[:-1])/2
first = hagis.diam_interp[0] -(binedges[0]-hagis.diam_interp[0])
last = hagis.diam_interp[-1]+ (hagis.diam_interp[-1]-binedges[-1])
binedges = np.append([first],binedges)
binedges=np.append(binedges,[last])
sizeDistr = sd.SizeDist_TS(dataframe,binedges, 'dNdlogDp')
f,a,b,c = sizeDistr.plot(vmax = pmax, vmin = pmin, norm='log', showMinorTickLabels=False, cmap=plt.cm.jet)
a.set_ylim((5,1000))
Explanation: In addition, we will need to convert the bin centers produced by the SMPS object to bin edges.  To do this, we will make the simple assumption that the bin edges are just the halfway points between the centers.  For the edge cases, we will simply take the difference between the smallest bin center and the halfway point between the first and second bin centers and subtract this value from the smallest diameter.  Similarly, for the largest diameter, we will take the difference between the halfway point between the largest and second largest bin centers and the largest bin center and add it to the largest bin center.
End of explanation
sfSD = sizeDistr.convert2dSdlogDp()
from imp import reload
reload(sd)
f,a,b,c = sfSD.plot(vmax = 1e10, vmin = 1e4, norm='log', showMinorTickLabels=False,removeTickLabels=['200','300','400',] ,cmap =plt.cm.jet)
a.set_ylim((5,1000))
Explanation: Once we have the corresponding SizeDistr object, we can now change the current distribution which is in $dN/d\log D_p$ space and change this to a surface area distribution in log space.  This will produce a new object that we will call sfSD.
End of explanation
avgAt = sizeDistr.average_overAllTime()
f,a = avgAt.plot(norm='log')
# a.set_yscale('log')
avgAtS = sfSD.average_overAllTime()
f,a= avgAtS.plot(norm='log')
a.set_yscale('log')
Explanation: To get an overall view, we can further manipulate the data to produce average distributions from the entire time series.
End of explanation
<END_TASK>