<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Inclusive B-tagging
Authors
Step1:  Old tagging
https
Step2:  MC
Taggers
Step3:  Taggers combination
We then tested a combination with two calibrations for individual taggers
Step4:  Additional information
Details see in the previous presentation
Step5:  Check calibration of mistag
before calibration
<img src='https
Step6:  Check calibration of mistag
before calibration
<img src='https
Step7:  (PIDNNp &lt; 0.6) &amp; (PIDNNpi &lt; 0.6) &amp; (ghostProb &lt; 0.4)
( (PIDNNk &gt; 0.1) | (PIDNNm &gt; 0.1) | (PIDNNe &gt; 0.1) )
Step8:  (PIDNNpi &lt; 0.6) &amp; (ghostProb &lt; 0.4)
( (PIDNNk &gt; 0.) | (PIDNNm &gt; 0.) | (PIDNNe &gt; 0.) )
Step9:  Preliminary estimation (track
Step10:  Preliminary estimation (track
Step11:  Checks on track
Step12:  This ROC AUC score is similar to the current tagging implementation
Charges asymmetry checks on MC sample
"OS" sample
"SS" sample
Full sample
Means of distributions for MC and data (with signal track and without it)
<ASSISTANT_TASK:>
Python Code:
from IPython.display import Image
import pandas
Explanation: Inclusive B-tagging
Authors:
Tatiana Likhomanenko (contact)
Alexey Rogozhnikov
Denis Derkach
Data (from working group):
real data $B^{\pm} \to J/\psi K^{\pm}$ (RECO 14), 2012
real data $B_d \to J/\psi K^*$ (RECO 14), 2012 (use EPM for assymetry estimation)
Apply sPlot to obtain sWeight ~ P(B)
Monte Carlo:
MC $B^{\pm} \to J/\psi K^{\pm}$ for training 
MC for cross check
$B_d \to J\psi K_s$
$B_d \to J\psi K^*$
End of explanation
pandas.set_option('display.precision', 4)
pandas.read_csv('img/old-tagging-parts.csv').drop(['AUC, with untag', '$\Delta$ AUC, with untag'], axis=1)
Explanation: Old tagging
https://github.com/tata-antares/tagging_LHCb/blob/master/old-tagging.ipynb
We first tested the current algorithm (OS taggers: muon, electron, kaon, vertex). 
TMVA original method was compared with XGBoost.
isotonic symmetric calibration
use different train-test divisions to calculate $D^2$
compute mean and std 
detail see below (the same formulas)
Data
Taggers: electron, muon, kaon and vertex
End of explanation
pandas.set_option('display.precision', 4)
pandas.read_csv('img/old-tagging-parts-MC.csv').drop(['AUC, with untag', '$\Delta$ AUC, with untag'], axis=1)
Explanation: MC
Taggers: electron, muon, kaon and vertex
End of explanation
pandas.set_option('display.precision', 4)
pandas.read_csv('img/old-tagging.csv').drop(['$\Delta$ AUC, with untag'], axis=1)
pandas.set_option('display.precision', 4)
pandas.read_csv('img/old-tagging-MC.csv').drop(['$\Delta$ AUC, with untag'], axis=1)
Explanation: Taggers combination
We then tested a combination with two calibrations for individual taggers: 
isotonic regression 
logistic regression. 
Combination was calibrated using isotonic regression.
End of explanation
pandas.set_option('display.precision', 5)
pandas.read_csv('img/eff_OS.csv').drop(['$\Delta$ AUC, with untag'], axis=1)
Explanation: Additional information
Details see in the previous presentation: https://indico.cern.ch/event/369520/contribution/3/attachments/1178333/1704665/15.10.28.Tagging.pdf
$\epsilon_{tag}$ calculation
$$N (\text{B events, passed selection}) = \sum_{\text{B events, passed selection}} sw_i$$
$$N (\text{all B events}) = \sum_{\text{all B events}} sw_i,$$
where $sw_i$ - sPLot weight
$$\epsilon_{tag} = \frac{N (\text{passed selection})} {N (\text{all events})}$$
$$\Delta\epsilon_{tag} = \frac{\sqrt{N (\text{passed selection})}} {N (\text{all events})}$$
Data for training
data_sw_passed - tracks/vertices with B-sWeight > 1, are used for training
data_sw_not_passed - tracks/vertices with B-sWeight <= 1, are tagged after training
Training
Tracks Features (sig = signal, part = tagger track):
cos_diff_phi = $\cos(\phi^{sig} - \phi^{\rm part})$
diff_pt = $\max(p_T^{part}) - p_T^{part}$
partPt= $p_T^{part}$
max_PID_e_mu = $\max(PIDNN(e), PIDNN(\mu))^{part}$
partP = $p^{part}$
nnkrec = Number of reconstructed vertices
diff_eta = $(\eta^{sig} - \eta^{\rm part})$
EOverP = E/P (from CALO)
sum_PID_k_mu = $\sum\limits_{i\in part}(PIDNN(K)+PIDNN(\mu))$
ptB = $p_T^{sig}$
sum_PID_e_mu = $\sum\limits_{i\in part}(PIDNN(e)+PIDNN(\mu))$
sum_PID_k_e = $\sum\limits_{i\in part}(PIDNN(K)+PIDNN(e))$
proj = $(\vec{p}^{sig},\vec{p}^{part})$
PIDNNe = $PIDNN(e)$
PIDNNk = $PIDNN(K)$
PIDNNm = $PIDNN(\mu)$
phi = $\phi^{part}$
IP = number of IPs in the event
max_PID_k_mu = $max(PIDNN(K)+PIDNN(\mu))$
IPerr = error of IP
IPs = IP/IPerr
veloch = dE/dx track charge from the VELO system
max_PID_k_e  = $max(PIDNN(K)+PIDNN(e))$
diff_phi  = $(\phi^{sig} - \phi^{\rm part})$
ghostProb = ghost probability 
IPPU =  impact parameter with respect to any other reconstructed primary vertex.
eta = pseudorapity of track particle
partlcs = chi2PerDoF for a track 
Vertex Selections
All selection are removed except DaVinci probability cuts
Vertex Features:
mult = multiplicity in the event
nnkrec = number of reconstructed vertices 
ptB = signal B transverse momentum 
vflag = number of tracks in the vertex
ipsmean = mean of tracks IPs
ptmean  = mean pt of the tracks
vcharge = charge of the vertex weigthed by pt
svm = mass of the vertex 
svp = momentum of the vertex
BDphiDir = angle betwen B and vertex
svtau  = lifetime of the vertex
docamax = mean DOCA of the tracks
Classifier
Try to define B sign using track/vertex sign (to define they have the same signs or opposite).
target = signB * signTrack/signVertex > 0
classifier returns 
$$P(\text{track/vertex same sign as B| B sign}) = $$
$$ =P(\text{B same sign as track/vertex| track/vertex sign})$$
2-folding training on the full training sample to use full sample for further analysis (folding scheme provides not overfitted model, details: http://yandex.github.io/rep/metaml.html#module-rep.metaml.folding)
Calibration of $P(\text{track/vertex same sign as B| B sign})$
use 2-folding logistic/isotonic calibration for track/vertex classifier's prediction
compare with isotonic/logistic calibration
compare with absent calibration (bad, have shift predictions)
Computation of $p(B^+)$ using $P(\text{track/vertex same sign as B| B sign})$
Compute $p(B^+)$ using this probabilistic model representation (similar to the previous tagging combination):
$$ \frac{P(B^+)}{P(B^-)} = \prod_{track, vertex} \frac{P(\text{track/vertex}|B^+)} {P(\text{track/vertex} |B^-)} = \alpha
\qquad $$
$$\Rightarrow\qquad P(B^+) = \frac {\alpha}{1+\alpha},   \qquad \qquad    [1] $$
where
$$
\frac{P(B^+)}{P(B^-)} = \prod_{track, vertex} 
\begin{cases}
\frac{P(\text{track/vertex same sign as } B| B)}{P(\text{track/vertex opposite sign as } B| B)}, \text{if track/vertex}^+ \ \
\frac{P(\text{track/vertex opposite sign as } B| B)}{P(\text{track/vertex same sign as } B| B)}, \text{if track/vertex}^- 
\end{cases}
$$
$$p_{mistag} = min(p(B^+), p(B^-))$$
Intermediate estimation $ < D^2 > $ for tracking
Do calibration of $p(B^+)$ and compute $ < D^2 > $ :
use Isotonic calibration (generalization of bins fitting by linear function) - piecewise-constant monotonic function
randomly divide events into two parts (1-train, 2-calibrate)
symmetric isotonic fitting on train and $ < D^2 > $ computation on test
take mean and std for computed $ < D^2 > $
$ < D^2 > $  formula for sample:
$$ < D^2 > =  \frac{\sum_i[2(p^{mistag}_i - 0.5)]^2 * sw_i}{\sum_i sw_i} = $$
$$ =  \frac{\sum_i[2(p_i(B^+) - 0.5)]^2 * sw_i}{\sum_i sw_i}$$
Formula is symmetric and it is not necessary to compute mistag probability
Preliminary estimation
$\epsilon$ calculation
$$\epsilon = < D^2 > * \epsilon_{tag}$$
$$\Delta \epsilon = \sqrt{ \left(\frac{\Delta < D^2 > }{ < D^2 > }\right)^2 + \left(\frac{\Delta \epsilon_{tag} }{\epsilon_{tag}} \right)^2 }$$
Combine track-based and vertex-based tagging using formula [1]
symmetric isotonic calibration on random subsample with $D^2$ calculation
take mean and std for computed $ < D^2 > $
Full estimation of systematic error
set random state
train the best model (track and vertex taggers with 2-folding with fixed random state)
do calibration for track and vertex taggers with 2-folding with fixed random state
compute $p(B^+)$
do calibration with isotonic 2-folding (random state is fixed)
compute $ < D^2 > $ 
This procedure is repeated (from the scratch) for 30 different random states and then we compute mean and std for these 30 values of $ < D^2 > $.
Check calibration of mistag
axis x: predicted mistag probability 
$$p_{mistag} = min(p(B^+), p(B^-))$$
axis y: true mistag probability (computed for bin)
$$p_{mistag} = \frac{N_{wrong}} {N_{wrong} + N_{right}}$$
$$\Delta p_{mistag} = \frac{\sqrt{N_{wrong} N_{right}}} {(N_{wrong} + N_{right})^{1.5}}$$
Stability of calibration
Add random noise after isotonic calibration of $p(B^+)$ for stability:
$$ 0.001 * normal(0, 1)$$
Inclusive tagging (NEW)
Check "OS" and "SS" regions separately (to check that tagging includes "SS" and "OS")
Check dependences on lifetime, lifetime error, number of tracks, momentum, transverse momentum, mass
Asymmetry of charges in events: understanding of high tagging quality or what information we use
Tracking "OS" tagging
https://github.com/tata-antares/tagging_LHCb/blob/master/track-based-tagging-OS.ipynb
Take all possible tracks for all B-events.
Apply:
(IPs > 3) & ((abs(diff_eta) > 0.6) | (abs(diff_phi) > 0.825)) - geometrical cuts
(PIDNNp < 0.5) & (PIDNNpi < 0.5) & (ghostProb < 0.4)
((PIDNNk > trk) | (PIDNNm > trm) | (PIDNNe > tre)), trk=0., trm=0., tre=0.
B mass before sWeight cut
B mass after sWeight cut
Number of tracks in event
PIDNN distributions after selection
<img src='https://raw.githubusercontent.com/tata-antares/tagging_LHCb/master/img/PID_selected_OS.png' style='height: 700px;'/>
Preliminary estimation (track OS + vertex OS)
https://github.com/tata-antares/tagging_LHCb/blob/master/combined-tagging-OS.ipynb
End of explanation
pandas.set_option('display.precision', 5)
pandas.read_csv('img/eff_tracking_SS.csv').drop(['$\Delta$ AUC, with untag'], axis=1)
Explanation: Check calibration of mistag
before calibration
<img src='https://raw.githubusercontent.com/tata-antares/tagging_LHCb/master/img/Bprob_calibration_check_percentile_OS.png' style='height: 600px;'/>
Symmetric isotonic calibration + random noise * 0.001 (noise for stability of bins)
<img src='https://raw.githubusercontent.com/tata-antares/tagging_LHCb/master/img/Bprob_calibration_check_iso_percentile_OS.png' style='height: 600px;'/>
Tracking "SS" tagging
https://github.com/tata-antares/tagging_LHCb/blob/master/track-based-tagging-SS.ipynb
Take all possible tracks for all B-events.
Apply:
(IPs < 3) & (abs(diff_eta) < 0.6) & (abs(diff_phi) < 0.825) & (ghostProb < 0.4)
((PIDNNk > {trk}) | (PIDNNm > {trm}) | (PIDNNe > {tre}) | (PIDNNpi > {trpi}) | (PIDNNp > {trp})), trk=0, trm=0, tre=0, trpi=0, trp=0
B mass before sWeight cut
B mass after sWeight cut
PIDNN distributions after selection
<img src='https://raw.githubusercontent.com/tata-antares/tagging_LHCb/master/img/PID_selected_SS.png' style='height: 700px;'/>
Preliminary estimation (track "SS" only)
https://github.com/tata-antares/tagging_LHCb/blob/master/combined-tagging-SS.ipynb
End of explanation
pandas.set_option('display.precision', 5)
pandas.read_csv('img/new-tagging.csv').drop(['$\Delta$ AUC, with untag'], axis=1)
Explanation: Check calibration of mistag
before calibration
<img src='https://raw.githubusercontent.com/tata-antares/tagging_LHCb/master/img/Bprob_calibration_check_percentile_SS.png' style='height: 600px;'/>
Symmetric isotonic calibration + random noise * 0.001 (noise for stability of bins)
<img src='https://raw.githubusercontent.com/tata-antares/tagging_LHCb/master/img/Bprob_calibration_check_iso_percentile_SS.png' style='height: 600px;'/>
Tracking inclusive tagging
https://github.com/tata-antares/tagging_LHCb/blob/master/track-based-tagging-PID-less.ipynb
Take all possible tracks for all B-events.
Apply:
(ghostProb < 0.4)
((PIDNNk > {trk}) | (PIDNNm > {trm}) | (PIDNNe > {tre}) | (PIDNNpi > {trpi}) | (PIDNNp > {trp})), trk=0, trm=0, tre=0, trpi=0, trp=0
B mass before sWeight cut
B mass after sWeight cut
Number of tracks in event
PIDNN distributions after selection
<img src='https://raw.githubusercontent.com/tata-antares/tagging_LHCb/master/img/PID_selected_less_PID.png' style='height: 700px;'/>
Dependence on PIDNN cuts
(PIDNNp &lt; 0.6) &amp; (PIDNNpi &lt; 0.6) &amp; (ghostProb &lt; 0.4)
( (PIDNNk &gt; 0.7) | (PIDNNm &gt; 0.4) | (PIDNNe &gt; 0.6) )
End of explanation
pandas.set_option('display.precision', 5)
pandas.read_csv('img/new-tagging_relax1.csv')
Explanation: (PIDNNp &lt; 0.6) &amp; (PIDNNpi &lt; 0.6) &amp; (ghostProb &lt; 0.4)
( (PIDNNk &gt; 0.1) | (PIDNNm &gt; 0.1) | (PIDNNe &gt; 0.1) )
End of explanation
pandas.set_option('display.precision', 5)
pandas.read_csv('img/new-tagging_relax2.csv')
Explanation: (PIDNNpi &lt; 0.6) &amp; (ghostProb &lt; 0.4)
( (PIDNNk &gt; 0.) | (PIDNNm &gt; 0.) | (PIDNNe &gt; 0.) )
End of explanation
pandas.set_option('display.precision', 5)
pandas.read_csv('img/new-tagging-PID-less.csv').drop(['$\Delta$ AUC, with untag'], axis=1)
Explanation: Preliminary estimation (track: OS+SS, OS vertex)
https://github.com/tata-antares/tagging_LHCb/blob/master/combined-tagging-PID-less.ipynb
End of explanation
pandas.set_option('display.precision', 5)
pandas.read_csv('img/new-tagging_full_tracks.csv')
Explanation: Preliminary estimation (track: OS+SS, no vertex)
https://github.com/tata-antares/tagging_LHCb/blob/master/track-based-tagging-PID-less.ipynb
End of explanation
pandas.set_option('display.precision', 5)
pandas.read_csv('img/track_signs_assymetry_means.csv', index_col='name')
Explanation: Checks on track: OS+SS, OS vertex model
Check calibration of mistag
for signal (B-like events)
for background
before calibration
<img src='https://raw.githubusercontent.com/tata-antares/tagging_LHCb/master/img/Bprob_calibration_check_percentile_PID_less.png' style='height: 600px;'/>
Symmetric isotonic calibration + random noise * 0.001 (noise for stability of bins)
<img src='https://raw.githubusercontent.com/tata-antares/tagging_LHCb/master/img/Bprob_calibration_check_iso_percentile_PID_less.png' style='height: 600px;'/>
Tagging power dependency on ...
For B mass, B momentum, B transverse momentum, B lifetime use sidebands as bck and peak region as signal:
mask_signal = ((Bmass &gt; 5.27) &amp; (Bmass &lt; 5.3))
mask_bck = ((Bmass &lt; 5.25) | (Bmass &gt; 5.32))
For B lifetime error and number of tracks use sWeights 
Procedure:
* divide variable into 5 percentile bins
* for each bin plot mistag vs true mistag
Signal dependence
<img src='https://raw.githubusercontent.com/tata-antares/tagging_LHCb/master/img/dependence_sig_B_mass.png' style='height: 500px;'/>
<img src='https://raw.githubusercontent.com/tata-antares/tagging_LHCb/master/img/dependence_sig_B_P.png' style='height: 500px;'/>
<img src='https://raw.githubusercontent.com/tata-antares/tagging_LHCb/master/img/dependence_sig_B_Pt.png' style='height: 500px;'/>
<img src='https://raw.githubusercontent.com/tata-antares/tagging_LHCb/master/img/dependence_sig_life_time.png' style='height: 500px;'/>
<img src='https://raw.githubusercontent.com/tata-antares/tagging_LHCb/master/img/dependence_sig_life_time_error.png' style='height: 500px;'/>
<img src='https://raw.githubusercontent.com/tata-antares/tagging_LHCb/master/img/dependence_sig_N_tracks.png' style='height: 500px;'/>
Bck dependence
<img src='https://raw.githubusercontent.com/tata-antares/tagging_LHCb/master/img/dependence_bck_B_mass.png' style='height: 500px;'/>
<img src='https://raw.githubusercontent.com/tata-antares/tagging_LHCb/master/img/dependence_bck_B_P.png' style='height: 500px;'/>
<img src='https://raw.githubusercontent.com/tata-antares/tagging_LHCb/master/img/dependence_bck_B_Pt.png' style='height: 500px;'/>
<img src='https://raw.githubusercontent.com/tata-antares/tagging_LHCb/master/img/dependence_bck_life_time.png' style='height: 500px;'/>
<img src='https://raw.githubusercontent.com/tata-antares/tagging_LHCb/master/img/dependence_bck_life_time_error.png' style='height: 500px;'/>
<img src='https://raw.githubusercontent.com/tata-antares/tagging_LHCb/master/img/dependence_bck_N_tracks.png' style='height: 500px;'/>
Why effective efficiency is so high for this model (model of tracks probability combination to obtain B probability)?
Let's see on the following characteristic of the event:
$$ -\sum_{track} charge_{track}$$
It seems, that for $B^+$ event it should be around +1 + constant (because we exclude signal part)
Regions:
'OS' region: (IP > 3)  & ((abs(diff_eta) > 0.6) | (abs(diff_phi) > 0.825))
'SS' region: (IP < 3)  & (abs(diff_eta) < 0.6) & (abs(diff_phi) < 0.825)
full data
"OS" data
"SS" data
Full sample
Add signal track
"OS" data
"SS" data
Full sample
Means of distributions (with signal track and without it)
End of explanation
pandas.set_option('display.precision', 5)
pandas.concat([pandas.read_csv('img/track_signs_assymetry_means.csv', index_col='name'),
               pandas.read_csv('img/track_signs_assymetry_means_mc.csv', index_col='name')])
Explanation: This ROC AUC score is similar to the current tagging implementation
Charges asymmetry checks on MC sample
"OS" sample
"SS" sample
Full sample
Means of distributions for MC and data (with signal track and without it)
End of explanation
<END_TASK>