<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 False positive and false negatives
This notebook explores the two sources of systematic error that we identify and trim in our datasets.
Step1:  False positives
False positives are defined as algorithms that, for a given gene, infer an outsized number of losses for that orthogroup.
My programs output mean number of taxa that the algorithms inferred to have had lost the orthogroup, and variance of this number for each gene.
It also identifies algorithms that have an outsize number of taxa (2 standard deviations above the mean). These are listed in the outlier column, if they were found.
Step2:  Let's look at distribution of the mean and variance
Step3:  Count number of outliers for each gene
Step4:  Get number of false positives (outliers) for each algorithm
Step5:  False Negatives
False negatives are defined as oversplitting co-ortholog groups. See the paper for an in-depth description.
My programs output a file that, for each gene, says whether or not each algorithm was found to oversplit.
Step6:  Get number of false negatives for each algorithm
Step7:  Combine counts of false-negatives and false-positives for each algorithm
Step8:  Plot counts of errors for each algorithm
Step9:  Proportional error by database
Normalized error counts by database.
<ASSISTANT_TASK:>
Python Code:
%matplotlib inline
from matplotlib import pyplot as plt
import pandas as pd
import numpy as np
Explanation: False positive and false negatives
This notebook explores the two sources of systematic error that we identify and trim in our datasets.
End of explanation
stats2 = pd.read_csv("lossStats_HUMAN.csv",index_col=0)
stats2.fillna({"mean":np.nan,"variance":np.nan,"outliers":0},inplace=True)
stats2.head()
Explanation: False positives
False positives are defined as algorithms that, for a given gene, infer an outsized number of losses for that orthogroup.
My programs output mean number of taxa that the algorithms inferred to have had lost the orthogroup, and variance of this number for each gene.
It also identifies algorithms that have an outsize number of taxa (2 standard deviations above the mean). These are listed in the outlier column, if they were found.
End of explanation
ax = stats2["variance"].hist(bins=50,color='grey')
ax.set_title("Variance histogram, all genes")
ax.set_ylabel("Number of genes")
#plt.savefig("variance_histogram.svg")
stats_outliers = stats2[stats2["outliers"] != 0]
ax = stats_outliers["variance"].hist(bins=50,color='grey')
ax.set_title("Variance, genes with outliers")
ax.set_ylabel("Number of genes")
ax = stats2["mean"].hist(bins=50,color='grey')
ax.set_title("Histogram of mean values, all genes")
ax.set_ylabel("Number of genes")
#plt.savefig("mean_histogram.svg")
ax = stats_outliers["mean"].hist(bins=50,color='grey')
ax.set_title("Mean, genes with outliers")
ax.set_ylabel("Number of genes")
Explanation: Let's look at distribution of the mean and variance
End of explanation
stats2['numOutliers'] = stats2['outliers'].map(lambda x: len(x.split(" ")) if x != 0 else 0)
stats2.head()
stats2["numOutliers"].value_counts()
Explanation: Count number of outliers for each gene
End of explanation
FalsePos = pd.Series([db for row in stats2["outliers"] for db in str(row).split()]).value_counts()
FalsePos = FalsePos[FalsePos.index != '0'] # don't care about these
FalsePos
Explanation: Get number of false positives (outliers) for each algorithm
End of explanation
ldos = pd.read_csv("HUMAN_LDO_results.csv",index_col=0)
ldos.head()
Explanation: False Negatives
False negatives are defined as oversplitting co-ortholog groups. See the paper for an in-depth description.
My programs output a file that, for each gene, says whether or not each algorithm was found to oversplit.
End of explanation
FalseNeg = ldos.apply(pd.value_counts).ix[True]
FalseNeg.sort(ascending=False, inplace=True)
FalseNeg
Explanation: Get number of false negatives for each algorithm
End of explanation
dbs = ["InParanoid","InParanoidCore","OMA_Groups","OMA_Pairs","PANTHER8_LDO","RSD","EggNOG","Orthoinspector",
       "Hieranoid_2","EnsemblCompara_v2","Metaphors","PhylomeDB","PANTHER8_all"]
errors = pd.DataFrame({"FalsePositive":FalsePos,"FalseNegative":FalseNeg})
errors = errors.reindex(dbs)
errors.head()
# errors.to_csv("errors_byDatabase.csv")
Explanation: Combine counts of false-negatives and false-positives for each algorithm
End of explanation
width = .35
fig, ax1 = plt.subplots()
errors["FalseNegative"].plot(kind='bar', ax=ax1, color='grey', width=width, position=1)
ax1.set_ylabel("Number Genes False Negative")
ax2 = ax1.twinx()
errors["FalsePositive"].plot(kind='bar', ax=ax2, color='black', width=width, position=0)
ax2.set_ylabel("Number Genes False Positive")
ax1.yaxis.grid(False)
ax2.yaxis.grid(False)
ax1.xaxis.grid(False)
ax2.xaxis.grid(False)
#plt.savefig("errors_byDatabase.svg")
Explanation: Plot counts of errors for each algorithm
End of explanation
normErrors = errors/errors.sum()
normErrors["sumErrors"] = normErrors["FalseNegative"] + normErrors["FalsePositive"]
normErrors["normSum"] = normErrors["sumErrors"]/normErrors["sumErrors"].sum()
normErrors.sum()
normErrors["normSum"].plot(kind='bar',color='grey')
#plt.savefig("totalErrors.svg")
Explanation: Proportional error by database
Normalized error counts by database.
End of explanation
<END_TASK>