<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Microbiome data manipulation tutorial
This is a jupyter notebook example of how to sort, filter and handle sample metadata
Setup
Step1:  Load the data
we use two datasets
Step2:  Moving pictures dataset. from
Step3:  sorting the samples based on a metadata field (sort_samples)
Sort the samples of the experiment based on the values in the given field.
is the original data sorted by the Subject field?
Step4:  and is the new data sorted?
Step5:  consecutive sorting using different fields
Keeps the order of the previous fields if values for the new field are tied.
For the moving pictures dataset, we want the data to be sorted by individual, and within each individual to be sorted by timepoint
Step6:  filter samples based on metadata field (filter_samples)
Keep only samples matching the values we supply for the selected metadata field.
lets keep only samples from participant F4
Step7:  we can supply a list of values instead of only one value
now lets only keep skin and fecal samples
Step8:  we can also reverse the filtering (removing samples with the supplied values)
We use the negate=True parameter
let's keep just the non-skin and non-feces samples
Step9:  filter low abundance features (filter_abundance)
Remove all features (bacteria) with < 10 reads total (summed over all samples, after normalization).
This is useful for getting rid of non-interesting features. Note that differently from filtering based of fraction of samples where feature is present (filter_prevalence), this method (filter_abundance) will also keep features present in a small fraction of the samples, but in high frequency.
Step10:  Keeping the low abundance bacteria instead
By default, the function removes the low abundance feature. This can be reversed (i.e. keep low abundance features) by using the negate=True parameter)
Step11:  filter non-common bacteria (filter_prevalence)
Remove bacteria based on fraction of the samples where this bacteria is present.
Step12:  Filter bacteria based on the mean frequency over all samples (filter_mean)
Remove bacteria which have a mean (over all samples) lower than the desired threshold.
<ASSISTANT_TASK:>
Python Code:
import calour as ca
ca.set_log_level(11)
%matplotlib notebook
Explanation: Microbiome data manipulation tutorial
This is a jupyter notebook example of how to sort, filter and handle sample metadata
Setup
End of explanation
cfs=ca.read_amplicon('data/chronic-fatigue-syndrome.biom',
                     'data/chronic-fatigue-syndrome.sample.txt',
                     normalize=10000,min_reads=1000)
print(cfs)
Explanation: Load the data
we use two datasets:
the Chronic faitigue syndrome data from:
Giloteaux, L., Goodrich, J.K., Walters, W.A., Levine, S.M., Ley, R.E. and Hanson, M.R., 2016.
Reduced diversity and altered composition of the gut microbiome in individuals with myalgic encephalomyelitis/chronic fatigue syndrome.
Microbiome, 4(1), p.30.
End of explanation
movpic=ca.read_amplicon('data/moving_pic.biom',
                     'data/moving_pic.sample.txt',
                     normalize=10000,min_reads=1000)
print(movpic)
Explanation: Moving pictures dataset. from:
Caporaso, J.G., Lauber, C.L., Costello, E.K., Berg-Lyons, D., Gonzalez, A., Stombaugh, J., Knights, D., Gajer, P., Ravel, J., Fierer, N. and Gordon, J.I., 2011.
Moving pictures of the human microbiome.
Genome biology, 12(5), p.R50.
End of explanation
print(cfs.sample_metadata['Subject'].is_monotonic_increasing)
cfs=cfs.sort_samples('Subject')
Explanation: sorting the samples based on a metadata field (sort_samples)
Sort the samples of the experiment based on the values in the given field.
is the original data sorted by the Subject field?
End of explanation
print(cfs.sample_metadata['Subject'].is_monotonic_increasing)
Explanation: and is the new data sorted?
End of explanation
movpic=movpic.sort_samples('DAYS_SINCE_EXPERIMENT_START')
movpic=movpic.sort_samples('HOST_SUBJECT_ID')
print(movpic.sample_metadata['DAYS_SINCE_EXPERIMENT_START'].is_monotonic_increasing)
print(movpic.sample_metadata['HOST_SUBJECT_ID'].is_monotonic_increasing)
Explanation: consecutive sorting using different fields
Keeps the order of the previous fields if values for the new field are tied.
For the moving pictures dataset, we want the data to be sorted by individual, and within each individual to be sorted by timepoint
End of explanation
tt=movpic.filter_samples('HOST_SUBJECT_ID','F4')
print('* original:\n%s\n\n* filtered:\n%s' % (movpic, tt))
Explanation: filter samples based on metadata field (filter_samples)
Keep only samples matching the values we supply for the selected metadata field.
lets keep only samples from participant F4
End of explanation
print(movpic.sample_metadata['BODY_HABITAT'].unique())
yy=tt.filter_samples('BODY_HABITAT', ['UBERON:skin', 'UBERON:feces'])
print(yy)
Explanation: we can supply a list of values instead of only one value
now lets only keep skin and fecal samples
End of explanation
yy=tt.filter_samples('BODY_HABITAT', ['UBERON:skin', 'UBERON:feces'], negate=True)
print(yy)
Explanation: we can also reverse the filtering (removing samples with the supplied values)
We use the negate=True parameter
let's keep just the non-skin and non-feces samples
End of explanation
tt=cfs.filter_abundance(25)
print('* original:\n%s\n\n* filtered:\n%s' % (cfs, tt))
Explanation: filter low abundance features (filter_abundance)
Remove all features (bacteria) with < 10 reads total (summed over all samples, after normalization).
This is useful for getting rid of non-interesting features. Note that differently from filtering based of fraction of samples where feature is present (filter_prevalence), this method (filter_abundance) will also keep features present in a small fraction of the samples, but in high frequency.
End of explanation
tt=cfs.filter_abundance(25, negate=True)
print('* original:\n%s\n\n* filtered:\n%s' % (cfs,tt))
Explanation: Keeping the low abundance bacteria instead
By default, the function removes the low abundance feature. This can be reversed (i.e. keep low abundance features) by using the negate=True parameter)
End of explanation
# remove bacteria present in less than half of the samples
tt=cfs.filter_prevalence(0.5)
print('* original:\n%s\n\n* filtered:\n%s' % (cfs, tt))
Explanation: filter non-common bacteria (filter_prevalence)
Remove bacteria based on fraction of the samples where this bacteria is present.
End of explanation
# keep only high frequency bacteria (mean over all samples > 1%)
tt=cfs.filter_mean(0.01)
print('* original:\n%s\n\n* filtered:\n%s' % (cfs, tt))
Explanation: Filter bacteria based on the mean frequency over all samples (filter_mean)
Remove bacteria which have a mean (over all samples) lower than the desired threshold.
End of explanation
<END_TASK>