<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 title
Step1:  Open Dataset and Load Package
This dataset examines the relationship between multitasking and working memory. Link here to original paper by Uncapher et al. 2016.
Step2:  Familiarize Yourself with the Data
Quick review from Data Cleaning
Step3:  Selecting Relevant Variables
Sometimes datasets have many variables that are unnecessary for a given analysis. To simplify your life, and your code, we can select only the given variables we'd like to use for now.
Step4:  Basic Descriptives
Summarizing data
Let's learn how to make simple tables of summary statistics.
First, we will calculate summary info across all data using describe(), a useful function for creating summaries. Note that we're not creating a new object for this summary (i.e. not using the = symbol), so this will print but not save.
Step5:  Grouping Data
Next, we will learn how to group data based on certain variables of interest.
We will use the groupby() function in pandas, which will automatically group any subsequent actions called on the data.
Step6:  We can group data by more than one factor. Let's say we're interested in how levels of adhd interact with groupStatus (multitasking
Step7:  Then we'll check how evenly split these groups are by using groupby() the size() functions
Step8:  Then we'll calculate some summary info about these groups
Step9:  A note on piping / stringing commands together
In R, we often use the pipe %&gt;% to string a series of steps together. We can do the same in Python with many functions in a row.
This is how we're able to take the output of df.groupby(["groupStatus","adhdF"]) and then send that output into the mean() function
Extra
Step10:  How many trials were there per subject?
Step11:  Combine summary statistics with the full dataframe
For some analyses, you might want to add a higher level variable (e.g., subject average hitRate) alongside your long data. We can do this by summarizing the data in a new dataframe and then merging it with the full data.
<ASSISTANT_TASK:>
Python Code:
# load packages we will be using for this lesson
import pandas as pd
Explanation: title: "Data Manipulation in Python"
subtitle: "CU Psych Scientific Computing Workshop"
weight: 1301
tags: ["core", "python"]
Goals of this Lesson
Students will learn:
How to group and categorize data in Python
How to generative descriptive statistics in Python
Links to Files and Video Recording
The files for all tutorials can be downloaded from the Columbia Psychology Scientific Computing GitHub page using these instructions. This particular file is located here: /content/tutorials/python/3-datamanipulation/index.ipynb.
For a video recording of this tutorial from the Fall 2020 workshop, please visit the <a href="/workshop-recording/session3/" target="_blank">Workshop Recording: Session 3</a> page.
End of explanation
# use pd.read_csv to open data into python
df = pd.read_csv("uncapher_2016_repeated_measures_dataset.csv")
Explanation: Open Dataset and Load Package
This dataset examines the relationship between multitasking and working memory. Link here to original paper by Uncapher et al. 2016.
End of explanation
df.head()
df.shape
df.columns
Explanation: Familiarize Yourself with the Data
Quick review from Data Cleaning: take a look at the basic data structure, number of rows and columns.
End of explanation
df = df[["subjNum", "groupStatus", "adhd", "hitRate", "faRate", "dprime"]]
df.head()
Explanation: Selecting Relevant Variables
Sometimes datasets have many variables that are unnecessary for a given analysis. To simplify your life, and your code, we can select only the given variables we'd like to use for now.
End of explanation
df.describe()
Explanation: Basic Descriptives
Summarizing data
Let's learn how to make simple tables of summary statistics.
First, we will calculate summary info across all data using describe(), a useful function for creating summaries. Note that we're not creating a new object for this summary (i.e. not using the = symbol), so this will print but not save.
End of explanation
df.groupby(["groupStatus"]).mean()
Explanation: Grouping Data
Next, we will learn how to group data based on certain variables of interest.
We will use the groupby() function in pandas, which will automatically group any subsequent actions called on the data.
End of explanation
df["adhdF"] = pd.cut(df["adhd"],bins=2,labels=["Low","High"])
Explanation: We can group data by more than one factor. Let's say we're interested in how levels of adhd interact with groupStatus (multitasking: high or low). 
We will first make a factor for adhd (median-split), and add it as a grouping variable using the cut() function in pandas:
End of explanation
df.groupby(["groupStatus","adhdF"]).size()
Explanation: Then we'll check how evenly split these groups are by using groupby() the size() functions:
End of explanation
df.groupby(["groupStatus","adhdF"]).mean()
Explanation: Then we'll calculate some summary info about these groups:
End of explanation
subList = df["subjNum"].unique()
nSubs = len(subList)
nSubs
Explanation: A note on piping / stringing commands together
In R, we often use the pipe %&gt;% to string a series of steps together. We can do the same in Python with many functions in a row.
This is how we're able to take the output of df.groupby(["groupStatus","adhdF"]) and then send that output into the mean() function
Extra: Working with a Long Dataset
This is a repeated measures ("long") dataset, with multiple rows per subject. This makes things a bit tricker, but we are going to show you some tools for how to work with "long" datasets.
How many unique subjects are in the data?
End of explanation
nTrialsPerSubj = df.groupby(["subjNum"]).size().reset_index(name="nTrials")
nTrialsPerSubj.head()
Explanation: How many trials were there per subject?
End of explanation
avgHR = df.groupby(["subjNum"])["hitRate"].mean().reset_index(name="avgHR")
avgHR.head()
df = df.merge(avgHR,on="subjNum")
df.head()
Explanation: Combine summary statistics with the full dataframe
For some analyses, you might want to add a higher level variable (e.g., subject average hitRate) alongside your long data. We can do this by summarizing the data in a new dataframe and then merging it with the full data.
End of explanation
<END_TASK>