<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 W261 course notes
Week 1
Step1:  When asking the professor a question, use the STAR approach
Step2:  The wait command forces the system to finish processing the child process before the next command is run.
Step3:  Parallel grep (my attempt)
<ASSISTANT_TASK:>
Python Code:
# Use this for interactive plots
%matplotlib notebook
import matplotlib.pyplot as plt
import pandas as pd
pd.Series([1,2,3,4]).plot()
Explanation: W261 course notes
Week 1: 8/31/2016
Reading
Doing Data Science (ch 1 and 2)
* “Datify” all the things to maximize value
* Data science pipeline
The human face of big data: Rick Smolan
* Big data is like opening 1000s of more eyes. You can see dimensions you couldn’t see before. 
* Data that is garbage to some people are gold to others
Introduction to Information Retrieval - Ch 13
Model selection
* Bias-variance tradeoff. Bias = consistently get a prediction wrong. Variance = inconsistently make a prediction.
End of explanation
!grep Guido data/week1/LICENSE.txt
!cat data/week1/LICENSE.txt data/week1/LICENSE.txt | wc -w
!cat data/week1/LICENSE.txt data/week1/LICENSE.txt | head
%%bash
for term in Python Guido Scala license
    do 
        grep $term data/week1/LICENSE.txt | wc -l
    done
%%bash
for ((num=0; num<=5; num++))
    do 
        echo "I have $num cats"
    done
%%bash
tail -n 115 data/week1/LICENSE.txt | head | cut -f 1-2 -d " "
%%bash
find /Users/BlueOwl1/Documents -name pdf | paste -s -d : - | cat
!echo "scale=10; 4.32*(3/7)+1.23" | bc
%%bash
for num in {1..10}
    do
        # For modulo to work, scale must be 0
        echo "(1+$num) % 3" | bc
    done
%%bash
for num in {1..20}
    do
        if [ $[$num%15] = 0 ]; then
            echo fizzbuzz
        
        elif [ $[$num%3] = 0 ]; then
            echo fizz
            
        elif [ $[$num%5] = 0 ]; then
            echo buzz
        else
            echo $num
        fi
    done
%%bash
seq 15 | paste -sd+ -
Explanation: When asking the professor a question, use the STAR approach: Situation, Task, Action, Result.
Lectures
You must participate in the online Google group to get full participation credit.
Data modeling pipeline
* Understand domain, collect requirements
* Warehouse data
* EDA
* Feature engineering
* Modeling
* Lab-based experiments
* Deploy models and AB test
Bias-variance tradeoff
* Errors = bias^2 + variance + irreducible error
* Simple models have more bias and less variance
* Complex models have less bias and more variance
Squared bias = amount by which the expected model prediction differs from the true value over the training data
Variance = amount by which prediction over one training set differs from the expected predicted value over all training sets
$y = f(x) + \sigma^2$
Formula of bias = $E[h(x^)]-f(x^)$
Formula of variance = $E[h(x^)-E[h(x^)])^2]$
Formula of irreducible noise = $E[(y-f(x^*))^2] = E[\epsilon^2] = \sigma^2$
$h(x^*)$ is the model's prediction
$f(x^*)$ is the true value of the function
$y$ is the actual value
Standard procedure for calculating:
* Train model on bootstrap sample of data. Test model on datapoints that are not contained in the bootstrap sample. 
Before starting anything, do a back of an envelope calculation first.
Good rule of thumb. Reading 1TB = 3 hours on your machine. A computer will fail on average after 1000 days.
Problem solving using the command line
Allowed commands
* split: Split a file into many files
 * -l line_count: create smaller files line_count lines in length
 * -b byte_count: create smaller files byte_count bytes in length. Append k or m to make it kilobytes or megabytes
 * -a suffix_length: Defines how many total files can be created. suffix_length^26 total files. Default is 3.
 * usage: split [options][input file][prepend to output files]
* grep: Searches any given input files and selects lines that match one or more basic regular expressions. If the files are compressed, use zgrep. grep [search text] [file path]
 * -e pattern: include any regular expression. Can repeat to find multiple patterns
* wc: Counts things in the file
 * -c: count bytes
 * -l: count lines
 * -m: count characters
 * -w: count words
* cat: Concatenate files together. cat [list of files] > [output file]
* echo: prints text to screen
* for: allows for iteration
* cut: Cuts out selected portions of each line. 
 * -c list: The list specifies character positions
 * -f list: The list specifies fields, separated in the input by the field delimiter character.
 * -d delim: Use delim as the field delimiter character instead of the tab character.
 * End with file location
* paste: concatenates lines in separate files into one line. Can also use it to concatenate all the lines in a single file into one line.
 * -d list: characters to use to separate concatenated lines. Default is the tab character
 * usage to concatenate files: paste [file1] [file2] 
 * usage to concatenate all lines in a file or input: seq 10 | paste -sd[delimiter] -
* bc: Basic calculator. Usage echo "scale=10; math equation" | bc
 * scale = the precision of each calculation
* &: At the end of a line, sends the command to the background ctrl-z brings it back to the front. This causes the parent process to spawn off parallel processes and thus enables parallel computing.
* wait: waits for a specified subprocess to finish. Can reference subprocess with the actual process ID (i.e. 43532) or the relative reference (i.e. %4)
End of explanation
%%bash
seq 1000000 | wc &
echo "Finished waiting"
%%bash
seq 1000000 | wc &
wait; echo "Finished waiting"
Explanation: The wait command forces the system to finish processing the child process before the next command is run.
End of explanation
%%bash
mkdir pgrep_temp_files
cd pgrep_temp_files
split -l 10 -a 5 ../week1/CountOfMonteCristo.txt pgrep_temp_files
for file in pgrep_temp_files*
    do
        grep "Python" $file &
    done
# Return to original directory
cd ..
# Remove file that was created
rm -fr pgrep_temp_files
%%timeit
!grep Python week1/CountOfMonteCristo.txt > /dev/null
Explanation: Parallel grep (my attempt)
End of explanation
<END_TASK>