<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Statistics Cheat Sheet
Step1:  Associated is dependant variable. Both can't happen at the same time
Anecdotal evidence happen in haphazard fashion. Only represent one/two cases, not known if can be generalized to population, and sometimes it happen in extreme cases.
Prospective collects inf as event unfold/continue. Retrospective collects inf of events in the past.
Stratified sampling
Step2:  To calculate the percentiles
Step3:  Pnorm ranges
Step4:  Standard normal distributions, is mean 0 and sd 1. Used in default pnorm and qnorm R.
Always draw normal probability distributions, and shade the are we're interesed.
In evaluation the distribution through normal probability plot, imagine a straight line. If the data is skewed to the high end of the line, it's right skewed. Otherwise , low-end is left skewed.
To construct normal probability plot, for each observation, make a scatter plot with its Z-score(horizontal) against its percentile(vertical).
Binomial
Screenshot taken from Coursera video, 09
Step5:  Requirements
The trials are independent. 
The number of trials, n, is fixed. 
Each trial outcome can be classified as a success or failure. 
The probability of a success, p, is the same for each trial. 
For calculating combinations,
Step6:  Screenshot taken from Coursera video, 12
Step7:  To check whether the data is sufficiently large,
Step8:  For binorm norm, and for taking the probability of more than one events
Step9:  Because there's no exact value in binomial, we have to decrement by 0.05.
CLT
Requirements
independent
random sampling/assignment
larger than 30 or if population is normal, < 10% population avoid dependant.
Or Plot population/sample distribution and see if normal.
(>> n, less shape required)
Confidence Interval
Snapshot taken from Coursera 04
Step10:  Sample Size for ME
Step11:  Hypothesis Testing
Snapshot taken from Coursera 14
<ASSISTANT_TASK:>
Python Code:
%load_ext rpy2.ipython
Explanation: Statistics Cheat Sheet
End of explanation
%%R
mean = 1500
sd = 300
d = 1800
(d-mean)/sd
Explanation: Associated is dependant variable. Both can't happen at the same time
Anecdotal evidence happen in haphazard fashion. Only represent one/two cases, not known if can be generalized to population, and sometimes it happen in extreme cases.
Prospective collects inf as event unfold/continue. Retrospective collects inf of events in the past.
Stratified sampling: adv: each stratum similar output interest. Disadv, more complex analyze than SRS.
Cluster sampling, adv = more economic, not required strata like stratitified, useful when each cluster have diverse, but clusters perform similarity. Disadv: most complex technique
Randomized experiment:
Controlling. Do their best to control the groups difference.
Randomization. Randomly assign patient to groups, to avoid hidden lurking variable.
Replication. Collecting large sample or replicate entire previous study.
Blocking. Divide suspected variable into blocks, and randomly assign each block to treatment and placebo group.
Control treatment received fake treatment, placebo.
T-boxplot is called, whisker, try to capture the data outside the box. 1.5IQR(-Q1/+Q3). Outside whiskers is outliers.
Contigency table represent summary(often frequency) of two categorical variable. Frequency table showing one categorical variable.
Identify most useful row/column proportions in contigency. When classify based on first variable, we proportion in second variable.
In independent variables, we suspect that one variable should be equal regardless of dependency. But usually we will have small difference that occured due to chance
Probability
Disjoint events, probability one of them occured is P(1) + P(2)+ ..., it's the same as probability one of number in die is add up to 1.
events : set of outcomes.
Law of Addition Rule, disjoint vs not disjoint. Disjoint, P(A and B) automatically zero.
P (A or B) = P (A) + P (B) − P (A and B)
* Probability distributions : all disjoint, between 0 and 1, add up to 1.
* Sample space, list of all possible outcomes. die event {2,3}, then complement {1,4,5,6}
* Independence can allow two events to occurs at the same time. This will have a multiplication rule, where P( A and B occurs at the same time)
P(A and B) = P(A) × P(B)
For example, Since gender,handedness and person order(first vs third person) are independence event. We can, multiply together. (First or second or third) person female and right hand = P(female) * P(right hand).First two MR, third FL, 
P(MR) ** 2  * P(FL)
Marginal probability, row/columns total in propotion of total population. Not consider other variables.
Join probability, take intersection of two variables, in proportion of total population.
*Conditional probability:
P (A|B) = P (A and B) / P(B) or P(A and B) = P(A|B) × P(B)
We can use General multiplication rule in conjunction with marginal and conditional probability.
with replacement, data can picked over and over again, thus make it independent every pick. Without replacement make it dependant events.
Distributions
For z-score
End of explanation
%%R
mean = 1500
sd = 300
point = 2100
LT = T
pnorm(point,mean=mean,sd=sd,lower.tail=LT)
%%R
mean = 1500
sd = 300
percentile = 0.4
LT = T
qnorm(percentile,mean=mean,sd=sd,lower.tail=LT)
Explanation: To calculate the percentiles
End of explanation
%%R
mean = 70
sd = 3.3
lower = 69
upper = 74
LT = T
# IN RANGE
# 1 - pnorm(upper,mean=mean,sd=sd,lower.tail=!LT) - pnorm(lower,mean=mean,sd=sd,lower.tail=LT)
# OUT RANGE
# pnorm(upper,mean=mean,sd=sd,lower.tail=!LT) + pnorm(lower,mean=mean,sd=sd,lower.tail=LT)
Explanation: Pnorm ranges
End of explanation
%%R
k = 8
n = 10
p = 0.3
dbinom(k,size=n,p=p)
Explanation: Standard normal distributions, is mean 0 and sd 1. Used in default pnorm and qnorm R.
Always draw normal probability distributions, and shade the are we're interesed.
In evaluation the distribution through normal probability plot, imagine a straight line. If the data is skewed to the high end of the line, it's right skewed. Otherwise , low-end is left skewed.
To construct normal probability plot, for each observation, make a scatter plot with its Z-score(horizontal) against its percentile(vertical).
Binomial
Screenshot taken from Coursera video, 09:53
End of explanation
%%R
k_success = 8
trials = 8
choose(trials,k_success)
Explanation: Requirements
The trials are independent. 
The number of trials, n, is fixed. 
Each trial outcome can be classified as a success or failure. 
The probability of a success, p, is the same for each trial. 
For calculating combinations,
End of explanation
import numpy as np
n = 40
p = 0.35
sd = round(np.sqrt(n*p*(1-p)),2)
print 'Expected Value of point success is', n*p
print 'standard deviation is', sd
print 'variance is' , sd**2
import numpy as np
n = 2500
p = 0.7
sd = round(np.sqrt(n*p*(1-p)),2)
print 'Expected Value of point success is', n*p
print 'standard deviation is', sd
print 'variance is' , sd**2
Explanation: Screenshot taken from Coursera video, 12:07
If for in fact the data is not sufficiently large, then it can't use normal distribution advantage. Then the only is one to do this, is manually compute for each of the value.
P(p,n,k, k &lt; 2) = P(0) + P(1)
When we know mean and standard deviation of binomial-normal distribution, we can determine the probability using z=score.
Screenshot taken from Coursera video, 09:59
If all binomial requirements are followed, then this could also work.
End of explanation
p = 0.01
n = 300
assert (n*p >= 10 and n*(1-p) >= 10), 'Not large enough to take advantage of normal approximation'
Explanation: To check whether the data is sufficiently large,
End of explanation
%%R
val = 59
mu = 80
sd = 8
tail = T
# Add 0.5 to val if observe small range of observation
ifsmall = (-0.5+!tail)
pnorm (val,mean=mu,sd=sd,lower.tail=tail)
Explanation: For binorm norm, and for taking the probability of more than one events
End of explanation
%%R
#95% = 1.96
#99% = 2.58
n = 50
mu = 3.2
s = 1.74
# z = 1.96
CL = 0.95
z = round(qnorm((1-CL)/2,lower.tail=F),digits=2)
SE = s/sqrt(n)
ME = z*SE
c(mu-ME,mu+ME)
Explanation: Because there's no exact value in binomial, we have to decrement by 0.05.
CLT
Requirements
independent
random sampling/assignment
larger than 30 or if population is normal, < 10% population avoid dependant.
Or Plot population/sample distribution and see if normal.
(>> n, less shape required)
Confidence Interval
Snapshot taken from Coursera 04:26
Let's us state that:
n = 50
mu  =  3.2
s = 1.74
z = 1.96
$$SE = \frac{s}{\sqrt{n}} = \frac{1.74}{\sqrt{50}} \approx 0.246$$
$$\bar{x} \pm z * SE = 3.2 \pm 1.96 * 0.246 $$
$$3.2 \pm 0.48 = (2.72,3.68)$$
CI formula
End of explanation
%%R
CL = 0.9
ME = 4
sd = 18
#CONFIDENCE LEVEL
#95% = 1.96
#99% = 2.58
#90% = 1.65
#####
z_star = round(qnorm((1-CL)/2,lower.tail=F),digits=2)
#z_star = 1.65
((z_star*sd)/ME)**2
%%R
CL = 0.9
ME = 4
sd = 18
#CONFIDENCE LEVEL
#95% = 1.96
#99% = 2.58
#90% = 1.65
#####
z_star = round(qnorm((1-CL)/2,lower.tail=F),digits=2)
#z_star = 1.65
((z_star*sd)/ME)**2
Explanation: Sample Size for ME
End of explanation
%%R
xbar = 118.2
mu = 100
sd = 6.5
n = 36
SE = round(sd/sqrt(n),digits=2)
pnorm(xbar, mean=mu,sd=SE, lower.tail=xbar < mu) * 2
Explanation: Hypothesis Testing
Snapshot taken from Coursera 14:01
P-value
Snapshot taken from Coursera 05:19
Interpreting p-value:
X% chance that random sample of n will draw average of z
% chance is higher than threshold, therefore it could be happen by chance.
This is one-sided HT. two-sided will need to double the counting 0.209*2 = 0.418
To calculate HT for parameter mean in two-sided, lose x2 for one sided
End of explanation
<END_TASK>