Understanding Data Mining

desertcockatooData Management

Nov 20, 2013 (4 years and 5 months ago)


Understanding Data Mining

Lesson 2: Using R for Data Mining


I. Many times, a situation depends on numerous factors. Someone’s credit score, for example,
isn’t just determined by a single answer to a question on a loan application. List some of
things you think a credit company asks for on a credit application.




Obviously, some pieces of information you listed above are more relevant than others in
determining whether or not someone should get a loan. As a result, we need a process that can
analyze these la
rge amounts of data in order to determine which factors are the BEST predictors
of someone’s “credit
worthiness.” One such process is called
data mining

II. Search the internet for five facts about data mining. Record them below:

1. ________________

2. ___________________________________________________________________________

3. ___________________________________________________________________________

4. _____________________________

5. ___________________________________________________________________________

III. We can use R to experiment with the data mining process. Let’s first try out the process on
a data set that is already i
n R.

1. Type

to see the data set we are going to use.

2. Type

to see the details of the data set. It is always important to understand
the source of data when trying to make predictions and draw conclusions!

3. List the varia
bles p
resented in this data set below, and then circle the dependent variable:


Now return to your R console and load the data using the command

5. Let’s
find a linear model to relate the variables that impact a person’s overall rating. Our
command will be:

lm(attitude$rating ~ attitude$complaints
+attitude$privileges +attitude$learning + attitude$raises + attitude$critical
+ attitude$advance).

Understanding Data Mining

cord the equation below:


6. We can use an analysis of variance to draw initial conclusions about which variables are the
most influential in a person’s overall rating. To access this, type the command

The variables with the highest F
values have the highest degree of influence on a person’s
overall rating. List the variables in order from the highest to the lowest F


8. While F
values help us identify the overall relevance of each variable, it doesn’t allow us to
draw conclusions about which variables may have no measurable effect on a person’s overall
rating. As a result, statisticians often times rely on variable

selection processes to identify the
variables that should be considered. We will try three methods of variable selection:

forward selection
: start with the first variable and keep adding variables to the

prediction equation

backward selection
: sta
rt with all variables and eliminate variables from the prediction

equation one at a time

stepwise selection
: like forward selection, except some variables may be deleted after

they have been added

9. Let’s try forward selection first. The command i
data=data.frame(attitude)), scope=list(lower=~1, upper=~attitude
$complaints +
+attitude$advance), direction = “forward”).

10. Look at the f
inal step and record the relevant variables as identified by forward selection.


11. Now, let’s try backward selection. The command is
step(att, direction =
. Look at the final step and record t
he relevant variables according to backward


12. Finally, let’s try stepwise selection. The command is
step(att, direction =

As before, record the relevant variables according to stepwi
se selection.


13. Our conclusion is that the factors that should be considered when determining someone’s
overall rating are:


Understanding Data Mining

14. Ho
w could our conclusion be helpful to people studying the results of the survey? ________


IV. Use the internet to research three uses of data mining. Make sure to explain h
ow data
mining is used in each of the three cases:

1. ___________________________________________________________________________


2. _______________________________________


3. ___________________________________________________________________________