How to Build a Logistic Regression Model in R?

Whether it is predicting the likelihood of having a heart attack based on weight and workout routine or predicting the probability of email being spam based on the country of origin and word count -logistic regression is widely used because of its remarkable results. It is a machine learning method to solve a classification problem by differentiating one class from another in a given dataset.


Build a Logistic Regression Model in Python from Scratch

Downloadable solution code | Explanatory videos | Tech Support

Start Project

What is Logistic Regression?

Logistic regression is a binary classification machine learning model and is an integral part of the larger group of generalized linear models, also known as GLM. Logistic regression can also be extended to solve a multinomial classification problem. Applications for simple logistic regression could include determining whether a student would be admitted into a particular college with different academic parameters or using customer data to determine the type of advertisements or promotional offers best suited for a specific person. Given a dataset with some feature columns and a target, applications like these can be implemented with a decent performance using the logistic regression model.

logistic regression in r

The core of logistic regression lies in the sigmoid function (or the logistic function), which compresses any real value to bring it between 0 and 1.

Thus, in the logistic regression formula above, for any real value of x, f(x) would be a number between 0 and 1. We can decide a threshold (like 0.5) and give a binary decision as output based on this.

Further, logistic regression uses a linear equation to represent its weights in the modeling. Thus, for a binomial logistic regression model with two parameters βâ‚€ and  βâ‚,

Z = βâ‚€ + βâ‚X

The final representation will be,

hΘ(x) = sigmoid(Z) = σ(Z) or,

And, after training a logistic regression model, we can plot the mapping of the output logits before (Z) and after the sigmoid function is applied (σ(Z)).

The above figure shows the graph of the sigmoid function. The typical threshold for determining a decision or a decision boundary is 0.5, as shown above.

Logistic regression is not precisely a linear model like linear regression, so the learning process and cost function would not be the same. Instead, applying natural logarithm to the hΘ(x) equation and simplifying it to compare the prediction hΘ(x) with ground-truth y would give us the following property for the cost function:

Putting it together, for a set of m training samples, the average loss would be measured by

The goal, just like any other machine learning algorithm, is to minimize the cost function J(Θ). You must bring the derivative as close to 0 as possible to minimize it. And the gradient descent method updates the parameters (or weights) to achieve this.


The model converges when the change in weights is less than a certain threshold.

Multivariate logistic regression is simply an extension of the above equations. Instead of two parameters, the model would have to learn p+1 parameters for p feature columns. Thus the intermediate equation of the values inserted into the sigmoid function would be as below.

Why use logistic regression?

As a regression problem, logistic regression learns the relationship or mapping between many feature columns (also known as dependent variables) and an independent variable. Thus, this model helps categorize or classify new data into two predefined categories after learning. The model gives out the probability of the unknown/new sample belonging to a specific category, after which we can determine the outcome (class 1 vs. class 0).

If logistic regression is just an extension of linear regression, what advantages does it add? Well, first of all, logistic regression relaxes some assumptions of linear regression. A linear regression model makes the following four assumptions before modeling:

  • Linearity

  • Homoscedasticity

  • Independence

  • Normality

In brief, the relationship between X and y (mean of y) variables of the dataset must be linear, the variances of the residuals are the same for all subsets of X, the feature columns must have minimal multicollinearity (they must be independent of each other) and no autocorrelation, and the dataset must be normally distributed. 

In logistic regression, X and y need not have a linear relationship. Similarly, the constraints for normal distribution and homoscedasticity are removed as well. This relaxation allows for more realistic datasets to be used for modeling.

However, the assumption for independence still applies to logistic regression. The feature columns must not have a high correlation among them. Use the Variance Inflation Factor (VIF) to check for multicollinearity among the feature columns. Moreover, drawing a Residual Series plot can check for independence among the data samples in your dataset. Repeated or correlated samples might be detrimental to the model training. You should also check for outliers or any highly significant anomalies in your data using Cook’s distance. One should remove such abnormalities before training.

Moreover, for binary logistic regression, it is required that the target column has binary numeric values. Similarly, if the expected output is multinomial or ordinal, then the data in the target column must also be multinomial or ordinal.

Get Closer To Your Dream of Becoming a Data Scientist with 150+ Solved End-to-End ML Projects

Next, as we saw in the equations above, the logistic regression function depends on a linear combination of specific coefficients. Hence, another assumption of logistic regression is that the feature columns must be linearly related to these particular coefficients (also known as log-odds). The Box-Tidwell test can be used to check for this condition in your data.

The final assumption is merely a suggestion related to the dataset size during training. It is recommended that one would need at least ten samples with the least frequent outcomes for each unique feature. Thus, for N features and the probability of the least frequent outcome for each feature x, the number of recommended samples is (10*N/x).

Learn to Implement Logistic Regression Model for Customer Churn Prediction

Logistic Regression in R Example - How to do Logistic Regression in R?

Since, for most people, this might be the first implementation of logistic regression in R, we will start by using the primary Titanic dataset found on Kaggle. The dataset covers several points worth tackling before training any models: handling categorical variables, univariate vs. multivariate modeling, data cleaning, and more. The dataset aims to classify each sample into the binary class: survived (1) or not survived (0) based on 11 features of different datatypes.

Once downloaded, you can load the training data using R's read.csv() function.

Displaying the first few samples using[5:8,]:

As we see, the column “Cabin” and “Age” have some “NA” values which might be problematic in training. Let’s see how many null values exist in each column.

Some data cleaning is required to handle the null values. “Cabin” has more null values that can be handled safely using data cleaning techniques, so we will simply drop the column. “PassengerId” is also a redundant column adding no information since it is merely a primary key column. This will be done using the subset function of R.


Next, we can simply drop the two samples having NULL in the Embarked column and replace null values in the Age column with the mean (or median, mode, or fixed value). The data.frame data type in R helps do these operations quickly.

Next, we can see that the “Sex” and “Embarked” columns have categorical (or text) values. From our understanding, mathematical models can only understand numbers, and thus, these columns need to be numeric. There are different methods to handle this, such as one-hot encoding, vectorization, or binary label encoding. Fortunately, R handles this internally when we use the read.csv function. The two categorical columns are converted to numeric by one-hot encoding them. The “factor” column type identifies such numeric categorical columns in R. We can check if our columns are factors by the is.factor() function.

If the result is False, then we can force the factor data type onto the columns:

contrasts() will display how R reads the specified column.

How to fit a Logistic Regression Model in R? 

Now that our data is ready, we can fit the logistic regression model in R.

First, the data is divided into train and test samples. Next, we train the GLM model using the binomial distribution. In the glm() function, the first parameter would be as 


Since our dependent_column is “Survived,” and we wish to use all the selected feature columns, our input format is Survived~.

Next, we select the binomial family object since it contains the implementation of logistic regression, which we specify by setting link to “logit.” Finally, the data.frame variable containing the training data is provided.

How to Evaluate and Interpret the Logistic Regression Model in R?

summary() displays some statistical quantities regarding our model, including the assigned weights for each feature column. We can see below that “Sex” and “Pclass” have a high significance in determining the final output of “Survived”. Moreover, we can see that the sex “male” has a highly negative influence over the target (meaning that samples with sex “male” are doubtful to survive).

Since our model is trained and ready, we can utilize the test set to evaluate the performance.

The predict() function generates the logits (probabilities) for each sample in the test set. As discussed in theory above, a decision boundary would finally decide the binary classification of the sample. In this case, we use ifelse() and a decision boundary of 0.5 to get the final output vector.

mean() is used to calculate the average error rate or misclassification rate, giving us the accuracy of the logistic regression model. The accuracy of our model is 84% on a small dataset of around 800 samples. Other training and evaluation techniques such as k-cross-validation can yield better results.

Before moving on, we can evaluate our model statistically using the ROC curve and AUC measure.

AUC score for our logistic regression model is around 0.84, while an ideal score is 1.0. And the ROC curve is decent, too as shown below.

How to Interpret Logistic Regression Coefficients in R?

Once the model is trained, we can see the coefficients assigned to each feature column, just like in linear regression. coef(model) will display the following information.

Each value represents the relative importance of the corresponding feature in determining the final output decision (whether the passenger survived). As observed in the previous statistical quantities, Sex (male) and Pclass have significantly high influence over the final output as denoted by higher absolute values in their coefficients.

However, these values are intermediate states in the logistic regression equation and not directly proportional to changes in the final output, unlike linear regression. According to the formula, adding an exponential term ahead of these coefficients would give us the real influence or “impact” the feature columns have over the output “Survived.”

This shows that for one unit increase in “Pclass”, the “Survived” variable changes by a factor of around 0.33. Similarly, for a one-unit increase in “Sex,” the output changes drastically by a factor of 0.06. The factor of changes in other features is closer to 1.0 which would have a very insignificant change in the output. This corroborates our findings and observations. 


To test it empirically, let’s train and test our model on just two feature variables: Pclass and Sex.

The new accuracy is 77%, instead of the original accuracy of 84%.

Similarly, by adding “Age” and “Embarked” to the list, we get an accuracy of 79%. However, “SibSp” has a relatively significant impact. If we add that to our list of features, our test accuracy jumps to 82%. Thus, we can understand the importance of feature selection for model training from such combinations. Fewer features might be computationally cheaper, but a wrong choice of features would greatly deteriorate test accuracy. On the contrary, a wise choice of features would simplify and increase efficiency in training and not harm the model performance.

Note: To check the index numbers of these columns before selecting them, simply display the column list and select them in the order as shown.

How to Interpret the Odds Ratio in Logistic Regression?

Each of the coefficients we received using coef(model) is also known as the log of odds. In theory, odds are calculated by taking the probability of success and failure ratio. For instance, taking a sample data distribution of genders of students who have or have not taken an honors class:

So, in this case, the probability of choosing a male student who has taken the class is 17/200. However, selecting a student in the honors class from the male students would be 17/91. Thus, the odds that a male student is in the class would be

{prob success}/{prob failure} = (17/91)/(74/91) = 0.23

Similarly, the coefficients we get from coef(model) are the natural logarithm form of these odds, which can be reverted to an “odds ratio” using an exponential. The “odds ratio” for the feature of gender would be the {odds of male}/{odds of female} and the log of this value would be the “Sexmale” coefficient in our Titanic model. Alternatively, “Sexfemale” would be log({odds of female}/{odds of male}).


Request a demo

In the honors class example, the odds ratio favoring females would be around 1.809. So, if a logistic regression model is trained on such data, the coefficient for the feature “female” (0/1) would be log(1.809)=0.593. And thus, one unit increase in females would have an around 81% increase in the odds favoring females (and therefore the model's coefficient).

Intuitively, we can understand that the exponential of the coefficients would simply be the odds ratio. Hence, the “impact” computed earlier is merely the odds ratio. By exponentiating the coefficients, we attempt to check the expected increase in the target value for each unit increase in the specific feature. It is easy to observe this in the univariate honors class example above.

For multivariate models like our Titanic Survival Prediction logistic regression model, you can interpret the odds for each feature by keeping all other values constant before considering different values for the selected feature, for example, considering the “male” type of “Sex.” Keeping everything else constant, we can say that a sample with “Sex”= ”male” has exp(-2.75) = 0.06 odds of having “Survived”=1. In terms of percentage change, males are at a 6% chance of survival compared to females on board the Titanic.

How to Interpret Multinomial Logistic Regression?

Multinomial logistic regression simply extends binary logistic regression to allow more than two classes. This idea fundamentally changes the way probabilities are computed, and training is done for a logistic regression model. Firstly, we cannot give a single possibility as output. Instead, for a given sample, the probability of it belonging to each class needs to be computed, after which, using a decision boundary, we can determine the final class.

Multinomial logistic regression can be implemented in R using a different package and functions than glm(). We use the multinom() function from the nnet package of R. We use this function to train the UCI Breast Tissue dataset. This dataset contains six classes originally, but for simplicity, we reduce it to 4 by combining samples of 3 different classes under a single class, “other.” 

In the model summary, we can see how the log of odds or the coefficients is computed for each feature variable regarding each output class. Since “adi” was the reference class, coefficients are not computed. Reference class is analogous to the class “0” when the coefficients are calculated for class being “1”. Thus, if none of the calculated odds rank above decision boundary, the reference class is assigned to the sample in question. Further interpretation for each class coefficient and odds ratio remains the same as that in a binomial logistic regression model.

Next, we can see the output of a multinomial logistic regression model as discussed previously in the form of logits for each target class. Note that the probabilities in each row add up to 1.0.

We found the training accuracy 98.57%; now, we can check how well this model performs on the test set. You can evaluate the classification results in a tabular format.

Thus, we can observe 100% accuracy in the test set of this small and simple dataset.

This article looked at a type of regression algorithm called logistic regression. We understood its mathematics and why it is better than linear regression. We also learned how logistic regression is implemented in R. A hands-on experience certainly provides a great introduction to any such machine learning algorithm. Moreover, we interpreted the model coefficients and various statistical quantities from this hands-on exercise to evaluate our model numerically. Such a balance of theory, practical, and empirical observations is crucial to understanding the fundamentals of logistic regression in a smooth and retainable way.




Access Solved Big Data and Data Science Projects

aws image

Start Your First Project

Learn By Doing

What Users are saying..

profile image

Juan Solis

Senior Data Scientist at en DUS Software Engineering
linkedin profile url
