Statistics Toolbox

Credit Rating by Bagging Decision Trees

This example shows how to build an automated credit rating tool.


Assigning a credit grade involves analyzing information on the borrower. If the borrower is an individual, information of interest could be the individual's income, outstanding debt (mortgage, credit cards), household size, residential status, etc. For corporate borrowers, one may consider certain financial ratios (e.g., sales divided by total assets), industry, etc. Here, we refer to these pieces of information about a borrower as features or predictors. Different institutions use different predictors, and they may also have different rating classes or score ranges to rank their customers. For relatively small loans offered to a large market of potential borrowers (e.g., credit cards), it is common to use credit scores, and the process of grading a borrower is usually automated. For larger loans, accessible to small- to medium-sized companies and larger corporations, credit ratings are usually used, and the grading process may involve a combination of automated algorithms and expert analysis.


This example shows how MATLAB® can help with the automated stage of a credit rating process. In particular, we take advantage of one of the statistical learning tools readily available in Statistics Toolbox™, a classification algorithm known as a bagged decision tree.



In practice, one needs to train a classifier first, then use it to assign a credit rating to new customers, and finally one also needs to profile or evaluate the quality or accuracy of the classifier, a process also known as validation or back-testing. We discuss some readily available back-testing tools, as well.

Loading the Existing Credit Rating Data

We load the historical data from the comma-delimited text file CreditRating_Historical.dat. We choose to work with text files here, but users with access to Database Toolbox™ can certainly load this information directly from a database.

The data set contains financial ratios, industry sector, and credit ratings for a list of corporate customers. This is simulated, not real data. The first column is a customer ID. Then we have five columns of financial ratios. These are the same ratios used in Altman's z-score (see Altman [1]; see also Loeffler and Posch [4] for a related analysis).

  • Working capital / Total Assets (WC_TA)

  • Retained Earnings / Total Assets (RE_TA)

  • Earnings Before Interests and Taxes / Total Assets (EBIT_TA)


  • Sales / Total Assets (S_TA)

Next, we have an industry sector label, an integer value ranging from 1 to 12. The last column has the credit rating assigned to the customer. We load the data into a table array.

creditDS = readtable('CreditRating_Historical.dat');

We copy the features into a matrix X, and the corresponding classes, the ratings, into a vector Y. This is not a required step, since we could access this information directly from the dataset or table array, but we do it here to simplify some repeated function calls below.

The features to be stored in the matrix X are the five financial ratios, and the industry label. Industry is a categorical variable, nominal in fact, because there is no ordering in the industry sectors. The response variable, the credit ratings, is also categorical, though this is an ordinal variable, because, by definition, ratings imply a ranking of creditworthiness. We can use this variable "as is" to train our classifier. Here we choose to copy it into an ordinal array because this way the outputs come out in the natural order of the ratings and are easier to read. The ordering of the ratings is established by the cell array we pass as a third argument in the definition of Y. The credit ratings can also be mapped into numeric values, which can be useful to try alternative methods to analyze the data (e.g., regression). It is always recommended to try different methods in practice.

X = [creditDS.WC_TA creditDS.RE_TA creditDS.EBIT_TA creditDS.MVE_BVTD...
     creditDS.S_TA creditDS.Industry];
Y = ordinal(creditDS.Rating);


Constructing the Tree Bagger

The first step to construct our classification ensemble will be to find a good leaf size for the individual trees; here we try sizes of 1, 5 and 10. (See Statistics Toolbox documentation for more on TreeBagger.) We start with a small number of trees, 25 only, because we mostly want to compare the initial trend in the classification error for different leaf sizes. For reproducibility and fair comparisons, we reinitialize the random number generator, which is used to sample with replacement from the data, each time we build a classifier.

leaf = [1 5 10];
nTrees = 25;
savedRng = rng; % save the current RNG settings

color = 'bgr';
for ii = 1:length(leaf)
   % Reinitialize the random number generator, so that the
   % random samples are the same for each leaf size
   % error 'oobError'
   b = TreeBagger(nTrees,X,Y,'OOBPred','on',...
   hold on;
xlabel('Number of grown trees');
ylabel('Out-of-bag classification error');
title('Classification Error for Different Leaf Sizes');
hold off;

The errors are comparable for the three leaf-size options. We will therefore work with a leaf size of 10, because it results in leaner trees and more efficient computations.


Next, we want to find out whether all the features are important for the accuracy of our classifier. We do this by turning on the feature importance measure (oobvarimp), and plot the results to visually find the most important features. We also try a larger number of trees now, and store the classification error, for further comparisons below.

nTrees = 50;
leaf = 10;
b = TreeBagger(nTrees,X,Y,'OOBVarImp','on',...

xlabel('Feature number');
ylabel('Out-of-bag feature importance');
title('Feature importance results');

oobErrorFullX = b.oobError;




Let us fit a new classification ensemble using only predictors RE_TA, MVE_BVTD, and Industry. We compare its classification error with the previous classifier, which uses all features.

X = [creditDS.RE_TA creditDS.MVE_BVTD creditDS.Industry];


oobErrorX246 = b.oobError;

hold on;
xlabel('Number of grown trees');
title('Classification Error for Different Sets of Predictors');
hold off;

The accuracy of the classification does not deteriorate significantly when we remove the features with relatively low importance (1, 3, and 5), so we will use the more parsimonious classification ensemble for our predictions.

In this example, we have started with a set of six features only, and used the feature importance measure of the classifier, and the out-of-bag classification error as criteria to screen out three of the variables. Feature selection can be a time consuming process when the initial set of potential predictors contains dozens of variables. Besides the tools we have used here (variable importance and a "visual" comparison of out-of-bag errors), other variable selection tools in Statistics Toolbox can be helpful for these types of analyses (see documentation). However, in the end, a successful feature selection process requires a combination of quantitative tools and an analyst's judgement.


At this point, the classifier could be saved (e.g., save classifier.mat b), to be loaded in a future session (load classifier) to classify new customers. For efficiency, it is recommended to keep a compact version of the classifier once the training process is finished.

b = b.compact;

Classifying New Data

Here we use the previously constructed classification ensemble to assign credit ratings to new customers. Because the ratings of existing customers need to be reviewed, too, on a regular basis, especially when their financial information has substantially changed, the data set could also contain a list of existing customers under review. We start by loading the new data.

newDS = readtable('CreditRating_NewCompanies.dat');

To predict the credit rating for this new data, we call the predict method on the classifier. The method returns two arguments, the predicted class and the classification score. We certainly want to get both output arguments, since the classification scores contain information on how certain the predicted ratings seem to be. We could copy variables RE_TA, MVE_BVTD and Industry into a matrix X, as before, but since we will make only one call to predict, we can skip this step and use newDS directly.

[predClass,classifScore] = b.predict([newDS.RE_TA newDS.MVE_BVTD...

At this point, we can create a report. Here we only display on the screen a small report for the first three customers, for illustration purposes, but MATLAB's deployment tools could greatly improve the workflow here. For example, using MATLAB Builder™ JA, credit analysts could run this classification remotely, using a web browser, and get a report, without even having MATLAB on their desktops.

for i = 1:3
   fprintf('   Classification score : \n');
   for j = 1:length(b.ClassNames)
      if (classifScore(i,j)>0)
Customer 60644:
   RE/TA    =  0.22
   MVE/BVTD =  2.40
   Industry =  6
   Predicted Rating : AA
   Classification score : 
      A : 0.3064 
      AA : 0.6778 
      BBB : 0.0158 
Customer 33083:
   RE/TA    =  0.24
   MVE/BVTD =  1.51
   Industry =  4
   Predicted Rating : BBB
   Classification score : 
      A : 0.0656 
      BBB : 0.9344 
Customer 63830:
   RE/TA    =  0.18
   MVE/BVTD =  1.69
   Industry =  7
   Predicted Rating : A
   Classification score : 
      A : 0.7020 
      AA : 0.0113 
      B : 0.0022 
      BB : 0.0002 
      BBB : 0.2842 

Keeping records of the predicted ratings and corresponding scores can be useful for periodic assessments of the quality of the classifier. We store this information here in the table array predDS.

classnames = b.ClassNames;
predDS.Properties.VariableNames = {'ID','PredRating',classnames{:}};



or written directly to a database using Database Toolbox.

Back-Testing: Profiling the Classification Process





exPostDS = readtable('CreditRating_ExPost.dat');

If this were a new MATLAB session, besides reading the follow up data from CreditRating_ExPost.dat we would need to load the predicted ratings information stored in predDS, for example, with the command

  predDS = readtable('PredictedRatings.dat')


The first tool we can use to compare predicted vs. actual ratings is a confusion matrix, readily available in Statistics Toolbox:

C = confusionmat(exPostDS.Rating,predDS.PredRating,...
C =

   204    12     0     0     0     0     0
     8   110    20     0     0     0     0
     0    18   157    29     0     0     0
     0     1    28   247    47     0     0
     0     0     0    41   227    17     0
     0     0     0     0    48    42     3
     0     0     0     0     0    10    42

The rows in C correspond to the actual ratings, and the columns to the predicted ratings. The amount in the position (i,j) in this matrix indicates how many customers received an actual rating i and were predicted as rating j. For example, position (3,2) tells us how many customers received a rating of 'A' by the credit committee, but were predicted as 'AA' with the automated classifier. One can also present this matrix in percentage form with a simple transformation:

Cperc = diag(sum(C,2))\C
Cperc =

    0.9444    0.0556         0         0         0         0         0
    0.0580    0.7971    0.1449         0         0         0         0
         0    0.0882    0.7696    0.1422         0         0         0
         0    0.0031    0.0867    0.7647    0.1455         0         0
         0         0         0    0.1439    0.7965    0.0596         0
         0         0         0         0    0.5161    0.4516    0.0323
         0         0         0         0         0    0.1923    0.8077

Good agreement between the predicted and the actual ratings would result in values in the main diagonal that dominate the rest of the values in a row, ideally values close to 1. In this case, we actually see an important disagreement for 'B,' since about half of the customers that were rated as 'B' by the credit committee had been predicted as 'BB' by the automated classifier. On the other hand, it is good to see that ratings differ in at most one notch in most cases, with the only exception of 'BBB.'


For each specific rating, we can compute yet another measure of agreement between predicted and actual ratings. We can build a Receiver Operating Characteristic (ROC) curve using the perfcurve function from Statistics Toolbox, and check the area under the curve (AUC). The perfcurve function takes as an argument the actual ratings, which are our benchmark, the standard we are comparing against, and the 'BBB' classification scores determined by the automated process. Let us build a ROC and calculate the AUC for rating 'BBB' in our example.

[xVal,yVal,~,auc] = perfcurve(exPostDS.Rating,predDS.BBB,'BBB');
xlabel('False positive rate');
ylabel('True positive rate');
title('ROC curve BBB, predicted vs. actual rating');

Here is an explanation of how the ROC is built. Recall that for each customer the automated classifier returns a classification score for each of the credit ratings, in particular, for 'BBB,' which can be interpreted as how likely it is that this particular customer should be rated 'BBB.' In order to build the ROC curve, one needs to vary the classification threshold. That is, the minimum score to classify a customer as 'BBB.' In other words, if the threshold is t, we only classify customers as 'BBB' if their 'BBB' score is greater than or equal to t. For example, suppose that company XYZ had a 'BBB' score of 0.87. If the actual rating of XYZ (the information in exPostDS.Rating) is 'BBB,' then XYZ would be correctly classified as 'BBB' for any threshold of up to 0.87. This would be a true positive, and it would increase what is call the sensitivity of the classifier. For any threshold greater than 0.87, this company would not receive a 'BBB' rating, and we would have a false negative case. To complete the description, suppose now that XYZ's actual rating is 'BB.' Then it would be correctly rejected as a 'BBB' for thresholds of more than 0.87, becoming a true negative, and thus increasing the so called specificity of the classifier. However, for thresholds of up to 0.87, it would become a false positive (it would be classified as 'BBB,' when it actually is a 'BB'). The ROC curve is constructed by plotting the proportion of true positives (sensitivity), versus false positives (1-specificity), as the threshold varies from 0 to 1.



It is not hard to see that the perfcurve function can also be used to construct the CAP. The standard we compare against is not a rating, as before, but the default flag that we loaded from the CreditRating_ExPost.dat file. The score we use is a "dummy score" that indicates the ranking in creditworthiness implicit in the list of ratings. The dummy score only needs to satisfy that better ratings get lower dummy scores (they are "less likely to have a default flag of 1"), and that any two customers with the same rating get the same dummy score. A default probability could be passed as a score, of course, but we do not have default probabilities here, and in fact we do not need to have estimates of the default probabilities to construct the CAP, because we are not validating default probabilities. All we are assessing with this tool is how well the ratings rank customers according to their creditworthiness.


Nratings = length(ratingsList);
dummyDelta = 1/(Nratings+1);

D = exPostDS.Def_tplus1;
fracTotDef = sum(D)/length(D);
maxAcc = 0.5 - 0.5 * fracTotDef;

S = dummyRank(R);
[xVal,yVal,~,auc] = perfcurve(D,S,1);

accRatio = (auc-0.5)/maxAcc;


xlabel('Fraction of all companies');
ylabel('Fraction of defaulted companies');
title('Cumulative Accuracy Profile');
Accuracy ratio for actual ratings: 0.850

The key to reading the information of the CAP is in the "kinks," labeled in the plot for ratings 'CCC,' 'B,' and 'BB.' For example, the second kink is associated with the second lowest rating, 'B,' and it is located at (0.097, 0.714). This means that 9.7% of the customers were ranked 'B' or lower, and they account for 71.4% of the defaults observed.

In general, the accuracy ratio should be treated as a relative, rather than an absolute measure. For example, we can add the CAP of the predicted ratings in the same plot, and compute its accuracy ratio to compare it with the accuracy ratio of the actual ratings.

Spred = dummyRank(Rpred);
[xValPred,yValPred,~,aucPred] = perfcurve(D,Spred,1);

accRatioPred = (aucPred-0.5)/maxAcc;

xlabel('Fraction of all companies');
ylabel('Fraction of defaulted companies');
title('Cumulative Accuracy Profile');
Accuracy ratio for predicted ratings: 0.831


Final Remarks

MATLAB offers a wide range of machine learning tools, besides bagged decision trees, that can be used in the context of credit rating. In Statistics Toolbox you can find classification tools such as discriminant analysis and naive Bayes classifiers. MATLAB also offers Neural Networks Toolbox™. Also, Database Toolbox and MATLAB's deployment tools such as MATLAB Builder JA may provide you with more flexibility to adapt the workflow presented here to your own preferences and needs.

No probabilities of default have been computed here. For credit ratings, the probabilities of default are usually computed based on credit-rating migration history. See the transprob reference page in Financial Toolbox™ for more information.


[1] Altman, E., "Financial Ratios, Discriminant Analysis and the Prediction of Corporate Bankruptcy," Journal of Finance, Vol. 23, No. 4, (Sep., 1968), pp. 589-609.

[2] Basel Committee on Banking Supervision, "Studies on the Validation of Internal Rating Systems," Bank for International Settlements (BIS), Working Papers No. 14, revised version, May 2005. Available at:

[3] Basel Committee on Banking Supervision, "International Convergence of Capital Measurement and Capital Standards: A Revised Framework," Bank for International Settlements (BIS), comprehensive version, June 2006. Available at:

[4] Loeffler, G., and P. N. Posch, Credit Risk Modeling Using Excel and VBA, West Sussex, England: Wiley Finance, 2007.

[5] Merton, R., "On the Pricing of Corporate Debt: The Risk Structure of Interest Rates," Journal of Finance, Vol. 29, No. 2, (May, 1974), pp. 449-70.