Credit Risk Classification¶

Credit risk poses a classification problem that’s inherently imbalanced. This is because healthy loans easily outnumber risky loans. In this Challenge, you’ll use various techniques to train and evaluate models with imbalanced classes. You’ll use a dataset of historical lending activity from a peer-to-peer lending services company to build a model that can identify the creditworthiness of borrowers.

Instructions:¶

This challenge consists of the following subsections:

Split the Data into Training and Testing Sets
Create a Logistic Regression Model with the Original Data
Predict a Logistic Regression Model with Resampled Training Data

Split the Data into Training and Testing Sets¶

Open the starter code notebook and then use it to complete the following steps.

Read the lending_data.csv data from the Resources folder into a Pandas DataFrame.
Create the labels set (y) from the “loan_status” column, and then create the features (X) DataFrame from the remaining columns.

Note A value of 0 in the “loan_status” column means that the loan is healthy. A value of 1 means that the loan has a high risk of defaulting.
Check the balance of the labels variable (y) by using the value_counts function.
Split the data into training and testing datasets by using train_test_split.

Create a Logistic Regression Model with the Original Data¶

Employ your knowledge of logistic regression to complete the following steps:

Fit a logistic regression model by using the training data (X_train and y_train).
Save the predictions on the testing data labels by using the testing feature data (X_test) and the fitted model.
Evaluate the model’s performance by doing the following:
- Calculate the accuracy score of the model.
- Generate a confusion matrix.
- Print the classification report.
Answer the following question: How well does the logistic regression model predict both the 0 (healthy loan) and 1 (high-risk loan) labels?

Predict a Logistic Regression Model with Resampled Training Data¶

Did you notice the small number of high-risk loan labels? Perhaps, a model that uses resampled data will perform better. You’ll thus resample the training data and then reevaluate the model. Specifically, you’ll use RandomOverSampler.

To do so, complete the following steps:

Use the RandomOverSampler module from the imbalanced-learn library to resample the data. Be sure to confirm that the labels have an equal number of data points.
Use the LogisticRegression classifier and the resampled data to fit the model and make predictions.
Evaluate the model’s performance by doing the following:
- Calculate the accuracy score of the model.
- Generate a confusion matrix.
- Print the classification report.
Answer the following question: How well does the logistic regression model, fit with oversampled data, predict both the 0 (healthy loan) and 1 (high-risk loan) labels?

Write a Credit Risk Analysis Report¶

For this section, you’ll write a brief report that includes a summary and an analysis of the performance of both machine learning models that you used in this challenge. You should write this report as the README.md file included in your GitHub repository.

Structure your report by using the report template that Starter_Code.zip includes, and make sure that it contains the following:

An overview of the analysis: Explain the purpose of this analysis.
The results: Using bulleted lists, describe the balanced accuracy scores and the precision and recall scores of both machine learning models.
A summary: Summarize the results from the machine learning models. Compare the two versions of the dataset predictions. Include your recommendation for the model to use, if any, on the original vs. the resampled data. If you don’t recommend either model, justify your reasoning.

In [5]:

# Import the modules
import numpy as np
import pandas as pd
from pathlib import Path
from sklearn.metrics import balanced_accuracy_score
from sklearn.metrics import confusion_matrix
from imblearn.metrics import classification_report_imbalanced

import warnings
warnings.filterwarnings('ignore')

Split the Data into Training and Testing Sets¶

Step 1: Read the lending_data.csv data from the Resources folder into a Pandas DataFrame.¶

In [6]:

# Read the CSV file from the Resources folder into a Pandas DataFrame
file_path = Path("./Resources/lending_data.csv")
df_lending = pd.read_csv(file_path)

# Review the DataFrame
df_lending.head()

Out[6]:

	loan_size	interest_rate	borrower_income	debt_to_income	num_of_accounts	derogatory_marks	total_debt
0	10700.0	7.672	52800	0.431818	5	1	22800
1	8400.0	6.692	43600	0.311927	3	0	13600
2	9000.0	6.963	46100	0.349241	3	0	16100
3	10700.0	7.664	52700	0.430740	5	1	22700
4	10800.0	7.698	53000	0.433962	5	1	23000

Step 2: Create the labels set (y) from the “loan_status” column, and then create the features (X) DataFrame from the remaining columns.¶

In [7]:

# Separate the data into labels and features

# Separate the y variable, the labels
y = df_lending['loan_status']

# Separate the X variable, the features
X = df_lending.drop(columns="loan_status")

In [8]:

# Review the y variable Series
y[:5]

Out[8]:

0    0
1    0
2    0
3    0
4    0
Name: loan_status, dtype: int64

In [9]:

# Review the X variable DataFrame
X.head()

Out[9]:

	loan_size	interest_rate	borrower_income	debt_to_income	num_of_accounts	derogatory_marks	total_debt
0	10700.0	7.672	52800	0.431818	5	1	22800
1	8400.0	6.692	43600	0.311927	3	0	13600
2	9000.0	6.963	46100	0.349241	3	0	16100
3	10700.0	7.664	52700	0.430740	5	1	22700
4	10800.0	7.698	53000	0.433962	5	1	23000

Step 3: Check the balance of the labels variable (y) by using the value_counts function.¶

In [10]:

# Check the balance of our target values
y.value_counts()

Out[10]:

loan_status
0    75036
1     2500
Name: count, dtype: int64

Step 4: Split the data into training and testing datasets by using train_test_split.¶

In [11]:

# Import the train_test_learn module
from sklearn.model_selection import train_test_split

# Split the data using train_test_split
# Assign a random_state of 1 to the function
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1)

Create a Logistic Regression Model with the Original Data¶

Step 1: Fit a logistic regression model by using the training data (X_train and y_train).¶

In [12]:

# Import the LogisticRegression module from SKLearn
from sklearn.linear_model import LogisticRegression

# Instantiate the Logistic Regression model
# Assign a random_state parameter of 1 to the model
logistic_regression_model = LogisticRegression(random_state=1)

# Fit the model using training data
lr_model = logistic_regression_model.fit(X_train, y_train)

Step 2: Save the predictions on the testing data labels by using the testing feature data (X_test) and the fitted model.¶

In [13]:

# Make a prediction using the testing data
testing_predictions = logistic_regression_model.predict(X_test)

Step 3: Evaluate the model’s performance by doing the following:¶

Calculate the accuracy score of the model.
Generate a confusion matrix.
Print the classification report.

In [14]:

# Print the balanced_accuracy score of the model
from sklearn.metrics import accuracy_score
accuracy_score(y_test, testing_predictions)

Out[14]:

0.9924680148576145

In [15]:

# Generate a confusion matrix for the model
test_matrix = confusion_matrix(y_test, testing_predictions)
print(test_matrix)

[[18655   110]
 [   36   583]]

In [16]:

# Print the classification report for the model
from sklearn.metrics import classification_report
testing_report = classification_report(y_test, testing_predictions)
print(testing_report)

              precision    recall  f1-score   support

           0       1.00      0.99      1.00     18765
           1       0.84      0.94      0.89       619

    accuracy                           0.99     19384
   macro avg       0.92      0.97      0.94     19384
weighted avg       0.99      0.99      0.99     19384

Step 4: Answer the following question.¶

Question: How well does the logistic regression model predict both the 0 (healthy loan) and 1 (high-risk loan) labels?

Answer: The model predicts the healthy loan (0) very well with high precision (1.00) and recall (0.99). The model doesn't do as great a job at predicting the high-risk loan. It doesn't predict as well whether the high risk loan is actually a high risk loan with .85 precision (has some False Positives) and doesn't do asgood job at identifying when a loan isn't a high risk loan wtih .91 recall (has some False Negatives).

Predict a Logistic Regression Model with Resampled Training Data¶

Step 1: Use the RandomOverSampler module from the imbalanced-learn library to resample the data. Be sure to confirm that the labels have an equal number of data points.¶

In [17]:

# Import the RandomOverSampler module form imbalanced-learn
from imblearn.over_sampling import RandomOverSampler

# Instantiate the random oversampler model
# # Assign a random_state parameter of 1 to the model
ros = RandomOverSampler(random_state=1)

# Fit the original training data to the random_oversampler model
X_resampled, y_resampled = ros.fit_resample(X_train, y_train)

In [18]:

# Count the distinct values of the resampled labels data
y_resampled.value_counts()

Out[18]:

loan_status
0    56271
1    56271
Name: count, dtype: int64

Step 2: Use the LogisticRegression classifier and the resampled data to fit the model and make predictions.¶

In [24]:

# Instantiate the Logistic Regression model
# Assign a random_state parameter of 1 to the model
logistic_regression_model = LogisticRegression(random_state=1)

# Fit the model using the resampled training data
lr_model = logistic_regression_model.fit(X_resampled, y_resampled)

# Make a prediction using the testing data
testing_predictions = logistic_regression_model.predict(X_test)

Step 3: Evaluate the model’s performance by doing the following:¶

Calculate the accuracy score of the model.
Generate a confusion matrix.
Print the classification report.

In [25]:

# Print the balanced_accuracy score of the model 
accuracy_score(y_test, testing_predictions)

Out[25]:

0.9936545604622369

In [26]:

# Generate a confusion matrix for the model
test_matrix = confusion_matrix(y_test, testing_predictions)
print(test_matrix)

[[18646   119]
 [    4   615]]

In [27]:

# w/o resampling
# [[18663   102]
# [   56   563]]

In [28]:

# Print the classification report for the model
testing_report = classification_report(y_test, testing_predictions)
print(testing_report)

              precision    recall  f1-score   support

           0       1.00      0.99      1.00     18765
           1       0.84      0.99      0.91       619

    accuracy                           0.99     19384
   macro avg       0.92      0.99      0.95     19384
weighted avg       0.99      0.99      0.99     19384

Step 4: Answer the following question¶

Question: How well does the logistic regression model, fit with oversampled data, predict both the 0 (healthy loan) and 1 (high-risk loan) labels?

Answer: The model predicts the healthy loan (0) very well with high precision (1.00) and recall (0.99). The model doesn't do as great a job at predicting whether the high risk loan is actually a high risk loan with .84 precision (has some False Positives) but does do a good job at identifying when a loan isn't a high risk loan wtih .99 recall (very few False Negatives).

In [ ]:

Credit Risk

Credit Risk Classification¶

Instructions:¶

Split the Data into Training and Testing Sets¶

Create a Logistic Regression Model with the Original Data¶

Predict a Logistic Regression Model with Resampled Training Data¶

Write a Credit Risk Analysis Report¶

Split the Data into Training and Testing Sets¶

Step 1: Read the lending_data.csv data from the Resources folder into a Pandas DataFrame.¶

Step 2: Create the labels set (y) from the “loan_status” column, and then create the features (X) DataFrame from the remaining columns.¶

Step 3: Check the balance of the labels variable (y) by using the value_counts function.¶

Step 4: Split the data into training and testing datasets by using train_test_split.¶

Create a Logistic Regression Model with the Original Data¶

Step 1: Fit a logistic regression model by using the training data (X_train and y_train).¶

Step 2: Save the predictions on the testing data labels by using the testing feature data (X_test) and the fitted model.¶

Step 3: Evaluate the model’s performance by doing the following:¶

Step 4: Answer the following question.¶

Predict a Logistic Regression Model with Resampled Training Data¶

Step 1: Use the RandomOverSampler module from the imbalanced-learn library to resample the data. Be sure to confirm that the labels have an equal number of data points.¶

Step 2: Use the LogisticRegression classifier and the resampled data to fit the model and make predictions.¶

Step 3: Evaluate the model’s performance by doing the following:¶

Step 4: Answer the following question¶

Previous PostML Trading Bot

Next PostA Whale Off the Port(folio)