11.5 Videos
Example Video 1. Preliminary Evaluation
This video describes the preliminary process of examining the correlation matrix for the strength of the correlation between numeric input variables and a categorical outcome variable. The binary outcome variable is converted to a numeric value so that correlations between it and the numeric input variables can be calculated.
The KNN algorithm uses the validation data to determine which k produces the best result. This means there is a possibility that the validation process introduces overfitting. To test for this, requires three data partitions: training, validation, and test. Then, KNN model is created and the three partitions are used to determine whether overfitting is being introduced during the validation process. In this example, overfitting is not being introducted by the validation process. Therefore, two data partitions, training and validation partitions, are sufficient for model comparisons.
You may use this data to walk through the process with the next two videos if you like.
Example Video 2. Comparing categorical prediction models using a template
This video shows an example of comparing models produced by KNN and Logistic Regression and how the results can be put into a template that facilitates clearer comparison of results across models. A template like this will be used in the second homework problem. So this video shows examples of how the template can be used to enter and evaluate results across multiple models.
Homework Problem 2, Video 1. Correlation matrix and collinearity
This video walks through the first part of homework problem 2. This video describes the preliminary process of examining the correlation matrix and looking for collinearity. In this example, collinearity exists. Explanation is provided on how to deal with collinearity.
Homework Problem 2, Video 2. Training, Validation, and Test Partitions when overfitting during validation exists
In this video, a three-way parition is used. In this example, there is evidence of overfitting being introducted by the validation process. An explanation is given of what to do in this circumstance.