Copies of the classnotes are on the internet in PDF format as given below. The notes and supplements may contain hyperlinks to posted webpages; the links appear in red fonts. The "Examples and Exercises" files were prepared in Beamer. The "Printout of Examples" are printable PDF files of the Beamer slides without the pauses. These notes have not been classroom tested and may have typographical errors.

"Foundations of Predictive Analytics" is not yet a class at ETSU, but it is related to the M.S. program in Applied Data Science (MSADS). Details on this program can be found on the Masters in Applied Data Science Program Overview webpage (accessed 3/24/2024).

Preface. Preface notes

Chapter 1. Introduction.

• 1.1. What is a Model? Section 1.1 notes
• 1.2. What is a Statistical Model? Section 1.2 notes
• 1.3. The Modeling Process.
• 1.4. Modeling Pitfalls.
• 1.5. Characteristics of Good Modelers.
• 1.6. The Future of Predictive Analytics.
• Study Guide 1.

Chapter 4. Linear Modeling and Regression.

• 4.1. Properties of Maximum Likelihood Estimators.
• 4.2. Linear Regression.
• 4.3. Fisher's Linear Discriminant Analysis.
• 4.4. Principal Component Regression (PCR).
• 4.5. Factor Analysis.
• 4.6. Partial Least Squares Regression (PLSR).
• 4.7. Generalized Linear Model (GLM).
• 4.8. Logistic Regression: Binary.
• 4.9. Logistic Regression: Multinomial Nominal.
• 4.10. Logistic Regression: Proportional Multiple Ordinal.
• 4.11. Fisher Scoring Method for Logistic Regression.
• 4.12. Tobit Model: A Censored Regression Model.
• Study Guide 4.

Chapter 5. Nonlinear Modeling.

• 5.1. Naive Bayesian Classifier.
• 5.2. Neural Network.
• 5.3. Segmentation and Tree Models.
• 5.4. Additive Models.
• 5.5. Support Vector Machine (SVM).
• 5.6. Fuzzy Logic System.
• 5.7. Clustering.
• Study Guide 5.

Chapter 6. Time Series Analysis.

• 6.1. Fundamentals of Forecasting.
• 6.2. ARIMA Models.
• 6.3. Survival Data Analysis.
• 6.4. Exonential Weighted Moving Average (EWMA) and GARCH(1,1).
• Study Guide 6.

Chapter 7. Data Preparation and Variable Selection.

• 7.1. Data Quality and Exploration.
• 7.2. Variable Scaling and Transformation.
• 7.3. How to Bin Variables.
• 7.4. Interpolation in One and Two Dimensions.
• 7.5. Weight of Evidence (WOE) Transformations.
• 7.6. Variable Selection Overview.
• 7.7. Missing Data Imputation.
• 7.8. Stepwise Selection Methods.
• 7.9. Mutual Information, KL Distance.
• 7.10. Detection of Multicollinearity.
• Study Guide 7.

Chapter 8. Model of Goodness Measures.

• 8.1. Training, Testing, Validation.
• 8.2. Continuous Dependent Variable.
• 8.3. Binary Dependent Variable (Two-Group Classification).
• 8.4. Population Stability Index Using Relative Entropy.
• Study Guide 8.

Chapter 9. Optimization Methods.

• 9.1. Lagrange Multiplier.
• 9.2. Gradient Descent Method.
• 9.3. Newton-Raphson Method.
• 9.4. Conjugate Gradient Method.
• 9.5. Quasi-Newton Method.
• 9.6. Genetic Algorithms (GA).
• 9.7. Simulated Annealing.
• 9.8. Linear Programming.
• 9.9. Nonlinear Programming.
• 9.10. Nonlinear Equations.
• 9.11. Expectation-Maximization (EM) Algortihm.
• 9.12. Optimal Desing of Experiment.
• Study Guide 9.

Chapter 10. Miscellaneous Topics.

• 10.1. Multidimensional Scaling.
• 10.2. Simulation.
• 10.3. Odds Normalization and Score Transformation.
• 10.4. Reject Inference.
• 10.5. Dempster-Shafer Theory of Evidence.
• Study Guide 10.

Additional Chapters and Appendices

• Chapter 2. Properties of Statistical Distributions.
• Chapter 3. Important Matrix Relationships.
• Appendix A. Useful Mathematical Relationships.
• Appendix B. DataMinerXL - Microsoft Excel Add-In for Building Predictive Models.

Return to Bob Gardner's home page