Watch the intro video
Note: if you can't see the video, you might need to allow cookies or disable the add blocker.
Soledad Galli, PhD
Instructor
Sole is a lead data scientist, instructor and developer of open source software. She created and maintains the Python library for feature engineering Feature-engine, which allows us to impute data, encode categorical variables, transform, create and select features. Sole is also the author of the book "Python Feature engineering Cookbook" by Packt editorial.
Course Description
Welcome to Hyperparameter Optimization for Machine Learning. In this course, you will learn multiple techniques to select the best hyperparameters and improve the performance of your machine learning models.
If you are regularly training machine learning models as a hobby or for your organization and want to improve the performance of your models, if you are keen to jump up in the leader board of a data science competition, or you simply want to learn more about how to tune hyperparameters of machine learning models, this course will show you how.
We'll take you step-by-step through engaging video tutorials and teach you everything you need to know about hyperparameter tuning. Throughout this comprehensive course, we cover almost every available approach to optimize hyperparameters, discussing their rationale, their advantages and shortcomings, the considerations to have when using the technique and their implementation in Python.
Specifically, you will learn:
- What hyperparameters are and why tuning matters
- The use of cross-validation and nested cross-validation for optimization
- Grid search and Random search for hyperparameters
- Bayesian Optimization
- Tree-structured Parzen estimators
- SMAC, Population Based Optimization and other SMBO algorithms
- How to implement these techniques with available open source packages including Hyperopt, Optuna, Scikit-optimize, Keras Turner and others.
By the end of the course, you will be able to decide which approach you would like to follow and carry it out with available open-source libraries.
This comprehensive machine learning course includes over 50 lectures spanning about 8 hours of video, and ALL topics include hands-on Python code examples which you can use for reference and for practice, and re-use in your own projects.
Example Curriculum
- Cross-Validation (9:15)
- Bias vs Variance (Optional)
- Cross-Validation schemes (13:55)
- Estimating the model generalization error with CV - Demo (8:35)
- Cross-Validation for Hyperparameter Tuning - Demo (7:33)
- Special Cross-Validation schemes (7:07)
- Group Cross-Validation - Demo (5:03)
- Nested Cross-Validation (7:19)
- Nested Cross-Validation - Demo (6:43)
- Basic Search Algorithms - Introduction (5:10)
- Manual Search (6:35)
- Grid Search (3:21)
- Grid Search - Demo (7:50)
- Grid Search with different hyperparameter spaces (2:18)
- Random Search (7:34)
- Random Search with Scikit-learn (5:37)
- Random Search with Scikit-Optimize (7:30)
- Random Search with Hyperopt (11:06)
- More examples
- Sequential Search (5:49)
- Bayesian Optimization (5:10)
- Bayesian Inference - Introduction (7:11)
- Joint and Conditional Probabilities (7:40)
- Bayes Rule (12:02)
- Sequential Model-Based Optimization (15:54)
- Gaussian Distribution (7:28)
- Multivariate Gaussian Distribution (16:22)
- Gaussian Process (14:47)
- Kernels (6:41)
- Acquisition Functions (13:44)
- Additional Reading Resources
- Scikit-Optimize - 1-Dimension (14:11)
- Scikit-Optimize - Manual Search (5:20)
- Scikit-Optimize - Automatic Search (4:03)
- Scikit-Optimize - Alternative Kernel (3:24)
- Scikit-Optimize - Neuronal Networks (14:17)
- Scikit-Optimize - CNN - Search Analysis (6:00)
- Scikit-Optimize (5:45)
- Section content (2:10)
- Hyperparameter Distributions (4:37)
- Defining the hyperparameter space (2:36)
- Defining the objective function (1:59)
- Random search (5:12)
- Bayesian search with Gaussian processes (5:14)
- Bayesian search with Random Forests (2:53)
- Bayesian search with GBMs (3:03)
- Parallelizing a Bayesian search (2:53)
- Bayesian search with Scikit-learn wrapper (4:03)
- Changing the kernel of a Gaussian Process (3:24)
- Optimizing xgboost
- Optimizing Hyperparameters of a CNN (14:17)
- Analyzing the CNN search (6:00)
- Optuna (4:58)
- Optuna main functions (7:45)
- Section content (1:00)
- Search algorithms (7:38)
- Optimizing multiple ML models with simultaneously (7:21)
- Optimizing hyperparameters of a CNN (9:52)
- Optimizing a CNN - extended (4:48)
- Evaluating the search with Optuna's built in functions (9:41)
- References
- More examples