Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow

Transformation Pipelines
Select and Train a Model
Train and Evaluate on the Training Set
Better Evaluation Using Cross-Validation
Fine-Tune Your Model
Grid Search
Randomized Search
Ensemble Methods
Analyzing the Best Models and Their Errors
Evaluate Your System on the Test Set
Launch, Monitor, and Maintain Your System
Try It Out!
Exercises

3. Classification
MNIST
Training a Binary Classifier
Performance Measures
Measuring Accuracy Using Cross-Validation
Confusion Matrices
Precision and Recall
The Precision/Recall Trade-off
The ROC Curve
Multiclass Classification
Error Analysis
Multilabel Classification
Multioutput Classification
Exercises

4. Training Models
Linear Regression
The Normal Equation
Computational Complexity
Gradient Descent
Batch Gradient Descent
Stochastic Gradient Descent
Mini-Batch Gradient Descent
Polynomial Regression
Learning Curves
Regularized Linear Models
Ridge Regression
Lasso Regression
Elastic Net Regression
Early Stopping
Logistic Regression
Estimating Probabilities
Training and Cost Function
Decision Boundaries
Softmax Regression
Exercises
5. Support Vector Machines
Linear SVM Classification
Soft Margin Classification
Nonlinear SVM Classification
Polynomial Kernel
Similarity Features
Gaussian RBF Kernel
SVM Classes and Computational Complexity
SVM Regression
Under the Hood of Linear SVM Classifiers
The Dual Problem
Kernelized SVMs
Exercises

6. Decision Trees
Training and Visualizing a Decision Tree
Making Predictions
Estimating Class Probabilities
The CART Training Algorithm
Computational Complexity
Gini Impurity or Entropy?
Regularization Hyperparameters
Regression
Sensitivity to Axis Orientation
Decision Trees Have a High Variance
Exercises

7. Ensemble Learning and Random Forests
Voting Classifiers
Bagging and Pasting
Bagging and Pasting in Scikit-Learn
Out-of-Bag Evaluation
Random Patches and Random Subspaces
Random Forests
Extra-Trees
Feature Importance
Boosting
AdaBoost
Gradient Boosting
Histogram-Based Gradient Boosting
Stacking
Exercises

8. Dimensionality Reduction
The Curse of Dimensionality
Main Approaches for Dimensionality Reduction
Projection
Manifold Learning
PCA
Preserving the Variance
Principal Components
Projecting Down to d Dimensions
Using Scikit-Learn
Explained Variance Ratio
Choosing the Right Number of Dimensions
PCA for Compression
Randomized PCA
Incremental PCA
Random Projection
LLE
Other Dimensionality Reduction Techniques
Exercises

9. Unsupervised Learning Techniques
Clustering Algorithms: k-means and DBSCAN
k-means
Limits of k-means
Using Clustering for Image Segmentation
Using Clustering for Semi-Supervised Learning
DBSCAN
Other Clustering Algorithms
Gaussian Mixtures
Using Gaussian Mixtures for Anomaly Detection
Selecting the Number of Clusters
Bayesian Gaussian Mixture Models
Other Algorithms for Anomaly and Novelty Detection
Exercises

Part II. Neural Networks and Deep Learning
10. Introduction to Artificial Neural Networks with Keras
From Biological to Artificial Neurons
Biological Neurons
Logical Computations with Neurons
The Perceptron
The Multilayer Perceptron and Backpropagation
Regression MLPs
Classification MLPs
Implementing MLPs with Keras
Building an Image Classifier Using the Sequential API
Building a Regression MLP Using the Sequential API
Building Complex Models Using the Functional API
Using the Subclassing API to Build Dynamic Models
Saving and Restoring a Model
Using Callbacks
Using TensorBoard for Visualization
Fine-Tuning Neural Network Hyperparameters
Number of Hidden Layers
Number of Neurons per Hidden Layer
Learning Rate, Batch Size, and Other Hyperparameters
Exercises

11. Training Deep Neural Networks
The Vanishing/Exploding Gradients Problems
Glorot and He Initialization
Better Activation Functions
Batch Normalization
Gradient Clipping
Reusing Pretrained Layers
Transfer Learning with Keras
Unsupervised Pretraining
Pretraining on an Auxiliary Task
Faster Optimizers
Momentum
Nesterov Accelerated Gradient
AdaGrad
RMSProp
Adam
AdaMax
Nadam
AdamW
Learning Rate Scheduling
Avoiding Overfitting Through Regularization
Dropout
Monte Carlo (MC) Dropout
Max-Norm Regularization
Summary and Practical Guidelines
Exercises

12. Custom Models and Training with TensorFlow
A Quick Tour of TensorFlow
Using TensorFlow like NumPy
Tensors and Operations
Tensors and NumPy
Type Conversions
Variables
Other Data Structures
Customizing Models and Training Algorithms
Custom Loss Functions
Saving and Loading Models That Contain Custom Components
Custom Activation Functions, Initializers, Regularizers, and Constraints
Custom Metrics
Custom Layers
Custom Models
Losses and Metrics Based on Model Internals
Computing Gradients Using Autodiff
Custom Training Loops
TensorFlow Functions and Graphs
AutoGraph and Tracing
TF Function Rules
Exercises

13. Loading and Preprocessing Data with TensorFlow
The tf.data API
Chaining Transformations
Shuffling the Data
Interleaving Lines from Multiple Files
Preprocessing the Data
Putting Everything Together
Prefetching
Using the Dataset with Keras
The TFRecord Format
Compressed TFRecord Files
A Brief Introduction to Protocol Buffers
TensorFlow Protobufs
Loading and Parsing Examples
Handling Lists of Lists Using the SequenceExample Protobuf
Keras Preprocessing Layers
The Normalization Layer
The Discretization Layer
The CategoryEncoding Layer
The StringLookup Layer
The Hashing Layer
Encoding Categorical Features Using Embeddings
Text Preprocessing
Using Pretrained Language Model Components
Image Preprocessing Layers
The TensorFlow Datasets Project
Exercises

14. Deep Computer Vision Using Convolutional Neural Networks
The Architecture of the Visual Cortex
Convolutional Layers
Filters
Stacking Multiple Feature Maps
Implementing Convolutional Layers with Keras
Memory Requirements
Pooling Layers
Implementing Pooling Layers with Keras
CNN Architectures
LeNet-5
AlexNet
GoogLeNet
VGGNet
ResNet
Xception
SENet
Other Noteworthy Architectures
Choosing the Right CNN Architecture
Implementing a ResNet-34 CNN Using Keras
Using Pretrained Models from Keras
Pretrained Models for Transfer Learning
Classification and Localization
Object Detection
Fully Convolutional Networks
You Only Look Once
Object Tracking
Semantic Segmentation
Exercises

15. Processing Sequences Using RNNs and CNNs
Recurrent Neurons and Layers
Memory Cells
Input and Output Sequences
Training RNNs
Forecasting a Time Series
The ARMA Model Family
Preparing the Data for Machine Learning Models
Forecasting Using a Linear Model
Forecasting Using a Simple RNN
Forecasting Using a Deep RNN
Forecasting Multivariate Time Series
Forecasting Several Time Steps Ahead
Forecasting Using a Sequence-to-Sequence Model
Handling Long Sequences
Fighting the Unstable Gradients Problem
Tackling the Short-Term Memory Problem
Exercises
16. Natural Language Processing with RNNs and Attention
Generating Shakespearean Text Using a Character RNN
Creating the Training Dataset
Building and Training the Char-RNN Model
Generating Fake Shakespearean Text
Stateful RNN
Sentiment Analysis
Masking
Reusing Pretrained Embeddings and Language Models
An Encodera Decoder Network for Neural Machine Translation
Bidirectional RNNs
Beam Search
Attention Mechanisms
Attention Is All You Need: The Original Transformer Architecture
An Avalanche of Transformer Models
Vision Transformers
Hugging Faceâs Transformers Library
Exercises

17. Autoencoders, GANs, and Diffusion Models
Efficient Data Representations
Performing PCA with an Undercomplete Linear Autoencoder
Stacked Autoencoders
Implementing a Stacked Autoencoder Using Keras
Visualizing the Reconstructions
Visualizing the Fashion MNIST Dataset
Unsupervised Pretraining Using Stacked Autoencoders
Tying Weights
Training One Autoencoder at a Time
Convolutional Autoencoders
Denoising Autoencoders
Sparse Autoencoders
Variational Autoencoders
Generating Fashion MNIST Images
Generative Adversarial Networks
The Difficulties of Training GANs
Deep Convolutional GANs
Progressive Growing of GANs
StyleGANs
Diffusion Models
Exercises

18. Reinforcement Learning
Learning to Optimize Rewards
Policy Search
Introduction to OpenAI Gym
Neural Network Policies
Evaluating Actions: The Credit Assignment Problem
Policy Gradients
Markov Decision Processes
Temporal Difference Learning
Q-Learning
Exploration Policies
Approximate Q-Learning and Deep Q-Learning
Implementing Deep Q-Learning
Deep Q-Learning Variants
Fixed Q-value Targets
Double DQN
Prioritized Experience Replay
Dueling DQN
Overview of Some Popular RL Algorithms
Exercises

19. Training and Deploying TensorFlow Models at Scale
Serving a TensorFlow Model
Using TensorFlow Serving
Creating a Prediction Service on Vertex AI
Running Batch Prediction Jobs on Vertex AI
Deploying a Model to a Mobile or Embedded Device
Running a Model in a Web Page
Using GPUs to Speed Up Computations
Getting Your Own GPU
Managing the GPU RAM
Placing Operations and Variables on Devices
Parallel Execution Across Multiple Devices
Training Models Across Multiple Devices
Model Parallelism
Data Parallelism
Training at Scale Using the Distribution Strategies API
Training a Model on a TensorFlow Cluster
Running Large Training Jobs on Vertex AI
Hyperparameter Tuning on Vertex AI
Exercises
Thank You!

A. Machine Learning Project Checklist
B. Autodiff
C. Special Data Structures
D. TensorFlow Graphs

Index
About the Author