In Deep Learning: A Practical Introduction, a team of distinguished researchers delivers a book complete with coverage of the theoretical and practical elements of deep learning. The book includes extensive examples, end-of-chapter exercises, homework, exam material, and a GitHub repository containing code and data for all provided examples.
Combining contemporary deep learning theory with state-of-the-art tools, the chapters are structured to maximize accessibility for both beginning and intermediate students. The authors have included coverage of TensorFlow, Keras, and Pytorch. Readers will also find: - Thorough introductions to deep learning and deep learning tools- Comprehensive explorations of convolutional neural networks, including discussions of their elements, operation, training, and architectures- Practical discussions of recurrent neural networks and non-supervised approaches to deep learning- Fulsome treatments of generative adversarial networks as well as deep Bayesian neural networks
Perfect for undergraduate and graduate students studying computer vision, computer science, artificial intelligence, and neural networks, Deep Learning: A Practical Introduction will also benefit practitioners and researchers in the fields of deep learning and machine learning in general.
Table of Contents
About the Authors xv
Foreword xvii
Preface xix
Acknowledgment xxi
About the Companion Website xxiii
1 The Multilayer Perceptron 1
1.1 Introduction 1
1.2 The Concept of Neuron 2
1.3 Structure of a Neural Network 14
1.4 Activations 21
1.5 Training a Multilayer Perceptron 22
1.6 Conclusion 37
2 Training Practicalities 41
2.1 Introduction 41
2.2 Generalization and Overfitting 42
2.3 Regularization Techniques 45
2.4 Normalization Techniques 50
2.5 Optimizers 52
2.6 Conclusion 58
3 Deep Learning Tools 61
3.1 Python: An Overview 61
3.2 NumPy 72
3.3 Matplotlib 83
3.4 Scipy 97
3.5 Scikit-Learn 107
3.6 Pandas 116
3.7 Seaborn 125
3.8 Python Libraries for NLP 131
3.9 TensorFlow 138
3.10 Keras 141
3.11 Pytorch 144
3.12 Conclusion 149
4 Convolutional Neural Networks 153
4.1 Introduction 153
4.2 Elements of a Convolutional Neural Network 153
4.3 Training a CNN 160
4.4 Extensions of the CNN 166
4.5 Conclusion 184
5 Recurrent Neural Networks 187
5.1 Introduction 187
5.2 RNN Architecture 188
5.3 Training an RNN 191
5.4 Long-Term Dependencies: Vanishing and Exploding Gradients 199
5.5 Deep RNN 201
5.6 Bidirectional RNN 203
5.7 Long Short-Term Memory Networks 204
5.8 Gated Recurrent Units 218
5.9 Conclusion 221
6 Attention Networks and Transformers 225
6.1 Introduction 225
6.2 Attention Mechanisms 227
6.3 Transformers 242
6.4 BERT 249
6.5 GPT-2 256
6.6.1 Comparison between ViTs and CNNs 264
6.7 Conclusion 269
7 Deep Unsupervised Learning I 273
7.1 Introduction 273
7.2 Restricted Boltzmann Machines 274
7.3 Deep Belief Networks 278
7.4 Autoencoders 279
7.5 Undercomplete Autoencoder 284
7.6 Sparse Autoencoder 285
7.7 Denoising Autoencoders 287
7.8 Convolutional Autoencoder 288
7.9 Variational Autoencoders 291
7.10 Conclusion 297
8 Deep Unsupervised Learning II 301
8.1 Introduction 301
8.2 Elements of GAN 303
8.3 Training a GAN 305
8.4 Wasserstein GAN 309
8.5 DCGAN 312
8.6 cGAN 316
8.7 CycleGAN 318
8.8 StyleGAN 323
8.9 StackGAN 328
8.10 Diffusion Models 333
8.11 Conclusion 338
9 Deep Bayesian Networks 341
9.1 Introduction 341
9.2 Bayesian Models 342
9.3 Bayesian Inference Methods for Deep Learning 344
9.4 Conclusion 352
Problems 353
List of Acronyms 355
Notation 359
Bibliography 365
Index 387
