Minimax Game for Training Generative Adversarial Networks 10-07-2020 05-07-2023 blog 10 minutes read (About 1426 words)Some Intuitions to Training Generative Adversarial Networks Deep Learning, Generative Adversarial Networks, Optimization, GAN, Neural Networks Read More
Quantization for Neural Networks 05-17-2020 02-09-2023 article an hour read (About 6957 words)Mathematical Foundations to Neural Network Quantization Machine Learning, Deep Learning, Mathematics, Neural Network, Quantization, Matrix Multiplication Read More
PyTorch Distributed Training 04-26-2020 02-06-2022 blog 14 minutes read (About 2112 words)PyTorch Distributed Training for Dummies Deep Learning, PyTorch, Distributed Computing Read More
Gated Linear Units (GLU) and Gated CNN 03-09-2020 03-09-2020 blog 6 minutes read (About 827 words)Gated Linear Units and Gated CNN Explained Machine Learning, Deep Learning, Natural Language Processing Read More
Evolved Transformer Explained 03-07-2020 03-07-2020 blog 5 minutes read (About 777 words)Evolution Search on Transformer Neural Network Architectures Machine Learning, Deep Learning, Natural Language Processing, Transformer, Neural Architecture Search, Evolution Algorithm Read More
Save, Load and Inference From TensorFlow 2.x Frozen Graph 01-09-2020 01-09-2020 blog 6 minutes read (About 826 words)Continuing to Support Frozen Graph for TensorFlow 2.x Machine Learning, Deep Learning, TensorFlow Read More
TensorFlow Inference for Estimator 08-29-2019 08-29-2019 blog 9 minutes read (About 1392 words)Guidance for Fast TensorFlow Inference Deep Learning, TensorFlow Read More
PyTorch Model Export to ONNX Failed Due to ATen 07-03-2019 07-03-2019 blog 10 minutes read (About 1445 words)A Funny Story About PyTorch, ATen, and ONNX Deep Learning, Software Engineering, PyTorch, ATen, ONNX Read More
Dropout Explained 06-04-2019 06-04-2019 blog 8 minutes read (About 1155 words)The Math of Dropout You Know and Don't Know Machine Learning, Deep Learning Read More
Transformer Explained In One Single Page 06-01-2019 02-09-2023 blog 23 minutes read (About 3454 words)Attention is All You Need Mathematically Deep Learning, Natural Language Processing, Transformer, Machine Translation Read More
Layer Normalization Explained 05-31-2019 12-24-2024 blog 5 minutes read (About 707 words)Layer Normalization vs Batch Normalization vs Instance Normalization Deep Learning Read More
Data Parallelism VS Model Parallelism In Distributed Deep Learning Training 05-23-2019 05-23-2019 blog 7 minutes read (About 1067 words)Some Simple Logic and Theory Behind Deep Learning, Distributed System Read More