Cross Entropy, KL Divergence, and Maximum Likelihood Estimation 08-10-2019 08-10-2019 blog 6 minutes read (About 942 words)Some Theories for Machine Learning Optimization Machine Learning, Information Theory, Cross Entropy, KL Divergence, Maximum Likelihood Estimation Read more
Byte Pair Encoding 07-19-2019 07-19-2019 blog 16 minutes read (About 2377 words)Common Tokenization Method for Natural Language Processing Natural Language Processing, Information Theory, Byte Pair Encoding Read more
Entropy, Perplexity and Its Applications 07-17-2019 10-26-2020 blog 15 minutes read (About 2320 words)Foundations of Information Theory to Machine Learning Information Theory, Shannon Entropy, Perplexity Read more
Prefix-Free Code and Huffman Coding 03-17-2018 03-17-2018 blog 9 minutes read (About 1320 words)Understand Prefix-Free Code, Huffman Coding, and Try it Using Library Tools Information Theory Read more