Optimal Codeword Length 08-18-2024 08-18-2024 blog 8 minutes read (About 1255 words)Information Content, Shannon Entropy, and Optimal Codeword Length Information Theory, Shannon Entropy, Shannon Information, Optimal Codeword Length, Code Read More
Cross Entropy, KL Divergence, and Maximum Likelihood Estimation 08-10-2019 08-10-2019 blog 6 minutes read (About 942 words)Some Theories for Machine Learning Optimization Machine Learning, Information Theory, Cross Entropy, KL Divergence, Maximum Likelihood Estimation Read More
Byte Pair Encoding 07-19-2019 07-19-2019 blog 16 minutes read (About 2377 words)Common Tokenization Method for Natural Language Processing Information Theory, Natural Language Processing, Byte Pair Encoding Read More
Entropy, Perplexity and Its Applications 07-17-2019 12-21-2023 blog 15 minutes read (About 2316 words)Foundations of Information Theory to Machine Learning Information Theory, Shannon Entropy, Perplexity, Entropy Read More
Prefix-Free Code and Huffman Coding 03-17-2018 03-17-2018 blog 9 minutes read (About 1322 words)Understand Prefix-Free Code, Huffman Coding, and Try it Using Library Tools Information Theory Read More