Lei Mao's Log Book
Lei Mao's Log BookCurriculumBlogArticlesProjectsPublicationsReadingsLifeEssayPhotographyArchivesCategoriesTagsFAQs
  • Tags
  • Neural Network

Grouped Query Attention Performance Theoretical Analysis

 02-03-2025 03-02-2025 blog 11 minutes read (About 1612 words)
Sharing Key and Value Tensors for a Group of Query Tensors to Mitigate Transformer Attention Layer Performance Bottleneck

 
Deep Learning, 
Neural Network, 
Transformer, 
Computer Architecture, 
Performance Optimization, 
Large Language Model  
  Read More

Transformer Vanilla Attention Performance Theoretical Analysis

 01-27-2025 03-02-2025 blog 9 minutes read (About 1275 words)
Performance Bottleneck for Serving Transformer Models

 
Deep Learning, 
Neural Network, 
Transformer, 
Computer Architecture, 
Performance Optimization, 
Large Language Model  
  Read More

Neural Radiance Fields

 07-24-2024 07-24-2024 blog 6 minutes read (About 826 words)
Scene Representation and Differentiable Rendering with Neural Radiance Fields

 
Deep Learning, 
Computer Vision, 
Neural Network, 
NeRF, 
Neural Radiance Fields  
  Read More

LoRA and LoRAPrune

 07-11-2024 07-11-2024 blog 11 minutes read (About 1664 words)
Fine-Tuning and Pruning of Large Language Models Using Low-Rank Adaptation

 
Deep Learning, 
Neural Network, 
Neural Network Pruning, 
LoRA, 
LoRAPrune  
  Read More

PyTorch Variational Autoencoder

 06-14-2024 06-14-2024 blog 19 minutes read (About 2921 words)
Implementing Variational Autoencoder in PyTorch

 
Deep Learning, 
Statistics, 
Probability, 
Mathematics, 
Neural Network, 
Bayesian Inference, 
PyTorch, 
Variational Autoencoder, 
VAE  
  Read More

PyTorch Automatic Mixed Precision Training

 06-08-2024 06-08-2024 blog 3 minutes read (About 492 words)
Accelerating PyTorch Deep Learning Training with Automatic Mixed Precision

 
Deep Learning, 
Neural Network, 
PyTorch, 
Mixed Precision Training, 
Automatic Mixed Precision Training  
  Read More

PyTorch Pruning

 02-18-2021 02-18-2024 blog 12 minutes read (About 1834 words)
PyTorch Pruning for Creating Sparse Neural Networks

 
Deep Learning, 
Neural Network, 
PyTorch, 
Pruning  
  Read More

Quantization for Neural Networks

 05-17-2020 02-09-2023 article an hour read (About 6957 words)
Mathematical Foundations to Neural Network Quantization

 
Machine Learning, 
Deep Learning, 
Mathematics, 
Neural Network, 
Quantization, 
Matrix Multiplication  
  Read More
Lei Mao

Lei Mao

Artificial Intelligence Machine Learning Computer Science

Santa Clara, California

Posts

1173

Categories

8

Tags

742

  Follow   Sponsor

Advertisement


Categories

  • article20
  • blog531
  • essay296
  • life262
  • miscellaneous2
  • photography34
  • project20
  • reading8

follow.it

Recents

08-27-2025

Illegal Memory Access and Segmentation Fault

blog

08-24-2025

Alviso Marina County Park

photography

08-24-2025

Alviso Marina County Park 徒步

life

08-23-2025

Downtown Pleasanton

photography

08-23-2025

Marilyn Murphy Kane Trail

photography

Archives

  • August 202525
  • July 202523
  • June 202547
  • May 202527
  • April 202521
  • See All >>

Tags

Outdoors266
Hiking201
California197
CPP112
Mathematics98
Deep Learning82
CUDA57
Running51
Photography41
Software Engineering35
Machine Learning34
Python34
Racing33
Movie32
Statistics31
Linux30
Park30
China29
Docker26
Bird25
See All >>
Lei Mao's Log Book

© 2017-2025 Lei Mao  Powered by Hexo & Icarus
Site UV:  Site PV:

×