Lei Mao's Log Book
Lei Mao's Log BookCurriculumBlogArticlesProjectsPublicationsReadingsLifeEssayPhotographyArchivesCategoriesTagsFAQs
  • Tags
  • Deep Learning

Transformer Vanilla Attention Performance Theoretical Analysis

 01-27-2025 03-02-2025 blog 9 minutes read (About 1275 words)
Performance Bottleneck for Serving Transformer Models

 
Deep Learning, 
Transformer, 
Computer Architecture, 
Neural Network, 
Performance Optimization, 
Large Language Model  
  Read More

AWQ: Activation-Aware Weight Quantization

 01-01-2025 01-01-2025 blog 18 minutes read (About 2738 words)
Same Performance as Group-Wise Weight-Only Quantization But with Better Accuracy

 
Deep Learning, 
Mathematics, 
CUDA, 
Quantization, 
Accelerated Computing  
  Read More

NeurIPS 2024 Area Chair Experience

 12-26-2024 12-26-2024 blog 9 minutes read (About 1389 words)
First Time Serving as NeurIPS Area Chair

 
Deep Learning, 
NeurIPS, 
Conference  
  Read More

Neural Radiance Fields

 07-24-2024 07-24-2024 blog 6 minutes read (About 826 words)
Scene Representation and Differentiable Rendering with Neural Radiance Fields

 
Deep Learning, 
Computer Vision, 
Neural Network, 
NeRF, 
Neural Radiance Fields  
  Read More

LoRA and LoRAPrune

 07-11-2024 07-11-2024 blog 11 minutes read (About 1664 words)
Fine-Tuning and Pruning of Large Language Models Using Low-Rank Adaptation

 
Deep Learning, 
Neural Network, 
Neural Network Pruning, 
LoRA, 
LoRAPrune  
  Read More

Parameter Importance Approximation Via Taylor Expansion In Neural Network Pruning

 06-20-2024 07-07-2024 blog 12 minutes read (About 1778 words)
Scoring Neural Network Parameter Importance Faster

 
Deep Learning, 
Mathematics, 
Taylor Expansion, 
Neural Network Pruning  
  Read More

PyTorch Variational Autoencoder

 06-14-2024 06-14-2024 blog 19 minutes read (About 2921 words)
Implementing Variational Autoencoder in PyTorch

 
Deep Learning, 
Probability, 
Mathematics, 
Statistics, 
Bayesian Inference, 
PyTorch, 
Neural Network, 
Variational Autoencoder, 
VAE  
  Read More

PyTorch Automatic Mixed Precision Training

 06-08-2024 06-08-2024 blog 3 minutes read (About 492 words)
Accelerating PyTorch Deep Learning Training with Automatic Mixed Precision

 
Deep Learning, 
PyTorch, 
Neural Network, 
Mixed Precision Training, 
Automatic Mixed Precision Training  
  Read More

PyTorch Eager Mode Quantization TensorRT Acceleration

 05-24-2024 05-24-2024 blog 7 minutes read (About 1051 words)
TensorRT Acceleration for PyTorch Native Eager Mode Quantization Models

 
Deep Learning, 
Python, 
Inference, 
TensorRT, 
PyTorch, 
Quantization, 
NVIDIA, 
Accelerated Computing, 
GPU  
  Read More

TensorRT Python Inference

 05-18-2024 05-18-2024 blog 12 minutes read (About 1843 words)
TensorRT Python Inference Example

 
Deep Learning, 
Python, 
Inference, 
TensorRT, 
NVIDIA, 
GPU  
  Read More

Reparameterization Trick

 05-08-2024 05-08-2024 blog 12 minutes read (About 1749 words)
How Distribution of a Random Variable Changes when the Variable is Transformed in a Deterministic Way

 
Deep Learning, 
Probability, 
Mathematics, 
Statistics, 
Jacobian, 
Variational Autoencoder  
  Read More

Quantization Unit Test

 03-25-2024 03-25-2024 blog 5 minutes read (About 748 words)
How To Unit Test Quantization Implementation

 
Deep Learning, 
Mathematics, 
Software Engineering, 
Quantization, 
Unit Test  
  Read More
Previous
Next
  • 1
  • 2
  • 3
  • …
  • 8
Lei Mao

Lei Mao

Artificial Intelligence Machine Learning Computer Science

Menlo Park, California

Posts

1341

Categories

8

Tags

807

  Follow   Sponsor

Advertisement


Categories

  • article21
  • blog569
  • essay340
  • life310
  • miscellaneous2
  • photography71
  • project20
  • reading8

follow.it

Recents

04-22-2026

How Is FARS, The Fully Automated Research System?

blog

04-22-2026

算计: 七天的死亡游戏

essay

04-18-2026

Lake Chabot Regional Park 徒步

life

04-18-2026

Lake Chabot Regional Park

photography

04-16-2026

2023 年恐怖电影《感恩节》

essay

Archives

  • April 202614
  • March 202618
  • February 202617
  • January 202616
  • December 202536
  • See All >>

Tags

Outdoors315
California246
Hiking239
CPP121
Mathematics102
Deep Learning86
Photography85
CUDA74
Running68
Wildlife62
Bird56
Racing45
Movie37
Python36
Software Engineering36
Machine Learning34
NVIDIA32
Statistics32
China31
Linux31
See All >>
Lei Mao's Log Book

© 2017-2026 Lei Mao  Powered by Hexo & Icarus
Site UV:  Site PV:

×