Sharing Key and Value Tensors for a Group of Query Tensors to Mitigate Transformer Attention Layer Performance Bottleneck

Performance Bottleneck for Serving Transformer Models

Attention with Learned Spatial Feature Sampling

Generative Pre-Trained Transformer Models From OpenAI

Principles for Faster Transformer Inference

PyTorch Dynamic Quantization for Transformers

Evolution Search on Transformer Neural Network Architectures

Attention is All You Need Mathematically