Large Language Models (LLMs) have revolutionized AI applications, but their deployment often comes with challenges related to computational cost, memory usage, and latency. Optimizing LLMs is essential to make them more efficient while preserving accuracy. This talk explores three key optimization techniques: quantization, distillation, and parameter-efficient fine-tuning (PEFT). Quantization compresses model weights to lower precision, reducing memory footprint and accelerating inference. Distillation transfers knowledge from large models to smaller ones, maintaining performance with reduced computational requirements. PEFT methods, such as LoRA, Adapters, and Prefix-Tuning, enable task-specific fine-tuning with minimal additional parameters. We will discuss trade-offs, best practices, and real-world applications of these techniques in serving LLMs at scale. By the end of this talk, attendees will gain a clear understanding of how to optimize LLMs for their specific needs, whether for on-device deployment, cloud inference, or cost-efficient model adaptation.