1. Llama 2: Open Foundation and Fine-Tuned Chat Models 링크: https://arxiv.org/abs/2307.09288 2. Overview I.배경 II.llama의 주어진 과제 III.핵심기술 - a.RoPE - b.RMSNORM(Pre-Norm) - c.GQA - d.SwiGLU IV. Fine tuning - a.SFT - b.RLHF - c.GAtt V.결론 3. Reference I. Attention Is All You Need 링크: https://arxiv.org/abs/1706.03762 II. Language Models are Few-Shot Learners 링크: https://arxiv.org/abs/2005.14165 III. LLaMA: Open and Efficient Foundation Language Models 링크: https://arxiv.org/abs/2302.13971 IV. Llama 2: Open Foundation and Fine-Tuned Chat Models 링크: https://arxiv.org/abs/2307.09288 V. RoFormer: Enhanced Transformer with Rotary Position Embedding 링크: https://arxiv.org/abs/2104.09864 VI. Layer Normalization 링크: https://arxiv.org/abs/1607.06450 VII. Root Mean Square Layer Normalization 링크: https://arxiv.org/abs/1910.07467 VIII. Fast Transformer Decoding: One Write-Head is All You Need 링크: https://arxiv.org/abs/1911.02150 IX. GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints 링크: https://arxiv.org/abs/2305.13245 X. Rectified Linear Units Improve Restricted Boltzmann Machines 링크: https://www.cs.toronto.edu/~fritz/absff/reluICML.pdf XI. Gaussian Error Linear Units (GELU) 링크: https://arxiv.org/abs/1606.08415 XII. GLU Variants Improve Transformer 링크: https://arxiv.org/abs/2002.05202 XIII. Deep Reinforcement Learning from Human Preferences 링크: https://arxiv.org/abs/1706.03741