1. Llama 2: Open Foundation and Fine-Tuned Chat Models
링크: https://arxiv.org/abs/2307.09288

2. Overview
I.배경
II.llama의 주어진 과제
III.핵심기술
- a.RoPE
- b.RMSNORM(Pre-Norm)
- c.GQA
- d.SwiGLU
IV. Fine tuning
- a.SFT
- b.RLHF
- c.GAtt
V.결론

3. Reference
I. Attention Is All You Need
링크: https://arxiv.org/abs/1706.03762

II. Language Models are Few-Shot Learners
링크: https://arxiv.org/abs/2005.14165

III. LLaMA: Open and Efficient Foundation Language Models
링크: https://arxiv.org/abs/2302.13971

IV. Llama 2: Open Foundation and Fine-Tuned Chat Models
링크: https://arxiv.org/abs/2307.09288

V. RoFormer: Enhanced Transformer with Rotary Position Embedding
링크: https://arxiv.org/abs/2104.09864

VI. Layer Normalization
링크: https://arxiv.org/abs/1607.06450

VII. Root Mean Square Layer Normalization
링크: https://arxiv.org/abs/1910.07467

VIII. Fast Transformer Decoding: One Write-Head is All You Need
링크: https://arxiv.org/abs/1911.02150

IX. GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
링크: https://arxiv.org/abs/2305.13245

X. Rectified Linear Units Improve Restricted Boltzmann Machines
링크: https://www.cs.toronto.edu/~fritz/absff/reluICML.pdf

XI. Gaussian Error Linear Units (GELU)
링크: https://arxiv.org/abs/1606.08415

XII. GLU Variants Improve Transformer
링크: https://arxiv.org/abs/2002.05202

XIII. Deep Reinforcement Learning from Human Preferences
링크: https://arxiv.org/abs/1706.03741