LLM programming workshop.

About this event

Review programming fundamentals. Pytorch, HuggingFace apis, The popularity of LLMs has caused extensive copy and paste in open source projects such as vllm. We are going to cover some basic programming idioms needed for distilling models from big to smaller models. These skills are transferrable and are different than the classic RL exercises. pre distillation exercises (https://colab.research.google.com/drive/1B58tfIE-QT7tDTpFR2Ec18_sfpjR69He?usp=sharing) Distillation colab workbook reference (https://colab.research.google.com/drive/1B58tfIE-QT7tDTpFR2Ec18_sfpjR69He?usp=sharing) Distillation (https://www.youtube.com/watch?v=YH0YXgDWZXA) Transfer Teacher model to student model You will need a colab account. We will review each problem and leave time for everyone to complete their solutions without the aid of a LLM. This event is about repetition and practice. 1. HF apis to tokenizer and model 2. how to speed up model downloads 3. basic idioms, batching, padding, attention masks 4. logits and probability distributions with softmax 5. minimizing KL divergence 6. top-k and softening teacher distribution by raising T. Dark Knowledge 7. token level knowledge distillation. (distillation event we will stop here) 8. Sequence level distillation 9. DAgger Ross 2011 10. Off policy Distillation 11. On policy Distillation Agarwal 2024 12. Self Distilled Reasoning Zhao 2016 13. Self Distillation Fine Tuning Shenfeld 2026 Generated by Codex. [https://colab.research.google.com/drive/1EokTq_U-_JcPHtPTTjfVUcSjTW7dyDiB?usp=drive_link](lab)