llm.dyyln.dev

Introduction

As mentioned the book is two-fold :

  • Build an LLM
  • Explain the concepts behind an LLM We will build such that you can run most things on a local machine or laptop with a GPU to keep everything smaller scale. Whilst we will go through training and implement it ourselves, our end model will support open weights, so you don’t have to spend obsurd amounts of time/compute on training your own model to get good results.

There are lots of resources on LLMs in general which tend to stop at GPT-2 level LLMs. I would like to go further than that and expand on concepts used by frontier models (where the information is publically available) but that is a long way off for now.

You can read this book straight through, or with concepts only. Even if you choose not to implement the code yourself, you may still be interested in the code and implementation details to skim over.

The expected path is :

  • Read and understand the concepts in a chapter
  • Implement the code describe/guided in the chapter
  • Later chapters will likely build on code from earlier chapters

I will add references for each chapter for sources/further reading, and try to draw my own diagrams to explain topics where I see fit