Deep Dive: Building GPT from scratch - part 3

Hello and welcome back to the series on Starter AI. I’m Miko, writing this from the snowy Hokkaido!

In the previous two parts, we got our first exposure to building AI - we saw Andrej implement micrograd, build a tiny neural network using it, and compare it to PyTorch, the industry standard.

Today, it’s time to start on Generative AI. We’ll follow Andrej’s building makemore - a generative language model creating name-like words, implementing increasingly robust models in the consecutive videos.

You’re going to need your favourite beverage, and about 2 hours of your time.