Tutorial¶

ok so you are in the business of making language models. cool. me too! how auspicious. let me show you how I do it, and then you can show me how you do it.

Making GPUs warm, my way...¶

• running experiments - start here; we'll make a basic GPT to help me me cure my indomitable loneliness
• inspecting runs - say you did the thing above, how'd you know how it went?

...making someone else's GPU warm...¶

dispatching runs remotely - ok but I'm not going to sit around and wait for your GPT to train, so how about we send it off to a cluster

...and doing so your style¶

ok I know that training a basic GPT is so boring. You are dying to port your architecture/dataset/model/analysis/whatever to some guy's training repo. yeah, ok, let's do that.

Making it your own - add a dataset, mix training data, switch optimizers, and then extend GPT.