created: 20230218215510319
creator: Octt
modified: 20230228230210163
modifier: Octt
tags: AI
title: nanoGPT

; Primary Resources
: [[Code (Git)|https://github.com/karpathy/nanoGPT]]
: Video explanation: <<YoutubeL kCc8FmEb1nY "Let's build GPT: from scratch, in code, spelled out">>

; Additional Resources
: [[Training nanoGPT on my Journal|https://hut.pm/nanogpt.html]] - <<mywarc 114>>

!!! ''My Experience''

Done training on [[my main laptop|Octt/Laptop]], with PyTorch 2 nightly, and CPU training, because:
* I failed to install all required dependencies on [[my desktop|Octt/groso]], both on Void Linux musl (bare-metal) and Ubuntu Jammy (chroot).
* I now had to use my slow laptop, but issues with NVIDIA drivers meant no GPU training.

(TODO: Write my results)