OcttKB/Wiki-OcttKB/tiddlers/Normal/nanoGPT.tid

22 lines
769 B
Plaintext
Raw Normal View History

2023-03-04 23:12:24 +01:00
created: 20230218215510319
creator: Octt
modified: 20230228230210163
modifier: Octt
tags: AI
title: nanoGPT
; Primary Resources
: [[Code (Git)|https://github.com/karpathy/nanoGPT]]
: Video explanation: <<YoutubeL kCc8FmEb1nY "Let's build GPT: from scratch, in code, spelled out">>
; Additional Resources
: [[Training nanoGPT on my Journal|https://hut.pm/nanogpt.html]] - <<mywarc 114>>
!!! ''My Experience''
Done training on [[my main laptop|Octt/Laptop]], with PyTorch 2 nightly, and CPU training, because:
* I failed to install all required dependencies on [[my desktop|Octt/groso]], both on Void Linux musl (bare-metal) and Ubuntu Jammy (chroot).
* I now had to use my slow laptop, but issues with NVIDIA drivers meant no GPU training.
(TODO: Write my results)