nanochat
A minimal, hackable codebase for training your own small ChatGPT-style model on a single GPU node. It covers tokenizing, pretraining, finetuning, evaluation and chat inference at low cost.
Share on XLicense: MIT
Overview
nanochat is a minimal, hackable harness for training your own small ChatGPT-style language model on a single GPU node. It covers tokenization, pretraining, finetuning, evaluation and inference, and lets you chat with the result through a simple command line interface. One setting, depth, which is the number of transformer layers, drives the other hyperparameters automatically. The README says a GPT-2 level model can be trained for about 48 dollars in roughly two hours on an 8XH100 node.
Key features
- Covers tokenization, pretraining, finetuning, evaluation and inference
- Runs on a single GPU node with minimal code
- One depth setting derives the other hyperparameters
- Chat with the trained model from a simple CLI
Best for
Good for learners and researchers who want to understand and modify the full LLM training pipeline at low cost. It maintains a leaderboard for GPT-2 speedruns.
- Upstream
- karpathy/nanochat
- Fork on GitHub
- Guo-astro/nanochat
- Upstream stars
- 58k
- Category
- AI agents and LLM tools
- Language
- Python
- License
- MIT
- Forked
- 2025-10-14
- Sync status
- In syncLast synced 2026-10-10
More in AI agents and LLM tools
An open-source curriculum for learning AI engineering by implementing model internals, retrieval pipelines, and agent runtimes. Study through lessons, interactive labs, or staged coding projects, and inspect the code and evaluation results.
Forked 2026-10-09Last synced 2026-10-10License: MITAI agents and LLM toolsGitHub
A small, hackable character-level language model that learns patterns from line-based text files and generates similar items, such as name ideas.
Forked 2026-10-09Last synced 2026-10-10License: MITAI agents and LLM toolsGitHub
uzu is an inference engine for running AI models in apps, helping developers keep inference on-device instead of relying on remote services. Use it to download a model and build chat features.
Forked 2026-10-09Last synced 2026-10-10License: MITAI agents and LLM toolsGitHub
Anti Slop gives AI coding agents rules to avoid generic UI, filler text, and AI-shaped code. Use it as a filter alongside your own design direction.
Forked 2026-10-08Last synced 2026-10-10License: MITAI agents and LLM toolsGitHub