tinygrad — A Minimalist Deep Learning Framework

Overview

tinygrad is a deep learning framework sitting between PyTorch and micrograd, maintained by tiny corp. It pursues minimalism and hackability — the entire framework is just ~200 Python source files yet supports the full DL workflow from training to inference.

Core Components

• Tensor Library: PyTorch-like Eager API with autograd
• IR Compiler: Kernel fusion & lowering, JIT + Graph execution
• Training Utils: nn, optim, datasets — complete training suite
• LLM Inference: GGUF format support for open-source LLMs

Hardware Backends

NVIDIA CUDA/PTX, AMD ROCm/CDNA/RDNA3/RDNA4, Apple Metal, OpenCL, Qualcomm QCOM, WebGPU, LLVM IR, ONNX import.

Key Features

• Lazy Execution & Kernel Fusion (DEBUG=3/4 to see fused kernels)
• BEAM Search for optimal kernel configs
• Process Replay testing across changes