AI Engineering Field Manual
Tested architectural blueprints, from multi-agent graph orchestration to production-scale evaluation.
Full Lifecycle Review: From Pretraining Convergence to SFT and LoRA
Comprehensive lifecycle debrief: telemetry reviews of 1,000 pretraining steps on Intel Arc 130T (Loss, PPL, JIT compilation, 7,000 tok/s throughput), SFT and LoRA hands-on execution, head-to-head model evaluations, and eight golden engineering rules for LLM training.
Systematic Learning Tracks
Curated Series Tracks
Building an LLM from scratch
不依赖任何高层黑盒库，纯使用 Python 与 PyTorch 从底层字节流和矩阵运算出发，端到端实现 BPE 分词、现代 Transformer 架构、内存映射二进制数据流水线、混合精度预训练引擎、自回归 KV-Cache 推理、SFT 指令微调与 LoRA 适配器，并在消费级显卡上完整闭环。
AI Agent Development and Interview Guide
Start from「AI Agent Development and Interview Guide: 00-Recruitment Requirements and Competency Model Report」开始
AI AGENT Beginner's Guide
Start from「AI Agent Beginner Tutorial: 01 - Concepts, Examples, and Technical Background」开始
Ai Side Hustle
Start from「Systematizing Faceless YouTube: High Retention AI Workflows」开始
WorkBuddy In Action
Start from「WorkBuddy Practical: 01 - Introduction to WorkBuddy」开始
Curriculum
All guides
- 01IntermediateBuilding an LLM from scratchCost: Free
Full Lifecycle Review: From Pretraining Convergence to SFT and LoRA
Comprehensive lifecycle debrief: telemetry reviews of 1,000 pretraining steps on Intel Arc 130T (Loss, PPL, JIT compilation, 7,000 tok/s throughput), SFT and LoRA hands-on execution, head-to-head model evaluations, and eight golden engineering rules for LLM training.
- 02AdvancedBuilding an LLM from scratchCost: Free
Parameter-Efficient Fine-Tuning with Handcrafted LoRA on Modern LLMs
Dual-track parameter-efficient fine-tuning: exploring intrinsic rank hypotheses, coding custom LoRALinear bypass layers in PyTorch, executing dual tracks on local 0.04B (0.48% tuned weights) and industrial Qwen2.5-1.5B, and performing zero-latency offline weight merging.
- 03AdvancedBuilding an LLM from scratchCost: Free
Supervised Fine-Tuning (SFT) and Instruction Alignment
Transforming raw continuation base models into obedient conversational assistants: exploring Prompt Masking and ignore_index=-100 gradient firewalls, industrial Chat Templates with control tokens, synthesizing 30,000 diverse instruction pairs, implementing Meta's LIMA principle (quality over quantity), and coding dynamic padding collation.
- 04IntermediateBuilding an LLM from scratchCost: Free
0.04B End-to-End Hands-on Practice and Telemetry Diagnostics
A step-by-step hands-on lab: running the full 5-act closed loop across corpus sanitization, BPE training, binary compilation, pretraining (7,000 tok/s), and KV-Cache streaming; examining why downsizing vocabulary to 4096 saves 2.1M parameters for hidden layers, seamless checkpoint resumption, Shannon entropy lower bounds, and triple-tier remedies for punctuation monopolies.
- 05IntermediateBuilding an LLM from scratchCost: Free
Autoregressive Streaming Inference and KV-Cache Memory Optimization
Building an industrial-grade streaming autoregressive inference engine: resolving the O(N^2) repetitive compute disaster with KV-Cache sticky-note caching (consuming just 6.29MB VRAM for 1024 tokens), mastering repetition penalty, temperature, Top-K, and Top-P right-shift mathematics, and building a fluid typewriter interface.
- 06IntermediateBuilding an LLM from scratchCost: Free
Engineering Pretraining Engines and Optimization on Consumer GPUs
Building a rock-solid, production-grade LLM pretraining engine on consumer GPUs: covering cross-entropy loss, perplexity (PPL), AdamW decoupled weight decay (why normalization layers must not decay), warmup and cosine decay schedules, automatic mixed precision (AMP) with GradScaler, and gradient accumulation in 1.8GB VRAM.
- 07IntermediateBuilding an LLM from scratchCost: Free
Industrial Data Sanitization and Zero-Copy Binary Pipelines with mmap
Eliminating GPU starvation in LLM training: using uint16 binary compression to condense 34.63M tokens into just 66MB, leveraging np.memmap and pinned memory hardware DMA, avoiding double-shift bugs, masking metadata loss, and implementing two-stage MinHash deduplication on 460,000 poems.
- 08IntermediateBuilding an LLM from scratchCost: Free
First-Principles Mathematical Derivations for Modern Transformers
A high-school-friendly first-principles mathematical derivation of modern Transformers: covering vector projections, complex rotation and Euler's formula for RoPE, variance scaling proofs for 1/sqrt(d_k), Softmax numerical stability, and maximum likelihood cross-entropy.
- 09AdvancedBuilding an LLM from scratchCost: Free
Implementing Modern Transformer Architecture from Scratch
A deep dive into the four core engines of modern Transformers: replacing legacy Post-LN and absolute embeddings with hand-coded RMSNorm, RoPE rotary position embeddings, SwiGLU dual-channel gating, and GQA (8Q:2KV), assembling a complete 35.93M parameter MiniLLaMA core.
- 10IntermediateBuilding an LLM from scratchCost: Free
Building a Byte-Level BPE Tokenizer from Scratch
Implementing an industrial-grade Byte-Level BPE tokenizer in pure Python from raw UTF-8 byte streams without external libraries: covering 256 byte atom zero-OOV guarantees, manual pair merge derivations, fused punctuation regex cutting punctuation oscillations by 93%, and atomic special token protection.
- 11BeginnerBuilding an LLM from scratchCost: Free
0.04B Model Architecture Design, Mathematical Modeling, and Intuition
A comprehensive design blueprint for a modern 0.04B (35.93M parameter) autoregressive language model: covering conditional probability modeling, mixing console knob analogies, Chinchilla scaling law saturation, tensor shape flow, parameter breakdowns, AdamW memory mechanics, and FLOPs compute estimation.
- 12BeginnerBuilding an LLM from scratchCost: Free
Building a 0.04B (36M) LLM from Scratch: End-to-End Full-Stack Overview
An end-to-end full-stack overview of building a 0.04B (36M) classical poetry language model from scratch in pure Python and PyTorch without high-level abstractions: covering BPE tokenization, modern Transformer architecture, mmap pipelines, pretraining, KV-Cache inference, SFT, and LoRA on consumer GPUs.
- 13AdvancedCost: Free
From 0-Token Fast-Path to Evaluation Graphs: Deconstructing Enterprise Dual-Track Intent Routing for AI Agents
A comprehensive analysis of routing latency bottlenecks and intent drift risks in enterprise AI agents, detailing the dual-track architecture combining a 0-token deterministic Fast-Path with a multi-dimensional Evaluation Graph Slow-Path, specificity conflict arbitration, timeout circuit breakers, and production-grade Python implementations.
- 14IntermediateCost: Free
Tool Calling vs. Executing Code: Understanding Code-as-Action in AI Agents
Code-as-Action unifies agent tool invocation and decision-making into executable Python code. This guide systematically analyzes the CodeAct mechanism, execution loop, context shielding, self-debugging, and sandbox defense architecture with leading frontier model selection from SWE-bench Verified and LiveCodeBench, helping engineers build expressive, token-efficient, and secure agent runtimes.
- 15AdvancedCost: Free
Exploring Standalone Small Classifiers: Qwen LoRA and BGE
The opening separates the performance problem from the actual base models: locally trainable Qwen3-0.6B and BGE-small-zh-v1.5. For every round from V1 through V4, the article explains the change, measured effect, analysis, and reason for the next iteration, then summarizes final quality and performance; final selection still requires an independent blind test.
- 16IntermediateCost: FreeChromaPineconeMilvus
Building a Vector Storage
Creating vector storage is the fundamental first step in building high-dimensional semantic search infrastructure. This guide systematically explores document loading, semantic text splitting, embedding generation, and vector database comparative analysis (covering Chroma, Pinecone, Milvus, FAISS, Annoy, and sqlite-vector) to build reliable similarity search infrastructure.
- 17IntermediateCost: Free
Differences Between Agent and Harness
An in-depth analysis of AI Agent cognitive reasoning and Harness runtime chassis architectures, exploring dynamic spectrums, intertwined state and memory, shared control flows, code generation tools, multi-layered security, and boundary trade-offs.
- 18IntermediateCost: FreeCodeGraph
CodeGraph Usage Guide
Learn how to install CodeGraph, index a project, connect it over MCP, query code relationships, assess change impact, select affected tests, keep the graph fresh, and troubleshoot common failures.
- 19IntermediateCost: FreeSpec KitOpenSpec
Spec Kit vs. OpenSpec: Architectural Comparison and Selection Guide
A detailed comparison of GitHub's Spec Kit and Fission AI's OpenSpec, evaluating Spec-Driven Development (SDD) principles, architectural designs, spec delta mechanisms, and selection strategies across project lifecycles.
- 20IntermediateCost: FreeVoicebox Local AI Voice Studio
Open-Source ElevenLabs Alternative: Hands-On Guide to Voicebox
Voicebox is a free open-source local AI voice studio supporting 5-second voice cloning, text-to-speech, multi-track podcast editing, and system dictation. This guide covers setup, voice cloning, multi-track editing, and dictation usage.
- 21IntermediateCost: FreedbskillrnskillIndexTTS2
55 Open-Source AI Video Skills Architecture and Automated Pipeline Guide
A comprehensive breakdown of 55 open-source AI video skills across 12 architecture layers, detailing automated pipeline invocation, independent business diagnostics, and practical workflow implementation.
- 22IntermediateCost: FreeHyperFramesRemotion
Code Video Framework Selection: HyperFrames vs. Remotion Deep Dive and Decision Guide
A comprehensive evaluation of HyperFrames and Remotion for AI video generation and code-driven editing, assessing rendering architecture, agent compatibility, video composition, and decision criteria.
- 23IntermediateCost: FreeAnthropicClaude Opus 5
New Rules of Context Engineering for Claude 5: Thin Prompts, Progressive Disclosure, and Model Unhobbling
Based on Anthropic technical staff insights from Claude Code, this guide presents the context engineering evolution for Claude 5: trimming 80% of prompt rules, moving from rigid constraints to native model judgment, and employing Thin Prompts with progressive disclosure.
- 24IntermediateCost: Free
Agent Harness vs. Loop vs. Graph Engineering
This guide separates Agent Harness, Loop Engineering, and Graph Engineering, using faithful transcriptions of all seven source images to show how the layers work together. It also maps common failures to the runtime environment, feedback loop, or explicit workflow that should be changed.