tils (page 4)
-
Key Concepts Behind QLoRA Fine-Tuning
Quantization + low-rank adapters let you fine-tune huge LLMs on a single GPU.
-
Relationship Between L1 Norm and L1 Regularization
Exploring how L1 and L2 norms form the basis of L1 (lasso) and L2 (ridge) regularization, with concrete examples and geometric intuition.
-
Govern, Garden, Guide
Dividing parenting into Govern (0-6), Garden (6-12), and Guide (12+), aligning roles with developmental stages.
-
Why logits.exp() Equals Counts
Understanding neural network computations as log-domain operations, making multiplicative interactions additive through logs
-
Floating-Point Precision & Exploding Gradients
Floating-point rounding errors in backpropagation can accumulate and magnify across layers, leading to exploding gradients in deep networks.