Reading ML Papers and PyTorch Repos

ML Paper Structure              →  Blog Post Equivalent
─────────────────────────────────────────────────────
Abstract                        →  TL;DR at the top
1. Introduction                 →  "Here's the problem we're solving"
2. Related Work                 →  "Other approaches and why ours is different"
3. Method / Architecture        →  "Here's how we built it" (THE KEY SECTION)
4. Experiments                  →  "Here's how we tested it"
5. Results                      →  "Here's what happened"
6. Conclusion                   →  "Summary and future work"
Appendix                        →  Implementation details, hyperparameters

Typical PyTorch Repo            →  React/Next.js Equivalent
─────────────────────────────────────────────────────
model.py / net.py               →  components/ (the architecture)
train.py                        →  app entry point (the training script)
data.py / dataset.py            →  api/ or hooks/ (data loading)
utils.py                        →  lib/ or utils/ (helpers)
configs/ or args                →  .env or config files
requirements.txt                →  package.json
README.md                       →  README.md

import torch
import torch.nn as nn
import torch.nn.functional as F

class TimeSeriesEncoder(nn.Module):
  """Encodes a time series into a fixed-length representation."""

  def __init__(self, input_dim, hidden_dim=128, num_layers=2, dropout=0.1):
      super().__init__()
      self.lstm = nn.LSTM(
          input_dim, hidden_dim,
          num_layers=num_layers,
          batch_first=True,        # input shape: (batch, seq_len, features)
          dropout=dropout
      )
      self.attention = nn.Linear(hidden_dim, 1)
      self.fc = nn.Linear(hidden_dim, 64)
      self.norm = nn.LayerNorm(64)

  def forward(self, x):
      # x shape: (batch, seq_len, input_dim)
      lstm_out, _ = self.lstm(x)          # (batch, seq_len, hidden_dim)

      # Attention weights
      attn_weights = F.softmax(
          self.attention(lstm_out), dim=1  # (batch, seq_len, 1)
      )
      context = (lstm_out * attn_weights).sum(dim=1)  # (batch, hidden_dim)

      out = self.fc(context)               # (batch, 64)
      out = self.norm(out)                 # (batch, 64)
      return F.relu(out)

// Mental translation:
// 1. __init__ = constructor. Lists the layers (building blocks).
//    - LSTM layer: processes sequences (like processing array items in order)
//    - Linear layers: dense layers (tf.layers.dense)
//    - LayerNorm: normalization (tf.layers.layerNormalization)

// 2. forward() = the data flow. Read it top to bottom:
//    - Input goes through LSTM → get sequence of hidden states
//    - Compute attention weights (which time steps matter most)
//    - Weighted sum → single vector per batch item
//    - Dense layer → normalize → ReLU activation → output

// TF.js equivalent architecture:
const encoder = tf.sequential({
layers: [
  tf.layers.lstm({ units: 128, returnSequences: true, inputShape: [null, inputDim] }),
  // attention would be a custom layer
  tf.layers.dense({ units: 64 }),
  tf.layers.layerNormalization(),
  tf.layers.activation({ activation: 'relu' }),
]
});

# Pattern 1: Residual connection (skip connection)
# "Add the input back to the output"
out = self.layer(x) + x

# Pattern 2: Multi-head attention
# "Multiple parallel attention computations"
attn_output = F.multi_head_attention_forward(...)

# Pattern 3: Embedding lookup
# "Convert integer IDs to dense vectors" (like a lookup table)
embedded = self.embedding(token_ids)  # nn.Embedding(vocab_size, embed_dim)

# Pattern 4: Gradient clipping (in training code)
# "Prevent exploding gradients"
torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)

# Pattern 5: Learning rate scheduler
# "Decrease learning rate over time"
scheduler = optim.lr_scheduler.StepLR(optimizer, step_size=10, gamma=0.1)

Symbol    Meaning                    Code
──────────────────────────────────────────────
Σ         sum                        np.sum / tf.sum
∏         product                    np.prod / tf.prod
||x||     norm (length of vector)    np.linalg.norm / tf.norm
x̂         normalized / predicted     x_hat or x_pred in code
∂L/∂w     gradient of loss w.r.t w   loss.backward() / tf.grads
argmax    index of largest value     np.argmax / tf.argMax
softmax   normalize to probabilities F.softmax / tf.softmax
⊙         element-wise multiply      a * b / a.mul(b)
@         matrix multiply            a @ b / tf.matMul(a, b)