Handwired

Neural nets aren’t magic — they’re stacks of multiply/add plus nonlinearities like ReLU. Handwired’s move is to rewrite ordinary program control flow as those same primitives.

const result = (a > 4) ? 10 : 12

becomes something like:

const above4 = relu(a - 4) - relu(a - 5) // 1 or 0 for integer a
const result = above4 * 10 + (1 - above4) * 12

Same answer, but now it’s arithmetic over floats. Collect enough +, *, and relu and you get a giant matmul — a neural net whose weights were hand-written to match the program exactly, not trained. Differentiability comes from the ops, not from learning.

Why it matters

Three doors open at once:

1. Code and nets collapse. “Program” and “model” stop being different kinds of thing. Same substrate.
2. Selective learning. Freeze most weights (exact logic), unfreeze a few, train those — hybrid systems that are partly specified, partly learned.
3. GPU for free. Once it’s matmul, WebGPU and fusion apply. Write serial JS, compile to matrices, get massive parallelism. You can also bolt a frozen Handwired program onto something like nanoGPT so the model “speaks” both Shakespeare and your exact UI logic.

The boundary between symbolic software and neural software is thinner than it looks — mostly a compilation problem.