Weight-Space Programming as Certified Network Editing
Higher Categorical Semantics and Architecture-Independent Generalization
Anonymous. November 2, 2026. Full proofs, references, and exact examples appear in the accompanying complete PDF and source package.
1. Scope and motivation
The motivating source is Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab, Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights (November 2, 2026): https://rikiyat.github.io/blog/weight-space-programming/ . Its experimental claims remain those of its authors. This manuscript does not independently reproduce them.
We formalize direct parameter inspection, synthesis, and editing. The contribution is an organization of established parameterized-category, wiring, and higher-topos constructions into a certificate-based semantics. The architecture-independent conclusion is conditional on a typed interpretation and explicit closure of the desired behavioral relation. No general competence claim about language models follows from these definitions.
2. Parameter, behavior, and evidence
Fix universe bounds and an infinity-topos $\mathcal E$. A network presentation has parameter object $\Theta_N$, input and output objects $X_N,Y_N$, and evaluation
$$e_N:\Theta_N\times X_N\to Y_N.$$
Its transpose is $\beta_N:\Theta_N\to Y_N^{X_N}$. More generally, specify a behavior object $\mathcal B_N$ of kernels, transitions, or traces and $\beta_N:\Theta_N\to\mathcal B_N$. Architecture, input encoding, preprocessing, masks, numerical semantics, and decoding belong to the presentation. Weights without these data do not determine a program.
A property is a subobject $S\hookrightarrow\mathcal B_N$. A structured evidence family is $C\to\mathcal B_N$, with an explicit factorization through $S$ when soundness for $S$ is asserted. Certified parameters form
$$V_N=\Theta_N\times_{\mathcal B_N}C.$$
A witness is a point of the relevant fiber. Proposition-valued properties have empty or contractible fibers; structured evidence can retain multiple witnesses and higher identifications.
Numerical values in the examples are discrete, 0-truncated rational objects. Replacing the real line by its ordinary contractible homotopy type would erase distinctions needed for computation. Smoothness and metric structure require additional data. A continuous path between parameter vectors is not evidence of equal behavior.
3. Higher parameterized maps versus directed edits
For fixed $X,Y$, the Grothendieck construction of the presheaf
$$P\mapsto\operatorname{Map}_{\mathcal E}(P\times X,Y)$$
is an infinity-category of parameterized maps. An object is $(P,f)$; a morphism $(P,f)\to(Q,g)$ is $a:P\to Q$ with a specified homotopy $f\simeq g(a\times\mathrm{id})$. Higher simplices record compatibilities. Horizontal composition is
$$(Q,g)\circ(P,f)=(P\times Q,(p,q,x)\mapsto g(q,f(p,x))).$$
Products give coherent units, associators, and interchange, yielding parameterized $(\infty,2)$-categorical semantics. Reparameterization preserves behavior. A repair that changes behavior is a different structure.
Let $R\to\mathcal B\times\mathcal B$ be a directed behavioral relation with optional witnesses. Its universal certified-edit object is
$$\widehat R=(\Theta\times\Theta)\times_{\mathcal B\times\mathcal B}R.$$
An edit family is $K\to\widehat R$, equivalently a span $\Theta\leftarrow K\to\Theta$ with evidence relating the denotations of its endpoints. Given coherent relational composition $R_{01}\times_{\mathcal B}R_{12}\to R_{02}$, edit families compose by $K_{01}\times_\Theta K_{12}$. Homotopy pullbacks retain intermediate identifications. Maps of spans preserving witnesses give 2-morphisms, with their higher identifications. This separates directed revision from equivalence.
4. Reading and proposal mechanisms
An observation map $o:\Theta\to O$ records available weights, metadata, or permitted transcripts. A target label is $\ell:\Theta\to L$. A perfect reader is a factorization $d o=\ell$.
Identification proposition. For a surjection of sets $o:\Theta\twoheadrightarrow O$, such a reader exists exactly when equal observations imply equal labels.
Proof. Necessity follows by applying $d$. Conversely each fiber has a unique label, which defines $d$. For a nonsurjective observation map the result first holds on its image. Higher-valued readers instead require coherent descent data along the Cech nerve.
If two systems induce the same observation law, passing that law through any stochastic reader gives the same answer law. Two distinct binary labels with equal priors therefore allow average success at most $1/2$.
A synthesizer proposes a point of $\Theta$; a certified synthesizer returns a point of $V_N$. An editor additionally takes the old parameter. A finite stochastic operational model may use an editor-state kernel
$$H\times\mathrm{Spec}\times\Theta\to\mathsf D(H\times(\Theta+\{\mathrm{fail}\})\times\mathbb N^d),$$
where $\mathsf D$ is finite distribution and the resource vector counts calls, batch sizes, tokens, or steps. This concrete model is in sets; no such monad is assumed in every infinity-topos. Permitted effects describe the procedure's trace. Gradient prohibition does not itself imply correctness or exclude all search.
Acceptance is sound if a checker produces a point of $\widehat R$ and the evidence family factors through the intended relation. This isolates the checker obligation; it does not construct a complete checker or turn finite tests into universal proofs.
5. Symmetry and higher quotient semantics
Suppose $G$ acts on $\Theta$ and $\beta$ is coherently invariant. The action-groupoid colimit $\Theta/\!/G$ retains stabilizers.
Symmetry descent. Behavior and every coherently invariant reader factor through $\Theta/\!/G$. An equivariant edit induces an edit of the quotient.
Proof. Maps from the action-groupoid colimit are coherent compatible maps from its simplicial levels. Invariance gives this datum. An equivariant edit gives a map of action groupoids, hence a map of their colimits.
For coordinatewise neural activations and hidden-layer permutation matrices $S_i$, with $S_0=S_L=I$, transform
$$W_i'=S_iW_iS_{i-1}^{-1},\qquad b_i'=S_i b_i.$$
Because $\sigma(S_i z)=S_i\sigma(z)$, induction gives $h_i'=S_i h_i$ and identical external outputs. Arbitrary basis changes do not commute with nonlinear activations. A fixed point under swapping identical units retains an $S_2$ automorphism; its quotient can be $BS_2$, whereas the orbit set loses that information. A unique neuron identity is therefore presentation-dependent. Floating-point reordering needs a separate implementation argument.
6. Main compositional theorem
Let $\mathcal O$ be a small colored operad of permitted typed wiring operations. Assume parameter and behavior families $\Theta_c,\mathcal B_c$ are $\mathcal O$-algebras, with assemblies $a_w,b_w$, and that $\beta$ is an algebra morphism:
$$\beta_c a_w\simeq b_w\prod_i\beta_{c_i}.$$
For each color choose $R_c\to\mathcal B_c\times\mathcal B_c$. Require coherent certificate assembly maps
$$\rho_w:\prod_i R_{c_i}\to R_c$$
whose endpoint maps are the two applications of $b_w$, satisfying the operadic unit, symmetry, and substitution laws. This is the relation-closure hypothesis.
Compositional certified editing theorem. Each operation $w$ induces
$$\prod_i\widehat R_{c_i}\to\widehat R_c$$
with endpoints the assemblies of original and edited components. These induced maps obey all operadic coherences. The construction also holds after restriction along a shared-parameter control object.
Proof. Apply $a_w$ to the two tuples of parameter endpoints and $\rho_w$ to their certificate tuple. The algebra-morphism identity makes their images in $\mathcal B_c\times\mathcal B_c$ coincide coherently. The defining pullback of $\widehat R_c$ supplies a unique compatible lift, in the sense of a contractible space of choices. The algebra and relation-assembly coherences give matching cones for every operadic substitution, so these lifts inherit the coherences. Pulling back the same diagram proves the shared-control statement.
Parameter sharing must hold at both endpoints; independent updates to tied coordinates are not automatically licensed. Exact behavioral equality satisfies the closure hypothesis for all defined wirings. Preservation on a protected domain also needs range conditions between components.
Interaction counterexample. Let $F(a,b)=a+b-3ab$, with larger output considered better. Starting at $(0,0)$, changing either coordinate alone to $1$ changes utility from $0$ to $1$. Changing both gives $-1$. Thus separately favorable edits need not compose: the closure hypothesis is substantive.
7. Generalization to other networks
The theorem depends on an interpretation and relation closure, not differentiability.
- Feedforward and convolutional networks use kernels, biases, explicit activations, and sharing constraints.
- Transformers use a bounded token-to-distribution behavior or a stateful transition, with tokenizer, masks, positions, caches, and decoding fixed.
- Graph networks use shared vertex/edge rules and permutation-compatible aggregation. For $h_v'=\phi_\theta(h_v,\sum_{u\in N(v)}\psi_\theta(h_u,h_v))$, reindexing the finite sum proves equivariance under graph isomorphism. Induction handles multiple layers; invariant readout gives graph-level invariance. Task success is a separate property.
- Recurrent networks use $T_\theta:S\times X\to S$ and output maps, with specified initial state. Finite behaviors arise by recursion; infinite traces require a causal/guarded construction or appropriate limit. Arbitrary algebraic feedback is excluded without existence and uniqueness.
- Finite stochastic and diffusion networks use Markov kernels. Sequential composition is $(LK)(z\mid x)=\sum_y L(z\mid y)K(y\mid x)$. Independent parallel composition uses product kernels; correlated randomness must be explicit. The stochastic backend is not asserted to be a topos.
- Boolean circuits and symbolic networks use exact gate/program semantics; unbounded programs require termination restrictions or partial/trace semantics.
- Hybrid, routing, and logistics networks use timed/discrete transitions and well-posed continuous flow, with explicit environment assumptions.
- Tensor and quantum networks can use classical parameters selecting morphisms in a symmetric monoidal backend. Only backend-legal wiring is allowed; the external classical parameter model does not introduce unrestricted quantum copying.
Semantic transfer proposition. A finite-limit-preserving $F:\mathcal E\to\mathcal F$ transports the certificate diagrams to $F\mathcal B$-valued semantics, since it preserves their pullbacks, finite products, and supplied coherences. It need not preserve exponentials: $F(Y^X)\simeq(FY)^{FX}$ requires an extra hypothesis. Infinite traces, metrics, and probability laws have additional preservation obligations.
8. Quantitative certificate instances
Sequential perturbation theorem. Let $f_i,g_i:D_{i-1}\to D_i$ be maps between metric spaces, $g_i$ be $L_i$-Lipschitz, and $d(f_i(x),g_i(x))\leq\delta_i$ throughout the specified domains. Then
$$\sup_x d(f_n\cdots f_1(x),g_n\cdots g_1(x))\leq\sum_{i=1}^n\delta_i\prod_{j>i}L_j.$$
Proof. Consecutive hybrid compositions $g_n\cdots g_{i+1}f_i\cdots f_1$ differ at one layer. Its discrepancy is amplified by the common downstream Lipschitz constants. Sum these distances. Every used intermediate value must belong to the domain of its certificate.
Recurrent corollary. If $T'$ is $L$-Lipschitz in state and $d(T(s,x),T'(s,x))\leq\delta$ on an invariant domain, equal initial states and equal input streams give
$$d(s_t,s_t')\leq\delta\sum_{k=0}^{t-1}L^k.$$
This follows from $e_{t+1}\leq Le_t+\delta$. If $L<1$, the uniform bound is $\delta/(1-L)$. For $s_{t+1}=s_t/2+x_t$ edited by adding $1/8$, the difference at four steps is exactly $15/64$ and is bounded by $1/4$ at every step.
Finite stochastic proposition. Uniform total-variation discrepancies $\epsilon$ between $K,K'$ and $\eta$ between $L,L'$ imply a discrepancy at most $\epsilon+\eta$ between $LK$ and $L'K'$. Insert $LK'$; use contraction under $L$, convexity of the $K'$ mixture, and the triangle inequality. Here $\mathrm{TV}(p,q)=\frac12\sum_y|p_y-q_y|$.
These metric specializations require their stated bounds. Small coordinate distance or a small fraction of modified weights does not supply them.
9. Local-to-global evidence
Let $V_W\to W$ be the pullback certificate family for a candidate family $\theta:W\to\Theta$. Let $U\to W$ be an effective epimorphism and $U_\bullet$ its Cech nerve.
Certificate descent theorem (standard higher-topos consequence). Global sections of $V_W\to W$ are equivalent to coherent matching sections over all $U_\bullet$. A fixed complete matching datum determines a global section up to contractible choice.
Proof. In $\mathcal E_{/W}$, effectiveness says $W\simeq\operatorname{colim}U_\bullet$. Mapping into $V_W$ gives
$$\operatorname{Map}_{/W}(W,V_W)\simeq\operatorname{Tot}\operatorname{Map}_{/W}(U_\bullet,V_W).$$
For proposition-valued certificates, witness coherences are contractible. Structured certificates can require nontrivial overlap and higher-overlap data. A finite benchmark is generally not an effective-epimorphism cover of the complete input domain. Independent explanations of separate layers do not automatically define matching local evidence.
10. Validation limits
Finite-query proposition. A deterministic validator with only value-oracle access to arbitrary Boolean functions on finite $X$, using fewer than $|X|$ queries, cannot accept the zero function while soundly certifying universal zero output. Run it on the zero function and change the function only at an unqueried input: the complete adaptive transcript is unchanged. This does not restrict symbolic proofs obtained by inspecting a program or its weights.
Unbounded-program proposition. If generalized networks effectively represent arbitrary unbounded programs, no total computable reader decides whether every presented trace eventually emits a particular symbol. Simulate a machine and emit exactly on halting; a reader would decide halting. The statement does not apply to every fixed finite feedforward family.
Fresh-validation proposition. For at most $m$ adaptively proposed candidates, draw $n$ fresh conditionally independent Bernoulli trials for each candidate from the fixed task distribution. Simultaneously $|\widehat p_j-p_j|\leq\epsilon$ with probability at least $1-\alpha$ if
$$n\geq\log(2m/\alpha)/(2\epsilon^2).$$
Conditional Hoeffding inequality bounds each failure probability by $2e^{-2n\epsilon^2}$; integrate over history and take a union bound. Adapting candidates to a reused holdout does not meet the hypothesis. Distributional confidence is not universal safety.
11. Artifacts, authorship, and claim boundary
The complete paper contains full definitions, proofs, architecture mappings, limitations, and primary references. The package supplies its LaTeX source and a standard-library Python exact-arithmetic program. Its 184 passed checks illustrate hidden-unit symmetry; a one-coordinate repair of ReLU to absolute value with preservation on nonnegative inputs; graph equivariance; metric and recurrent propagation; finite stochastic laws; compatible and incompatible finite-cover witnesses; observation ambiguity; interacting edits; and the finite-oracle counterexample.
Finite enumeration is evidence about these executable witnesses, not machine-checked verification of all theorems. No trained model, source benchmark reproduction, GPU experiment, verified compiler, or proof-assistant certificate is claimed. An anonymous human requester is the responsible submitting party; substantial AI assistance was used for mathematical development, drafting, code, and artifact preparation. No independent referee report or editorial decision is asserted.
Primary references
1. Takehi, Teoh, Zhang, Khattab (2026), motivating source: https://rikiyat.github.io/blog/weight-space-programming/ 2. Fong, Spivak, Tuyeras, Backprop as Functor (2019): https://arxiv.org/abs/1711.10455 3. Cruttwell, Gavranovic, Ghani, Wilson, Zanasi, Categorical Foundations of Gradient-Based Learning (2021): https://arxiv.org/abs/2103.01931 4. Navon et al., Equivariant Architectures for Learning in Deep Weight Spaces (2023): https://proceedings.mlr.press/v202/navon23a.html 5. Lindner et al., Tracr (2023): https://arxiv.org/abs/2301.05062 6. Vagner, Spivak, Lerman, Algebras of Open Dynamical Systems on the Operad of Wiring Diagrams (2015): https://arxiv.org/abs/1408.1598 7. Fritz, A synthetic approach to Markov kernels, conditional independence and theorems on sufficient statistics (2020): https://arxiv.org/abs/1908.07021 8. Lurie, Higher Topos Theory (2009, updated): https://www.math.ias.edu/~lurie/papers/HTT.pdf 9. Lurie, Higher Algebra (2017): https://www.math.ias.edu/~lurie/papers/HA.pdf 10. Hoeffding, Probability Inequalities for Sums of Bounded Random Variables (1963): https://doi.org/10.1080/01621459.1963.10500830 11. Turing, On Computable Numbers, with an Application to the Entscheidungsproblem (1937, received 1936): https://doi.org/10.1112/plms/s2-42.1.230