Independent scholarship · Open accessThe monthly digest

Editorially reviewed

Evolutionary Selection for Agent Populations under Comparable Evaluation

We model inner agent evaluations as scored kernels and outer evolutionary selection as exponential reweighting followed by variation. We characterize exact aggregation and bound the effects of stale or mismatched scores on one step and on repeated updates under contracting variation.

Anonymous← All research

Execution and selection

An agent population may allocate work to specialists, evaluate alternative strategies, or deploy variants generated by an inner improvement process. Its outer update depends on which evaluations can be compared. We analyze exponential reweighting followed by variation, with measurable strategy spaces, exact quotient conditions, and quantitative guarantees for mismatched scores.

Let $S$ be a standard Borel space of immutable strategy versions. A context $c$ specifies a task law $\lambda_c$ on $D_c$, an execution budget and environment, an outcome space $Z_c$, a probability kernel $K_c:S\times D_c\to\mathcal P(Z_c)$, and a bounded measurable score $r_c:D_c\times Z_c\to\mathbb R$. Define

$$F_c(s)=\int_{D_c}\int_{Z_c} r_c(d,z)\,K_c(s,d,dz)\,\lambda_c(dd).$$

A scored failure and an omitted observation may define different estimands. Context comparisons therefore specify the applicable task distributions, execution conditions, and scoring rules. An endogenous score can depend on the current opponent law $m$. For a bounded measurable payoff $a$, put $F_m(s)=\int_S a(s,y)\,m(dy)$.

For a population law $m$ and a positive weight $w$ with finite integral, define selection and the outer transition by

$$\mathcal S_w(m)(B)=\frac{\int_B w\,dm}{\int_S w\,dm}, \qquad T(m)=\mathcal S_w(m)V ,$$

where $V:S\to\mathcal P(S)$ is a variation kernel. For score-based selection use $w=e^{\eta F}$ with $\eta\ge0$. A shared additive score offset cancels upon normalization. A common positive change of score units can be offset by rescaling $\eta$. Selection preserves the measure class of $m$ because $w>0$. Variation can then move mass into previously unoccupied regions.

For bounded $F$ and bounded measurable $\varphi$, differentiating numerator and denominator under the integral yields

$$\left.\frac{d}{d\eta}\right|_{\eta=0}\int_S\varphi\,d\mathcal S_{e^{\eta F}}(m) =\int_S\varphi\left(F-\int_S F\,dm\right)dm .$$

This is the weak replicator field with $F=F_m$ at the current population. For a fixed score $F$ and identity variation, the derivative of the expected score along the exponential family is $\operatorname{Var}_{\mathcal S_{e^{\eta F}}(m)}(F)$. The fixed-score assumption matters for that derivative. This connection places the update within replicator dynamics and multiplicative weights [1–3].

Exact reduction and recombination

Let $\pi:S\to Q$ be measurable with $Q$ standard Borel. Write $\bar m=\pi_*m$ and let $\bar w=d\pi_*(wm)/d\bar m$, with a positive version almost everywhere. A measurable $\bar m$-null set has $m$-null inverse image, so the derivative exists. Pushing forward the numerator of $\mathcal S_w(m)$ gives $\bar w\bar m$. Its total mass is $\int_S w\,dm$, which proves

$$\pi_*\mathcal S_w(m)=\mathcal S_{\bar w}(\pi_*m).$$

For an exponential weight, the reduced weight is the conditional average of the exponential. The distribution inside the fibre can change that average. If $w=v\circ\pi$, however, $v$ can be used for every $m$.

For a fixed $w$, assume the image of $\pi$ contains at least two points. If a single reduced weighting works for all $m$, then $w$ is constant on each fibre. Indeed, for $\pi(s)=\pi(s')$ choose $y$ over a different image point. The mixtures $(\delta_s+\delta_y)/2$ and $(\delta_{s'}+\delta_y)/2$ give the same reduced population. Their selected masses over the first point are $w(s)/(w(s)+w(y))$ and $w(s')/(w(s')+w(y))$. Equality and positivity of $w(y)$ imply $w(s)=w(s')$. A one-point image imposes no constraint. The forward factorization condition requires a measurable reduced weight.

For variation the corresponding condition is strong lumpability. Suppose a kernel $\bar V$ satisfies $V(s,\pi^{-1}C)=\bar V(\pi(s),C)$ for every $s$ and measurable $C\subseteq Q$. Integration against any probability law $n$ then gives $\pi_*(nV)=(\pi_*n)\bar V$. With $w=v\circ\pi$ we obtain

$$\pi_*(\mathcal S_w(m)V)=\mathcal S_v(\pi_*m)\bar V .$$

The identity applies at each round for an endogenous score satisfying $F_m=\bar F_{\pi_*m}\circ\pi$. In this case the reduced evolution can be run from the reduced population alone. This uses composition of Markov kernels [4].

For shards with $m=\sum_i\alpha_i m_i$, use the same weight $w$ and record $Z_i=\int_S w\,dm_i$. Direct expansion of the numerator gives

$$\mathcal S_w(m)=\frac{\sum_i\alpha_i Z_i\mathcal S_w(m_i)}{\sum_i\alpha_i Z_i}.$$

Thus exact recombination uses the local selected law and its normalizer. The equality holds for a countable mixture with finite positive total normalizer.

Sensitivity to score mismatches

For probability measures $m,n$, let $d_{\rm TV}(m,n)=\sup_B|m(B)-n(B)|$. If a function takes values in an interval of length $L$, the difference of its expectations is at most $L d_{\rm TV}(m,n)$. This follows by rescaling to $[0,1]$ and integrating the differences of superlevel probabilities.

Projective sensitivity. Suppose $w,\widehat w>0$ are $m$-integrable and $\log(\widehat w/w)\in[\ell_-,\ell_+]$ almost everywhere. Then

$$d_{\rm TV}(\mathcal S_{\widehat w}(m),\mathcal S_w(m)) \le \tanh((\ell_+-\ell_-)/4).$$

Set $p=\mathcal S_w(m)$, $r=\widehat w/w$, $u=\int r\,dp$, $a=e^{\ell_-}$, and $b=e^{\ell_+}$. The tilted law has density $r/u$ with respect to $p$. If $a=b$ the two laws coincide. Otherwise the secant bound for the convex function $x\mapsto|x-u|$ on $[a,b]$ gives

$$d_{\rm TV}=\frac{\int|r-u|\,dp}{2u} \le\frac{(u-a)(b-u)}{u(b-a)} \le\frac{\sqrt b-\sqrt a}{\sqrt b+\sqrt a}.$$

The last inequality uses $u+ab/u\ge2\sqrt{ab}$. The final ratio equals the stated hyperbolic tangent. This specializes the Hilbert projective distance control of total variation [5].

For a score estimate with $|\widehat F-F-b_0|\le\epsilon$ almost everywhere after some shared offset $b_0$, the exponential weight bound is $\tanh(\eta\epsilon/2)$. If a simultaneous uniform estimate holds with a specified probability, the selection bound holds on that same event.

If the range of $a(s,-)$ lies in an interval of length $L$ for each $s$, then $\sup_s|F_m(s)-F_n(s)|\le Ld_{\rm TV}(m,n)$. Consequently a stale score estimated against $n$ with additional uniform error $\epsilon$ incurs at most $\epsilon+Ld_{\rm TV}(m,n)$ of score error when used at $m$. The age of the estimate does not appear without information about the change in population.

For two contracts transported to common task and outcome spaces and using scores in $[0,1]$, replace the task law, execution kernel, and score rule one at a time. The change in the expected score, uniformly over strategies, is at most

$$d_{\rm TV}(\lambda_c,\lambda_{c'}) +\sup_{s,d} d_{\rm TV}(K_c(s,d,\cdot),K_{c'}(s,d,\cdot)) +\sup_{d,z}|r_c(d,z)-r_{c'}(d,z)|.$$

The first two terms use the bounded expectation inequality. The last uses the difference of the two scoring functions. This gives a concrete input to the projective bound when a comparable context has been specified.

Propagation under variation

For a fixed weight satisfying $0<a\le w\le b$,

$$d_{\rm TV}(\mathcal S_w(m),\mathcal S_w(n))\le (b/a)d_{\rm TV}(m,n).$$

For each event $B$, put $c_B=\mathcal S_w(n)(B)$. The difference is the integral of $w(\mathbf 1_B-c_B)$ against $m-n$, divided by $\int w\,dm$. The integrand ranges in an interval of length $b$ and the denominator is at least $a$.

Suppose the endogenous score $F_m$ has oscillation at most $M$ uniformly in $m$ and is $L$-Lipschitz in total variation as above. Changing the population with fixed weights costs at most $e^{\eta M}d_{\rm TV}(m,n)$. Changing scores at fixed population costs at most $(\eta L/2)d_{\rm TV}(m,n)$ by the projective estimate and $\tanh x\le x$ for $x\ge0$. Thus the selector is $C$-Lipschitz, where $C=e^{\eta M}+\eta L/2$.

Let $\kappa=\sup_{s,s'}d_{\rm TV}(V(s,\cdot),V(s',\cdot))$ be the Dobrushin coefficient of variation [6]. Applying the expectation inequality to $s\mapsto V(s,B)$ gives $d_{\rm TV}(mV,nV)\le\kappa d_{\rm TV}(m,n)$. Compare a reference sequence $m_t$ using the true scores and an approximate sequence $\widetilde m_t$ using scores with errors at most $\epsilon_t$ modulo common shifts at its current population. Let $d_t=d_{\rm TV}(\widetilde m_t,m_t)$ and $\rho=\kappa C$. Inserting the exact selector evaluated at $\widetilde m_t$ gives

$$\begin{aligned} d_{t+1}&\le\rho d_t+\kappa\tanh(\eta\epsilon_t/2),\\ d_t&\le\rho^t d_0+\kappa\sum_{j<t}\rho^{t-1-j}\tanh(\eta\epsilon_j/2). \end{aligned}$$

When $\rho<1$ and $\epsilon_t\le\epsilon$, the accumulated contribution is bounded by $\kappa\tanh(\eta\epsilon/2)/(1-\rho)$. This is a stability statement relative to the specified reference transition. The assumptions refer to the variation kernel and the actual score errors. The estimate gives no lower bound on the agents’ objective performance.

Relationship to prior work and reproducibility

Population-based training uses population-level selection and perturbation to improve trained models [7]. Policy-space response oracles grow strategy sets and adapt distributions over those sets [8]. Automated design of agentic systems searches over candidate agents [9]. The journal’s earlier editorial primer describes the inner adaptation and outer credit split [10]. Here the contribution is a mathematical analysis of when comparable inner evaluations permit exact quotient selection and how perturbing those evaluations affects the outer dynamics.

An audit record for the proposed update identifies the strategy versions, task distribution and execution contract, score or scoring rule, its evaluation population where relevant, the learning rate, and the variation rule. For shard recombination it records the normalizers. An empirical application would need to measure or bound its own error and dependence across scores. The manuscript supplies proofs and a separate reusable algebraic verification artifact.

Lean 4.19.0 with pinned mathlib checks the algebraic replicator vector field and deterministic aggregation over arbitrary finite strategy types. In particular, the artifact checks conservation, simplex preservation under strictly positive weights, and exact commutation of the rational normalized selection step with aggregation when weights descend. The measure-theoretic, exponential, and Dobrushin estimates above use the written proofs. The accompanying source supplies a build command and an axiom audit.

The manuscript, typesetting, and proof artifact were prepared with AI assistance. The submission uses the author's anonymous attribution.

References

1. J. McKenzie Alexander, Evolutionary Game Theory, Cambridge University Press, 2023. 2. S. Arora, E. Hazan, and S. Kale, The multiplicative weights update method, Theory of Computing 8, 2012. 3. S. Sorin, Replicator dynamics: old and new, Journal of Dynamics and Games 7, 2020. 4. T. Fritz, A synthetic approach to Markov kernels, conditional independence and theorems on sufficient statistics, Advances in Mathematics 370, 2020. 5. S. N. Cohen and E. Fausti, Hyperbolic contractivity and the Hilbert metric on probability measures, 2024 revision. 6. S. Gaubert and Z. Qu, Dobrushin’s ergodicity coefficient for Markov operators on cones, Integral Equations and Operator Theory 81, 2015. 7. M. Jaderberg et al., Population based training of neural networks, 2017. 8. M. Lanctot et al., A unified game-theoretic approach to multiagent reinforcement learning, NeurIPS, 2017. 9. S. Hu, C. Lu, and J. Clune, Automated design of agentic systems, ICLR, 2025. 10. Meta Journal of Agentic Meta Algorithms, Self-improvement as a bilevel program, editorial primer, 2026.

Associated artifacts

Discussion

Open a careful, constructive conversation about this note.

Comments appear after editorial approval.