# Autoresearch is moving into the systems that run AI

> Agents can optimize GPU kernels and inference systems. I care about whether those changes make the full workload faster, and where an LLM improves on conventional search.

Author: HARTRO
Published: 2026-10-06
Updated: 2026-10-15
Topics: Research
Canonical URL: https://hartro.com/blog/autoresearch-next-frontier/

![An isometric compute platform connects a processor, memory, and a gear, showing optimization within the wider system.](https://hartro.com/assets/blog/covers/infrastructure-illustrated-v3.svg)

Making one operation four times faster does not make the application four times faster. If that operation consumes 40% of a sequential workload, total runtime falls from 1 to `0.6 + 0.4/4`, or 0.7. The overall speedup is about 1.43 times before new overhead.

That distinction is why I am interested in autoresearch moving into inference infrastructure. Agents can investigate kernels, scheduling, and memory movement together. The useful result is a faster system under the conditions in which it will run.

Here, autoresearch means a loop in which a system proposes an implementation, runs an experiment, and uses the result to choose its next attempt. The examples below automate different parts of that loop, with different amounts of human direction.

## The useful simplicity of the original loop

Karpathy's [autoresearch repository](https://github.com/karpathy/autoresearch) offers a deliberately small setup. An agent changes a training file, runs a five-minute training experiment, checks validation bits per byte, and keeps or discards the change. Data preparation and evaluation utilities sit outside the editable training file; human-written instructions define the research loop.

Its strength is the tight contract. There is an editable surface, a bounded run, and a result that can be compared with the previous result. The fixed wall-clock budget also means the search is tied to the hardware and conditions under which it runs.

That pattern is portable. The five-minute limit and language-model training task are examples. The general requirement is a cheap enough experiment with an outcome that is both trustworthy and useful.

There are now several approaches under the broad autoresearch label: training-code search, evolutionary algorithm discovery, autonomous kernel optimization, and agents embedded in larger engineering teams. They share an experimental loop while differing substantially in how objectives are defined and who judges success.

## What Z.ai actually reported

In its September 17 account, [Z.ai describes an infrastructure agent](https://z.ai/blog/glm-built-its-inference-infrastructure) powered by GLM-5.3 helping engineers bring GLM-5.3-Flash's inference stack to production. The company reports roughly threefold end-to-end throughput improvement from the initial baseline, with adaptation to production taking less than two weeks. These are company-reported results, not our reproduction.

The diagnostic setup explains how the work became possible. Numerical checks, microbenchmarks, runtime traces, and serving measurements gave the agent different ways to test a hypothesis. One investigation concerned concurrency around KV transfer; kernel work also drew on techniques from existing implementations. Engineers retained responsibility for objectives, boundaries, and critical review. The authors explicitly say the work has not reached recursive self-improvement.

I read this as an account of an agent helping engineers investigate a system they could measure. That is the part I would try to reproduce.

## Kernels make the opportunity concrete

A GPU kernel is a small program carrying out an operation on the accelerator: a matrix operation, normalization, reduction, or attention-related calculation. Efficient implementations depend on shapes, data layout, memory access, and the target device. Small changes can have large effects, but only under the conditions for which they were designed.

[KernelBench](https://arxiv.org/abs/2502.10517) makes correctness and speed part of the same evaluation. Its workloads start from PyTorch operations, and its metric counts implementations that are both functionally correct and faster than a chosen threshold over the baseline. The original paper also examines iterative refinement with execution and profiling feedback.

The [AutoKernel preprint](https://arxiv.org/abs/2603.21331) takes a broader pipeline approach: profile a model, prioritize bottlenecks, generate Triton or CUDA implementations, and refine them through experiments. Its correctness checks include shape coverage, numerical behavior, and edge cases. The authors report substantial gains on selected operations, but those results depend on the operation, hardware, and comparator; they are not a universal multiplier for model inference.

Algorithm search also extends beyond kernels. DeepMind's [AlphaEvolve](https://deepmind.google/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/) combines model-generated programs, automated evaluation, and evolutionary search. Its reported applications include algorithms used in data-center and AI-training efficiency. This is a related research direction with its own system design, rather than another implementation of Karpathy's repository.

## The number that matters is downstream

In real inference systems, operations overlap. A kernel may consume resources needed by communication. Improvements on one batch size can regress another. A benchmark that omits synchronization can measure work being scheduled rather than completed.

An optimization campaign therefore needs two kinds of evidence. Local measurements tell the agent where to investigate. Representative end-to-end workloads establish whether a candidate is worth adopting. The relationship between them has to be checked, rather than assumed.

A practical record for every accepted change should preserve the workload, hardware, precision requirements, baseline implementation, correctness checks, and repeated timing results. It should also record where the change fails to help. Those applicability limits are part of the result.

For a kernel replacement, that acceptance record can be short:

```text
Workload: model revision, input shapes, batch sizes, concurrency
Environment: accelerator, driver, compiler, precision
Comparison: baseline and candidate revisions; timing procedure
Correctness: reference outputs, tolerances, edge cases, failures
Measurements: repeated local timings and end-to-end results
Decision: accepted workloads, regressions, and rollback conditions
```

This is a record template. Filling it in forces the local speedup and the deployment decision into the same review.

## An LLM is not automatically the best search algorithm

A [study of autoresearch and hyperparameter optimization](https://arxiv.org/abs/2603.24647) found classical methods outperforming pure LLM approaches in its fixed search spaces and compute budgets. A hybrid approach combining optimizer state with LLM proposals performed best in those experiments. This is a study of a particular setup, not a verdict on every possible research task.

I would use the agent where it changes what can be searched. A conventional optimizer may be the better way to explore a known numerical space. A language model becomes more distinctive when it can formulate a different implementation, investigate a failure, or connect a profiling observation to a plausible code change.

## Where I would use it

I would choose a bottleneck that is expensive enough to matter and cheap enough to test repeatedly. Profiling should explain where time goes; the full workload should decide which changes are retained. The kernel acceptance record above keeps both measurements in the same review.

A known numerical search space may need a conventional optimizer. A puzzling synchronization delay or a different data layout gives an agent more room to contribute. The question is whether it can find a useful implementation that the existing search would miss.

*Research and product references checked on October 15, 2026. Reported external results are attributed to their authors; preprints and vendor reports are not independent validation.*