From 100 Seconds to 26 Seconds: Two Techniques That Make LLM Agents 20x Faster on Home PCs

If a home PC running an LLM agent requires waiting 100 seconds for the first word to be generated, ensuring practical usability is difficult. However, a case reported on November 2, 2026, showed that reducing this latency to 26 seconds using only standard gaming GPUs and standard memory—without data centers or cloud infrastructure—resulted in a 20x improvement in perceived agent session speed ¹.

The core issue was not making computations faster, but rather reusing previously computed results instead of discarding them. Two techniques applied to the Strata engine, 'KV Splice' and 'Resident-only Routing', were validated for eliminating bottlenecks, particularly in coding agent environments where system prompts undergo frequent changes ¹, ². This article examines specifically which hardware was used, how the latency was eliminated through what principles, and what mechanisms were involved.

The bottleneck was not in 'inference' but in 'prompt reprocessing'

When LLM agents feel slow, we usually think of token generation speed, i.e., inference speed. However, the bottlenecks observed in the Strata project and the OpenCode agent environment were not in inference but in the prompt input stage ¹, ².

Every time a session opens, the coding agent OpenCode inserts contextual information such as the working directory, current date, and Git status at the top of the system prompt ². This information changes slightly each time, leading to frequent Prefix Cache Misses ¹, ². When the cache fails, the model must re-read content it already knows. Actual measurements showed that the system prompt in the OpenCode environment reached approximately 37K tokens, and full reprocessing of these 37K tokens caused approximately 37K token reprocessing delays to repeat with every session ¹, ².

As a result, the time taken to produce the first token soared to approximately 100 seconds ¹, ². On a home PC configured with an RX 9070 XT graphics card, Ryzen 5800X3D CPU, DDR4 64GB memory, and PCIe 3.0 NVMe SSD, the prompt processing time for the DeepSeek V4.1 Flash model recorded approximately 100 seconds ¹. The nature of the problem is clearly revealed by the fact that this stage is not the phase of actually 'generating' a large model, but rather the phase of refreshing context that has already been read.

KV Splice: Reading only changed lines instead of recalculating everything

The first technique that resolved this bottleneck is KV Splice. This method analyzes the KV Cache state already computed in previous sessions and reads only the lines where the prompt has changed ¹, ². In other words, instead of re-reading all 37K tokens from scratch, it preserves reusable caches and incrementally processes only the changed parts.

According to the proposed operational method, the engine aligns the model's internal state and maintained conversation into token runs of 32-token units to select the maximum reuse path ². When conversations are discontinuous, deleted lines from the maintained state are cut out, and subsequent source rows and window rings are adjusted ². At this point, RoPE gaps (rope_off) are recorded to handle jumps in position values, and this deletion table is stored in the slot with an DS4CUTS1 trailer ². This performs approximate reuse by moving cached rows via RoPE gaps ¹.

Applying this method reduced prompt processing time from approximately 100 seconds to 26 seconds ¹. This represents approximately a 4x speedup, achieved by reusing 36,097 tokens ¹, ². In simple synthetic session tests, the token reuse rate for the second session exceeded 99%, responses were returned in about 10 seconds, and efficiency was demonstrated by reusing 53,080 tokens when processing the follow-up to a 54,000-token conversation ². The reason a 4x speedup in prompt processing led to a 20x improvement in perceived agent session speed is that most of the wait time was concentrated in this input stage ¹.

Resident-only Routing: Eliminating SSD streaming bottlenecks during prompt reading

The second technique is 'Resident-only Routing'. This method routes processing to prioritize only Experts already loaded into VRAM and RAM during prompt reading, and was applied in parallel with KV Splice ¹, ².

Strata uses a method of streaming directly from SSD for models that cannot fit into memory ³, ⁴. Large-scale models like DeepSeek V4.1 require over 250GB of disk space, and data that does not fit in memory is read from the SSD ³, ⁴. The problem was that this streaming occurred unnecessarily during the prompt reading stage. In the previous approach, a bottleneck emerged where large amounts of data were streamed from the SSD even while reading the prompt; there were reports of situations where 58GB was streamed from the SSD for processing small prompts ¹.

Resident-only Routing suppresses SSD streaming during the prompt reading stage and forces the use of only Expert modules already loaded in VRAM and RAM ¹, ². This eliminates the I/O bottleneck in the input stage. For reference, in an environment with an RTX 4070 SUPER 12GB and 32GB RAM, a streaming load of reading a total of 2.2TB from the SSD over 19 minutes was observed, a phenomenon stemming from a structure that directly streams model weights rather than swapping ⁵. Resident-only Routing operates by bypassing this structural characteristic during the prompt input stage.

The arithmetic of perception that led this combination to '20x'

To summarize the numbers, KV Splice alone reduced prompt processing from 100 seconds to 26 seconds, resulting in approximately a 4x speedup ¹. With Resident-only Routing applied in parallel, the SSD streaming bottleneck in the input stage was additionally eliminated, and the overall response speed of the agent session was reported to improve by 20x ¹. The reason for the difference between 4x and 20x is that prompt input accounted for a significant portion of the session start latency, and within that input stage, two layers of bottlenecks—cache misses and SSD streaming—were superimposed ¹, ². It was a structure where perceived latency dropped sharply only when both layers were addressed simultaneously.

At the implementation stage, it was proposed as an opt-in option via STRATA_DS4_REORDER=1; in the published test branch, it is enabled by default and can be disabled using STRATA_DS4_REORDER=0 ². This shows that these experimental techniques are already being deployed in forms applicable to user environments, namely as environment variable-based switches ².

Why agents are practically usable on home PCs

Behind the attention this case is receiving lies the fact that an environment has been established where local agents can be used without dependence on the cloud. The Strata engine makes it possible to run DeepSeek V4.1 Flash at an output speed of 4–6 tok/s on consumer hardware, with environments like the RTX 5070 presented as benchmarks ³. Since it supports OpenAI and Anthropic-compatible APIs locally, coding tools like OpenCode or Codex CLI connect directly to the models on a home PC ³.

In other words, the agent framework remains unchanged, and optimizing only the input stage of the local inference engine improves perceived session speed. Strata is compatible with various lightweight models such as Qwen3.8-Flash-Next and provides an environment for building agents on home PCs ³. The fact that responses return from a local host instead of waiting for cloud API calls aligns well with the application point of this optimization.

The approximate nature left by the techniques and the scope of verification

This optimization is not a method that accurately replicates the computation itself, but rather an approximate approach that reuses caches by shifting them. Reused tokens are calculated including the context of deleted text, exhibiting the approximate characteristic of shifted lines being placed after the tail of the system prompt ². It was also noted that RoPE gaps cause jumps in position values, and host load may increase during the alignment process ². It was pointed out as a limitation that routing changes may cause answers to differ slightly ¹.

The scope of verification is also limited to the current state. This patch was verified only on the HIP (gfx1201) platform via a HIP build in an RX 9070 XT and ROCm environment; verification for CUDA, SYCL, and Metal builds and the C++ test suite has not yet been completed ¹, ². Therefore, the figure of '20x' is a result reported under the verified hardware and platform conditions, and should not be interpreted as a guarantee that applies identically to all local environments ¹, ².

Implications that change the perspective on improving agent speed

The perspective on making LLM agents fast on home PCs does not lie solely in buying a faster GPU. It was identified that the dominant variables for session start latency are how much of the already computed KV Cache is reused, and how much unnecessary SSD streaming is prevented during the input stage ¹, ². KV Splice transformed 37K token reprocessing into 36,097 token reuse, reducing the prompt stage by 4x ¹, ², and Resident-only Routing routed processing to use only experts loaded in VRAM and RAM during input, eliminating the SSD bottleneck ¹, ². When these two techniques were applied together on a home gaming GPU combination, the case of a 20x speedup in perceived session speed—from 100 seconds to 26 seconds—was established ¹.

Ultimately, the condition of 'without a data center' is not a premise that acknowledges hardware limitations, but rather a premise that bottlenecks in the input stage, such as cache reuse and routing control, can be bypassed through design. If you use local agents like OpenCode, where the top of the prompt changes frequently, the stages addressed by these two techniques—cache misses and streaming bottlenecks—become the actual starting points for speed improvement. This case, verified in HIP and ROCm environments, leaves follow-up verification as to whether the same benefits will be maintained when extended to other builds like CUDA ², but it is sufficient as a starting point that shifts the perspective on local agent speed from 'inference speed' to 'input bottlenecks'.