From aeba840c085417083e3afa8bf89d406c24e10580 Mon Sep 17 00:00:00 2001
From: Alex Polvi <alex@polvi.net>
Date: Thu, 8 Oct 2026 16:49:34 -0700
Subject: [PATCH] Explain suspend semantics, vLLM settings and concurrency
 performance

---
 README.md | 32 +++++++++++++++++++++++++++-----
 1 file changed, 27 insertions(+), 5 deletions(-)

diff --git a/README.md b/README.md
index c5e2bc0..d57e7ea 100644
--- a/README.md
+++ b/README.md
@@ -81,7 +81,7 @@ ax resume task sam-hello         # starts it; the first start can take a minute,
                                  # so re-run if it times out
 ax ssh sam-hello -- ls /workspace  # the repo is cloned into /workspace/<repo-name>
 ax ssh sam-hello -- sh -c 'cd /workspace/your-repo && pi -p "explain the build system"'
-ax suspend task sam-hello        # frees the worker; files and state are kept
+ax suspend task sam-hello        # frees the worker; /workspace is kept
 ax resume task sam-hello         # picks up where it left off
 ax delete task sam-hello         # when done
 ```
@@ -93,6 +93,9 @@ workspace.
 
 Tasks are immutable: to change one, delete it and apply again.
 
+Suspend saves only `/workspace`. It isn't a RAM snapshot: resume starts a fresh
+container with your `/workspace` restored, and everything outside it resets.
+
 ## 5. Using pi with Qwen
 
 ### The model
@@ -101,11 +104,30 @@ Tasks are immutable: to change one, delete it and apply again.
   tool calling and reasoning ("thinking").
 - 128K-token context (131,072). Large repos still need the agent to read
   selectively, which it does by default.
-- Shared: the server runs at most 8 requests at a time across everyone,
-  including Alex's other work, so responses slow down when it's busy.
+- One RTX A6000, shared by everyone including Alex's other work. The weights
+  are FP8, with an FP8 KV cache, prefix caching and MTP speculative decoding.
 - From inside a sandbox it's also a plain OpenAI-compatible API at
   `http://vllm.vllm.svc:8000/v1`, if you want to script against it directly.
-  It isn't reachable from your own machine.
+  It isn't reachable from your own machine. Read the live settings with
+  `curl -s http://vllm.vllm.svc:8000/metrics | grep config_info`. They're
+  read-only: changing them restarts the server for everyone.
+
+### Performance: one long request at a time is best
+
+Decode reads all ~29 GB of weights per token, so concurrent requests share the
+GPU's memory bandwidth. Long prompts also prefill in chunks that stall the
+other stream. Per-stream decode speed from the server's benchmark:
+
+| Prompt size | 1 request | 2 at once (each) |
+|---|---|---|
+| 1k tokens | 40 tok/s | 33 tok/s |
+| 16k | 35 | 22 |
+| 64k | 39 | 6 |
+| 120k | 44 | 2 |
+
+pi transcripts grow every turn, so two busy agents slow each other down a lot.
+When the server is busy (or for unrelated questions), start fresh with
+`--no-session` instead of `-c`, and keep context small.
 
 ### pi is preconfigured
 
@@ -123,7 +145,7 @@ T=sam-hello R=/workspace/your-repo
 # one-shot question or task
 ax ssh $T -- sh -c "cd $R && pi -p 'find where config is loaded and explain it'"
 
-# keep going in the same conversation (survives suspend/resume)
+# keep going in the same conversation (a session file in /workspace/.pi, so it survives suspend)
 ax ssh $T -- sh -c "cd $R && pi -p -c 'now add a test for the env-var override'"
 
 # feed it input