Galahad, une couche mémoire pour vLLM, SGLang et llama.cpp, évite de relire les mêmes textes

We ask how much of the budget beneath that ceiling is spent on work the model has already done. Serving is stateless across requests: a model that answers a second question about a document recomputes the document’s attention state from the first token.