mikhail/llama.cpp-mtp-turboquant
1
0
Fork 0
Code Issues Pull Requests Actions 28 Packages Projects Releases Wiki Activity

docs: document usage object in server timings response (#23110)

Browse Source 
* docs: document `usage` object in server timings response

Co-Authored-By: julien-agent <Agents+cyolo@huggingface.co>

* Apply suggestion from @julien-c

---------

Co-authored-by: julien-agent <Agents+cyolo@huggingface.co>
This commit is contained in:
Julien Chaumond
2026-05-15 19:33:12 +02:00
committed by GitHub
parent 72e60f500d
commit 6831fe470c
1 changed files with 16 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/server/README.md
+16
View File
@@ -1322,6 +1322,22 @@ This provides information on the performance of the server. It also allows calcu
The total number of tokens in context is equal to `prompt_n + cache_n + predicted_n` The total number of tokens in context is equal to `prompt_n + cache_n + predicted_n`
The response also includes a standard `usage` object:
```js
{
// ...
"usage": {
"completion_tokens": 48,
"prompt_tokens": 44,
"total_tokens": 92,
"prompt_tokens_details": {
"cached_tokens": 0
}
}
}
```
*Reasoning support* *Reasoning support*
The server supports parsing and returning reasoning via the `reasoning_content` field, similar to Deepseek API. The server supports parsing and returning reasoning via the `reasoning_content` field, similar to Deepseek API.