llama : refactor sampling_info to use buffer_view template (#19368)

* llama : refactor sampling_info to use buffer_view template

This commit updates the sampling_info struct in llama-context to use a
buffer_view template for the logits, probs, sampled tokens, and
candidates buffers.

The motivation for this is to simplify the code, improve type safety
and readability.
This commit is contained in:
Daniel Bevenius
2026-02-11 05:38:13 +01:00
committed by GitHub
parent 612db61886
commit 2cce9fddb7
3 changed files with 107 additions and 115 deletions
+10
View File
@@ -49,6 +49,16 @@ struct time_meas {
int64_t & t_acc;
};
template <typename T>
struct buffer_view {
T * data;
size_t size = 0;
bool has_data() const {
return data && size > 0;
}
};
void replace_all(std::string & s, const std::string & search, const std::string & replace);
// TODO: rename to llama_format ?