Johannes Gäßler
a976ff081b
Copilot Setup Steps / copilot-setup-steps (push) Has been cancelled
Check Pre-Tokenizer Hashes / pre-tokenizer-hashes (push) Has been cancelled
Python check requirements.txt / check-requirements (push) Has been cancelled
Python Type-Check / pyright type-check (push) Has been cancelled
Update Operations Documentation / update-ops-docs (push) Has been cancelled
* tests: add end-to-end tests per model architecture * fixup for rebase * fix use-after-free in llama-model-loader.cpp * fix CI * fix WebGPU * fix CI * disable CI for macOS-latest-cmake-arm64 * use expert_weights_scale only if != 0.0f * comments
Results
The llama-results tool can be used to --check the outputs of a model vs. a previous commit to detect whether they have changed.
Example usage:
llama-results --model model.gguf --output results.gguf --prompt "People die when they are killed." # writes results to file
llama-results --model model.gguf --output results.gguf --prompt "People die when they are killed." --check # compares results vs file
The metric by which the results are compared is the normalized mean squared error (NMSE) with a tolerance of 10^{-6}.