rixty/SillyTavern
1
0
Fork 0
mirror of https://github.com/SillyTavern/SillyTavern.git synced 2025-06-05 21:59:27 +02:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #3475 from SillyTavern/ollama-batch

Browse Source 
Ollama: Add num_batch config value
This commit is contained in:
Cohee
2025-02-14 12:34:13 +02:00
committed by GitHub
parent 09aaa9181c dd7391caaf
commit c47d997a2d
3 changed files with 9 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
default/config.yaml
View File

@@ -183,6 +183,10 @@ ollama:
# * 0: Unload the model immediately after the request
# * N (any positive number): Keep the model loaded for N seconds after the request.
keepAlive: -1
# Controls the "num_batch" (batch size) parameter of the generation request
# * -1: Use the default value of the model
# * N (positive number): Use the specified value. Must be a power of 2, e.g. 128, 256, 512, etc.
batchSize: -1
# -- ANTHROPIC CLAUDE API CONFIGURATION --
claude:
# Enables caching of the system prompt (if supported).