Custom --grammar support [for koboldcpp] (#1161)

* Basic kobold grammar implementation

This is probably jank as all hell, I don't write js, but I think I got it to work.

* No value by default

* Visual cleanup

+ reworded it a bit

* Conditionally enable Grammar based on version flag. Fix layout

---------

Co-authored-by: Cohee <18619528+Cohee1207@users.noreply.github.com>
This commit is contained in:
kalomaze
2023-09-21 07:21:59 -05:00
committed by GitHub
parent 49c26f3810
commit ad9382a98c
3 changed files with 46 additions and 3 deletions

View File

@ -375,6 +375,7 @@ app.post("/generate", jsonParser, async function (request, response_generate) {
mirostat: request.body.mirostat,
mirostat_eta: request.body.mirostat_eta,
mirostat_tau: request.body.mirostat_tau,
grammar: request.body.grammar,
};
if (!!request.body.stop_sequence) {
this_settings['stop_sequence'] = request.body.stop_sequence;