ea49b33e3c
Use abort endpoint for all supported koboldcpp generations
2023-06-14 19:56:55 -07:00
55fcebb4e1
Revert "Send abort signal to koboldcpp server on abort."
...
This reverts commits 0cf9d8895cee8b60b5fc
2023-06-14 13:28:00 -07:00
ee8b60b5fc
add async to abort request.
2023-06-14 12:52:32 -07:00
0cf9d8895c
Send abort signal to koboldcpp server on abort.
...
Simply aborting on client side does not end generation of
streaming tokens on the koboldcpp side, creating an awful user
experience where starting a new generation does not work, and
potentially causing segfaults in koboldcpp if another generation
is started while the previous one is still processing.
2023-06-14 12:25:15 -07:00
524cc881d4
= =/= ==
...
Can't believe I missed this yesterday.
2023-06-12 11:43:18 -07:00
b652b64863
Correct testing fail.
2023-06-11 19:14:41 -07:00
e9427a14d5
Add warning if streaming is unsupported on kobold version.
...
minimum version of koboldcpp 1.30 is the only known implementation
to support SSE token streaming, assuming 1.30 releases with the
feature enabled.
2023-06-11 18:36:19 -07:00
698461c188
Implement support for koboldcpp SSE streaming.
...
koboldcpp added SSE streaming to its API as of:
https://github.com/LostRuins/koboldcpp/pull/220
This has not yet been added to release, but I figued I'd try to
get the work done to make it ready.
Work mostly hijacked from:
10bbc97069
2023-06-11 13:24:07 -07:00
50526a16b9
Fix oobabooga
2023-04-21 14:38:38 +03:00
fc8e0c4918
Fix ooba/kobold compat mode #96
2023-04-18 12:33:56 +03:00
c5a3ec7d6e
Determine if stop_sequence could be used. Closes #93
2023-04-17 13:50:41 +03:00
9616b26402
Added stop sequences for Kobold.cpp
2023-04-16 23:02:23 +03:00
48e4e408a4
- ai config ranges more compact
...
- fixed misuse of range-block-range class on text areas
2023-04-12 14:58:00 +09:00
b3498a47eb
Horde support (untested)
2023-04-02 12:01:42 +03:00
19def2536f
Kobold API url formatting
2023-03-25 23:38:51 +02:00
15edcbb4ac
Add single-line mode for Kobold
2023-03-20 00:31:12 +02:00
d7547ef869
Move all settings to KAI script
2023-03-18 19:13:43 +02:00
7b6eaf6ed0
Add explanation for advanced settings
2023-03-18 17:58:09 +02:00
