Create one server-sent events stream class which implements the entire
spec (different line endings, chunking, etc) and use it in all the
streaming generators.
TabbyAPI is an exllamav2 only API server that aims to provide a simple
experience for loading and chatting with exl2 models.
SillyTavern currently doesn't have the ability to load and unload models,
so only add the OAI compatible completion endpoints.
The repository can be found here:
https://github.com/theroyallab/tabbyAPI
Signed-off-by: kingbri <bdashore3@proton.me>
These options essentially supersede repetition penalty, but since
ooba also gives the option, add these new sliders in conjunction
with the repetition penalty ones.
More testing is needed to find good default values.
Signed-off-by: kingbri <bdashore3@proton.me>
If an API doesn't use the negative prompt from generate, allow
the negative prompt to be refetched in the specific API. This
requires the full guidance scale object instead of just the number.
Signed-off-by: kingbri <bdashore3@proton.me>
Make the generate function build a negative prompt in addition to the
normal one. This allows for nonconflicting insertion with other extension
prompts and World Info.
Signed-off-by: kingbri <bdashore3@proton.me>
Insertion depth allows for CFG to variably inject itself into the
negative prompt. This is similar to how Author's note works.
However, this method of insertion depth conflicts with AN and
world info where negatives can be meshed between two lines
of those specific insertions.
A custom separator must be wrapped in quotes, otherwise the default
separator is a newline for negative cascading.
Signed-off-by: kingbri <bdashore3@proton.me>
