I'd like to migrate over to using "textgen" to mean text-generation APIs
in general, so I've renamed the /textgenerationwebui/* endpoints to
/backends/text-completions/*.
Create one server-sent events stream class which implements the entire
spec (different line endings, chunking, etc) and use it in all the
streaming generators.
TabbyAPI is an exllamav2 only API server that aims to provide a simple
experience for loading and chatting with exl2 models.
SillyTavern currently doesn't have the ability to load and unload models,
so only add the OAI compatible completion endpoints.
The repository can be found here:
https://github.com/theroyallab/tabbyAPI
Signed-off-by: kingbri <bdashore3@proton.me>
These options essentially supersede repetition penalty, but since
ooba also gives the option, add these new sliders in conjunction
with the repetition penalty ones.
More testing is needed to find good default values.
Signed-off-by: kingbri <bdashore3@proton.me>
If an API doesn't use the negative prompt from generate, allow
the negative prompt to be refetched in the specific API. This
requires the full guidance scale object instead of just the number.
Signed-off-by: kingbri <bdashore3@proton.me>
Make the generate function build a negative prompt in addition to the
normal one. This allows for nonconflicting insertion with other extension
prompts and World Info.
Signed-off-by: kingbri <bdashore3@proton.me>
Insertion depth allows for CFG to variably inject itself into the
negative prompt. This is similar to how Author's note works.
However, this method of insertion depth conflicts with AN and
world info where negatives can be meshed between two lines
of those specific insertions.
A custom separator must be wrapped in quotes, otherwise the default
separator is a newline for negative cascading.
Signed-off-by: kingbri <bdashore3@proton.me>
CFG with LLMs works differently than stable diffusion. The main principle
is prompt mixing and utilizing the differences between the two prompts
rather than a full "negative prompt" of what the user doesn't want.
SillyTavern its own way of formatting a prompt sent to an LLM backend.
Therefore, take that prompt and add negatives to it.
Signed-off-by: kingbri <bdashore3@proton.me>
Per-chat CFG applies a CFG setting per-chat only rather than character
or globally. This overrides all other CFG settings (this will be changed).
Also add fixes to remove character CFG entries properly and not to
apply CFG if the scale is 1 as that won't do anything to generation.
Signed-off-by: kingbri <bdashore3@proton.me>
CFG is bundled as an extension which is more flexible in terms of
storing settings and consolidating code.
Global CFG is currently supported and the old power user setting
is auto-migrated.
Signed-off-by: kingbri <bdashore3@proton.me>
Cohee
valadaptive
kingbri
RossAscends
valden80
50h100a
Cohee
ThisIsPIRI