OpenAI calls this the "Chat Completions API", in contrast to their
previous "Text Completions API", so that's what I'm naming it; both
because other services besides OpenAI implement it, and to avoid
confusion with the existing /api/openai route used for OpenAI extras.
I'd like to migrate over to using "textgen" to mean text-generation APIs
in general, so I've renamed the /textgenerationwebui/* endpoints to
/backends/text-completions/*.
Instead of "version" and "koboldVersion", have "koboldUnitedVersion" and
"koboldCppVersion", the latter of which is null if we're not connected
to KoboldCpp.
We now track the loop counter as a parameter of Generate that we
decrement with every recursive call, rather than a global variable,
and it *should* now work with quiet prompt generation.
Generate(), being async, now returns a promise-within-a-promise.
If called with `let p = await Generate(...)`, it'll wait for generation
to *start*. If you then `await p`, you'll wait for generation to
*finish*. This makes it much easier to tell exactly when generation's
done. generateGroupWrapper has been similarly modified.
I searched for all users of tokenizers.API, but missed that the menu
converts the numerical select values directly to enum values. I've used
the special tokenizer value 98 to represent "the tokenizer API for
whichever backend we're currently using".
Sorting with a random comparator doesn't actually shuffle an array.
Depending on the sorting algorithm used, there will be a bias to the
shuffle (see https://bost.ocks.org/mike/shuffle/compare.html).
If you open that link in Firefox, the bias will be especially bad.
Instead of implementing "random" character sort using a random sort
comparator, use the shuffle function instead.
Since we already have a template cache, it makes sense to store the
templates in it *after* compiling them, to avoid the overhead of
re-compiling them every time we call renderTemplate.
I've also changed the cache from an object to a Map--it's more
semantically correct, and avoids weird edge cases like a template named
"hasOwnProperty" or some other function that exists as an object
property.
This means we don't have to pass the "padding" parameter into every
function so they can add the padding themselves--we can do it in just
one place instead.
Store the URLs for each tokenizer's action in one place at the top of
the file, instead of in a bunch of switch-cases. The URLs for the
textgen and Kobold APIs don't change and hence don't need to be
function arguments.
The endpoint was one big if/else statement that did two entirely
different things depending on the value of main_api. It makes more sense
for those to be two separate endpoints.
They function differently and have different logic and API parameters,
so it makes sense to count them as two different APIs. Kobold's API
doesn't return tokens, so it can only be used to count them.
There's still a lot of duplicate code which I will clean up in the
following commits.
We'll be making a distinction between tokenizing *on* the server itself,
and tokenizing via the server having the AI service do it. It makes more
sense to use the term "remote" for the latter.
Instead of identifying each connect button by ID, we can just use the
.api_button class. The .menu_button class *would* override it due to CSS
cascade rules (specifically, declarations later in the stylesheet apply
over ones that appear earlier), but the `.menu_button.api_button`
selector has a higher *specificity* and hence works.
Have all the get(...)Status and event handler registrations in the same
areas, rather than having the NovelAI ones far away. I want to
eventually move all the API-specific stuff into separate modules, but
this will make things cleaner for the time being.
This adds a bit of duplicate code for the time being, but ultimately
makes the code less confusing because we only need to include the bits
that are relevant to the specific API in each function. We can also
remove API parameters that are useless depending on the endpoint.
based
Cohee
valadaptive
LenAnderson
artisticMink