Commit Graph

1689 Commits

Author SHA1 Message Date
valadaptive 014416546c Add padding once in getTokenCount
This means we don't have to pass the "padding" parameter into every
function so they can add the padding themselves--we can do it in just
one place instead.
2023-12-09 20:53:16 -05:00
valadaptive 2f2cd197cc Clean up tokenizer API code
Store the URLs for each tokenizer's action in one place at the top of
the file, instead of in a bunch of switch-cases. The URLs for the
textgen and Kobold APIs don't change and hence don't need to be
function arguments.
2023-12-09 20:48:41 -05:00
valadaptive 09465fbb97 Inline most get(...)TokenizerParams calls
For everything except textgenerationwebui, these params are now simple
enough that it doesn't make sense for them to be in a separate function.
2023-12-09 20:35:11 -05:00
valadaptive 30502ac949 Split up Kobold and textgenerationwebui endpoints
The endpoint was one big if/else statement that did two entirely
different things depending on the value of main_api. It makes more sense
for those to be two separate endpoints.
2023-12-09 20:26:24 -05:00
valadaptive 7486ab3886 Separate textgen and Kobold tokenization APIs
They function differently and have different logic and API parameters,
so it makes sense to count them as two different APIs. Kobold's API
doesn't return tokens, so it can only be used to count them.

There's still a lot of duplicate code which I will clean up in the
following commits.
2023-12-09 20:24:56 -05:00
valadaptive 18177c147d Separate remote and server tokenization code paths
This lets us remove extraneous API params from paths where they aren't
needed.
2023-12-09 20:08:48 -05:00
valadaptive ddd73a204a Remove "remote" language from tokenizer functions
We'll be making a distinction between tokenizing *on* the server itself,
and tokenizing via the server having the AI service do it. It makes more
sense to use the term "remote" for the latter.
2023-12-09 19:49:22 -05:00
valadaptive 8bad059a62 Rename /tokenize_via_api endpoint
No redirect for this since I don't expect any extensions to be calling this directly.
2023-12-09 19:29:24 -05:00
LenAnderson a3ec8d709d add support for tab and shift-tab in QR editor 2023-12-09 17:56:36 +00:00
Cohee 2d7811fa46
Merge pull request #1499 from LenAnderson/qr-execute-from-any-preset
Allow /run for quick replies from any preset
2023-12-09 19:49:03 +02:00
LenAnderson d00861d913 allow /run for quick replies from other presets
/run otherPresetName.quickReplyLabel
2023-12-09 17:20:15 +00:00
Cohee e79f6d82da
Merge pull request #1498 from artisticMink/bulkedit-delete-optimization
Only refresh character list after all deletions have been processed.
2023-12-09 18:07:44 +02:00
Cohee b915b89ca9 Fix ESLint 2023-12-09 16:09:10 +02:00
Cohee ad9130b538 Single doc mode: edit on click 2023-12-09 16:01:43 +02:00
artisticMink dfd41a110a Prevent 'no such file or directory' error when mass deleting characters 2023-12-09 14:36:37 +01:00
artisticMink ba3966e148 Only refresh character list after all deletions have been processed. 2023-12-09 14:31:18 +01:00
valadaptive d735b12399 Refactor event stream parsing
I was really overcomplicating this before; this is simpler and faster.
Passes my existing test suite.
2023-12-08 15:05:57 -05:00
valadaptive 699c369443 Remove ignoreBOM
Apparently the ignoreBOM option actually means "include the BOM". I've
added a test for this in my own repository, and will also be submitting
a pull request to MDN to clarify this in their documentation.
2023-12-07 23:15:55 -05:00
Cohee b0e7b73a32 Fix streaming processor error handler hooks 2023-12-08 02:01:08 +02:00
valadaptive 5569a63595 Remove legacy_streaming setting
This was a workaround for older versions of Slaude that implemented SSE
improperly. This was fixed in Slaude 7 months ago, so the workaround can
be removed.
2023-12-07 18:00:36 -05:00
valadaptive cdcd913805 Don't stream events if the API returned a 4xx code 2023-12-07 18:00:36 -05:00
valadaptive 5540c165cf Refactor server-sent events parsing
Create one server-sent events stream class which implements the entire
spec (different line endings, chunking, etc) and use it in all the
streaming generators.
2023-12-07 18:00:36 -05:00
Cohee 6f610204d6
Merge pull request #1488 from LenAnderson/add-index-argument-to-setvar-/-setglobalvar
Add index argument to setvar / setglobalvar
2023-12-08 00:40:14 +02:00
Cohee 2607e787ff Put comment command on hold 2023-12-08 00:38:13 +02:00
Cohee e2f886d796 Revert pipe caching 2023-12-08 00:33:27 +02:00
Cohee 7f7ad6f523 Merge branch 'staging' into backgrounds-endpoint 2023-12-07 23:16:33 +02:00
Cohee 404b1562e5
Merge pull request #1491 from valadaptive/stats-endpoint
Move stats endpoints into their own module
2023-12-07 23:11:12 +02:00
valadaptive 2367f4f38f Rename backgrounds API endpoints 2023-12-07 15:17:19 -05:00
LenAnderson 65f2cc1952 add a comment slash commend /#
for commenting longer ST scripts
2023-12-07 18:29:28 +00:00
LenAnderson d52b5fb947 pass prev pipeResult through if result undefined 2023-12-07 18:28:58 +00:00
LenAnderson 5a719d635a use isArray instead of instanceof 2023-12-07 18:27:11 +00:00
LenAnderson 7bfed3fca1 /addvar on array to push 2023-12-07 18:12:05 +00:00
Cohee c84b6a5101 Add WI inclusion groups 2023-12-07 20:06:06 +02:00
valadaptive 26ebb417f4 Rename stats API endpoints 2023-12-07 13:00:34 -05:00
Cohee e804f27368 Convert value to string for echo. Fix return value 2023-12-07 19:37:55 +02:00
LenAnderson 2ee57afe2a add helpString for index in setvar 2023-12-07 16:51:45 +00:00
LenAnderson db6f782e0f add index to /setvar and /setglobalvar 2023-12-07 16:48:24 +00:00
Cohee b1f68d886c Fix WI field access for UID = 0 2023-12-07 18:45:34 +02:00
Cohee 72adb4c8aa Fix window.ai streaming 2023-12-07 17:42:06 +02:00
Cohee 785be22e51 Merge branch 'staging' into st-object-vars 2023-12-07 16:46:02 +02:00
Cohee e9a49b7997
Merge pull request #1485 from LenAnderson/qr-expand-editor
Add expand button to QR textareas
2023-12-07 15:31:01 +02:00
Cohee b506a17260 Fix expand after drag 2023-12-07 15:29:40 +02:00
LenAnderson afd32d1f4d better handling of array / object vars 2023-12-07 12:56:28 +00:00
LenAnderson 0290916035 stop expanding textarea 2023-12-07 12:29:00 +00:00
LenAnderson a1bfca78b3 add custom titles (tooltips) to QR buttons 2023-12-07 12:20:02 +00:00
LenAnderson d62ef8badc add expand button for QR textareas 2023-12-07 12:04:39 +00:00
Cohee 75abc6101a /input input => /input default 2023-12-07 13:44:49 +02:00
Cohee 0e6802c9d9 Too much lint 2023-12-07 13:35:48 +02:00
Cohee 25db4b6fde Merge branch 'staging' into feat/slash-commands-missing-args 2023-12-07 13:19:15 +02:00
Cohee 8082144c5a Merge branch 'staging' into worldinfo-endpoint 2023-12-07 12:58:57 +02:00
Cohee 0138008031 Merge branch 'staging' into groups-endpoint 2023-12-07 12:50:24 +02:00
Cohee c0fb5a4787
Merge pull request #1473 from valadaptive/chats-endpoint
Move chat endpoints into their own module
2023-12-07 12:36:43 +02:00
RossAscends c51a821b1e fix zen slider rep pen range value transfer 2023-12-07 11:52:10 +09:00
SrJuggernaut d6d0371459 add missing args to some slash commands 2023-12-06 19:26:29 -06:00
valadaptive c746e6a426 Rename worldinfo API endpoints 2023-12-06 20:00:51 -05:00
valadaptive 8027372fc2 Rename group API endpoints 2023-12-06 19:59:58 -05:00
valadaptive 6efe95f4f1 Rename chat API endpoints 2023-12-06 19:58:24 -05:00
Cohee f575e0d61d Add {{currentSwipeId}} / {{lastSwipeId}} macros 2023-12-07 02:35:24 +02:00
Cohee 393a15cd81 Run auto-executed STscripts after message rendered 2023-12-07 02:12:49 +02:00
Cohee 24aff67d50 Fix {{original}} macro left out if original is empty 2023-12-07 01:27:55 +02:00
valadaptive 39d771cc4a Replace path "\" with "/" server-side 2023-12-06 17:22:37 -05:00
Cohee 100dae5fd0
Merge pull request #1444 from valadaptive/characters-endpoint
Move character endpoints into their own module
2023-12-06 23:45:14 +02:00
Cohee 5e282ac7b4 lint: infix spacing 2023-12-06 22:08:06 +02:00
LenAnderson ea7720a7de add /len slash command 2023-12-06 19:03:20 +00:00
LenAnderson a5c595d8ae add list accessors to /getvar and /getglobalvar 2023-12-06 19:03:10 +00:00
Cohee 64496425cc Save chat before reloading 2023-12-06 17:23:54 +02:00
Cohee 61f186764d Fix QR bar popout position 2023-12-06 14:53:29 +02:00
Cohee d36422e558 Fix/optimize bulk favorite 2023-12-06 01:56:07 +02:00
Cohee 356c3a4300 Fix amount gen on zen sliders 2023-12-05 14:12:06 +02:00
Cohee c9fbe75566 Rename WI logic ONE => ANY 2023-12-05 12:04:27 +02:00
Bronya Rand 537e9428e0 proper revert cuz vsc revert weird... 2023-12-05 03:00:26 -06:00
Bronya Rand f9ecc77b10 revert changes after cohee fix 2023-12-05 02:56:52 -06:00
valadaptive b689b8bd30 Rename character API endpoints
Precursor to moving the character API into its own module
2023-12-04 17:35:06 -05:00
Bronya Rand a90fe54e5b bak should not be here 2023-12-04 16:05:05 -06:00
Bronya Rand f71b83b14f add comment on new code and add () for logic checks 2023-12-04 16:00:02 -06:00
Bronya Rand 085daaeff7 simplify and proper NOT check fix 2023-12-04 15:57:04 -06:00
Cohee ddd16c1469
Merge pull request #1452 from valadaptive/assets-router
Use Express router for assets + "files" endpoints
2023-12-04 21:29:52 +02:00
Cohee 1ac494d468 Don't attempt to send files on dry runs. 2023-12-04 21:28:36 +02:00
valadaptive d24c1dde10 Use Express router for assets + "files" endpoints
I've split out the "file/upload" endpoint into its own module, and
renamed it to "files" to be consistent with the existing naming scheme.
2023-12-04 12:40:53 -05:00
valadaptive 5f1bed1e70 Enable object-curly-spacing lint 2023-12-04 12:32:41 -05:00
Cohee fcbfdc70af Merge branch 'staging' into sd-next-upscalers 2023-12-04 19:07:34 +02:00
Cohee 8476ec7570 Merge branch 'staging' into tokenizer-renames 2023-12-04 19:01:38 +02:00
Cohee 4bbb36a601 Merge branch 'staging' into viewsecrets-rename 2023-12-04 18:49:01 +02:00
Cohee 8bf18891b3 Add data model field for WI entry group 2023-12-04 18:47:35 +02:00
valadaptive 9d4990d25d Rename sd-next/upscalers to sd/next/upscalers 2023-12-04 10:22:47 -05:00
valadaptive 57bc95133e Rename tokenizer routes
They're all under tokenizers/ now, and there are "count", "encode", and
"decode" endpoints. This forms a clearer hierarchy.
2023-12-04 10:17:43 -05:00
valadaptive 2180610363 Rename /viewsecrets to /api/secrets/view 2023-12-04 10:14:51 -05:00
LenAnderson 76bfad8cdc fix: listeners calling wrong index after reorder 2023-12-04 13:33:28 +00:00
LenAnderson 8283d4dc08 fix: ctx button id after reordering 2023-12-04 13:28:58 +00:00
Cohee 604e62298c Concise welcome 2023-12-04 11:59:49 +02:00
Cohee 786e131c1e Disable scroll adjustment for firefox 2023-12-04 11:00:22 +02:00
Cohee 671df1f62e Fix constant usage 2023-12-04 00:24:23 +02:00
Cohee 7a58a095a4 Merge branch 'staging' into completion-source-refactor 2023-12-04 00:22:02 +02:00
Cohee 0a092629ce Fix undefined variable reference 2023-12-03 23:24:39 +02:00
valadaptive 24ccef3aba Fix "settings" variable being shadowed 2023-12-03 15:44:36 -05:00
valadaptive e33c8bd955 Replace use_[source] with chat_completion_source
Same as the is_[api] replacement--it's easier to have one enum field
than several mutually-exclusive boolean ones
2023-12-03 15:03:39 -05:00
valadaptive 9c33ddbafc Make textgen settings type checks more concise 2023-12-03 14:56:01 -05:00
valadaptive 047c897ead Remove is[API] functions
Just use an equality comparison. It's a bit longer, but only because
"textgenerationwebui_settings" is a long identifier.
2023-12-03 14:56:01 -05:00
valadaptive ba54e3dea0 Replaces is_[api] params with api_type param
These were 5 mutually-exclusive booleans, which can be replaced with one
param that takes on 5 values, one for each API type.
2023-12-03 14:56:01 -05:00
Cohee 8a1ead531c
Merge pull request #1439 from valadaptive/prompt-manager-class
Convert PromptManagerModule to a class
2023-12-03 21:52:27 +02:00