kingbri
6b656bf380
Expressions: Classify using LLM
...
Rather than using a separate BERT model to classify the last message,
use the LLM itself to get the classified expression label as a JSON
and set that as the current sprite. Doing this should take more information
into consideration and cut down on extra processing.
This is made possible by the use of constrained generation with JSON
schemas. Only available to TabbyAPI since it's the only backend that
supports the use of JSON schemas, but there can hopefully be a way
to use this with other backends as well.
Intercepts the generation and sets top_k = 1 (for greedy sampling)
and the json_schema to an emotion enum. Doing this also prevents
reingestion of the entire context every time a message is sent and
then asked to be classified, which doesn't compromise the chat
experience.
Signed-off-by: kingbri <bdashore3@proton.me>
2024-04-12 01:55:16 -04:00
Cohee
356f1f9c39
Clean-up non-character state checks
2024-04-11 23:50:01 +03:00
Cohee
123131074c
Change built-in extensions template render to async
2024-04-11 23:38:44 +03:00
Cohee
6290dff3d9
#2065 Add option to translate text before classification
2024-04-11 22:09:05 +03:00
Wolfsblvt
e26c7827bc
Fallback expression reset if custom expression deleted
2024-04-07 06:26:48 +02:00
Wolfsblvt
2ffb44b4e1
Add extension setting for fallback expression
2024-04-07 06:12:52 +02:00
Cohee
cdbd5c6130
/classify help text clarity
2024-04-06 00:45:38 +03:00
Cohee
e75f5550e3
Add /classify command
2024-04-05 22:33:16 +03:00
Cohee
c3129da879
[skip ci] Lint fix
2024-02-07 19:28:34 +02:00
Cohee
b158a86c25
Firefox copium for expression images
2024-02-05 02:21:20 +02:00
Juha Jeronen
ad48d6666a
fix bug: when switching talkinghead off, set character expression
2024-02-05 00:47:14 +02:00
Juha Jeronen
2a39db799a
auto-pause Talkinghead when ST tab is hidden to save GPU resources
2024-02-05 00:46:44 +02:00
Juha Jeronen
91c4de6605
add /th (alias /talkinghead) to toggle Talkinghead on/off
2024-02-05 00:46:23 +02:00
Juha Jeronen
5ad2a0d064
refresh talkinghead char on expression zip upload
2024-02-05 00:45:50 +02:00
Juha Jeronen
3b526ce207
remove some useless comments
2024-02-05 00:45:37 +02:00
Juha Jeronen
169b1c2c63
talkinghead check: always check also whether the module is enabled
2024-02-05 00:45:20 +02:00
Juha Jeronen
9e8f3e0def
one more debug message
2024-02-05 00:44:46 +02:00
Juha Jeronen
eb634d597f
add comment on TTS
2024-02-05 00:44:39 +02:00
Juha Jeronen
24b315a149
comments
...
The tech is "Talkinghead" (capital T), the Extras module is
"talkinghead" (lowercase t).
2024-02-05 00:44:26 +02:00
Juha Jeronen
83e264db9e
add some debug messages
2024-02-05 00:40:03 +02:00
Cohee
2846d0fd58
#1720 Fetch no-cache images when uploading
2024-01-20 19:48:56 +02:00
Juha Jeronen
b528520693
read and send talkinghead per-character configs
2024-01-12 18:26:14 +02:00
Cohee
810667e8e7
Slight increase processing delay
2024-01-09 20:17:17 +02:00
Cohee
63ab16161f
Add check for enabled module
2024-01-09 19:52:49 +02:00
Juha Jeronen
87b05e2e2d
enable talkinghead's talking animation while the LLM is streaming
2024-01-08 00:47:59 +02:00
Juha Jeronen
39c485ab25
some missed refactorings
2024-01-08 00:47:40 +02:00
Juha Jeronen
1184ea2c3b
clarity
2024-01-08 00:47:18 +02:00
Juha Jeronen
3f3e68c492
document some functions
2024-01-08 00:46:54 +02:00
Cohee
ad94ba57b1
Move clear function names
2024-01-07 21:14:29 +02:00
Juha Jeronen
5bc08ab8c1
check also that expressions is running with extras
2024-01-06 02:48:39 +02:00
Juha Jeronen
e6ade18b58
ehm, once more, correctly.
2024-01-06 02:23:10 +02:00
Juha Jeronen
d1e9c59e72
meh, don't check for static sprite, correctly
2024-01-06 02:17:07 +02:00
Juha Jeronen
2d85d7e1e1
talkinghead mode: don't check for existence of static emote sprite
2024-01-06 02:10:09 +02:00
Juha Jeronen
a7c6f4f56e
refresh live char when a new talkinghead image is uploaded
2024-01-06 02:09:34 +02:00
Juha Jeronen
7f4a45a4a3
add comment
2024-01-06 02:09:27 +02:00
Cohee
6c03e81d8d
chore: format fix
2024-01-05 15:45:05 +02:00
Juha Jeronen
77c11b8f2b
set_emotion is not supported by old talkinghead versions
2024-01-05 12:15:24 +02:00
Juha Jeronen
059f61fee8
add "/emote xxx" support for talkinghead (when VN mode off)
2024-01-02 02:18:54 +02:00
Cohee
30732ada32
Lint fix
2024-01-01 16:08:24 +02:00
LenAnderson
a2aa8ba6a0
add export and slash command for last set expressions
2023-12-30 11:37:13 +00:00
Cohee
e1afe41c91
Fix custom expression duplication
2023-12-21 16:50:30 +02:00
Cohee
64a3564892
lint: Comma dangle
2023-12-02 22:06:57 +02:00
Cohee
c63cd87cc0
lint: Require semicolons
2023-12-02 21:11:06 +02:00
valadaptive
a37f874e38
Require single quotes
2023-12-02 13:04:51 -05:00
valadaptive
518bb58d5a
Enable no-unused-vars lint
...
This is the big one. Probably needs thorough review to make sure I
didn't accidentally remove any setInterval or fetch calls.
2023-12-02 12:11:19 -05:00
valadaptive
b023312117
Enable no-useless-escape lint
2023-12-02 10:32:26 -05:00
Cohee
5dd6665505
Don't set negative zIndex for regular image when using group VN mode
2023-11-28 11:40:39 +02:00
Cohee
c742251c5a
Mobile sprites fixes: hide non-VN sprite, fix group VN position, fix live2d conflicts
2023-11-27 03:22:35 +02:00
RossAscends
fe21a7c25b
Add toggle for hiding muted group member sprites
2023-11-22 07:48:35 +09:00
Cohee
fadd92a6c3
Fix expressions sometimes getting stuck when switching chats
2023-10-29 23:15:40 +02:00
