STscript Parser Rewrite (#1965)

* set isForced to true on input * make floating auto-complete follow horizontal scrolling * add callable closure vars * changes to /let and /var for callable closures * fix error message * fix scope for closure arguments * if should return the pipe result from closures * use /run to call closures and no arguments on immediate closures * throw exception from QRs window-function if no match * when to show autocomplete vs info only * autocomplete positioning * autocomplete styling * add theming to autocomplete (theme, dark, light) * improve autocomplete show/hide logic and editor selection * use blur tint color instead of chat tint color and use blur setting * cleanup and docs * use scope macros for QR args * add enter to select autocomplete * fix no executor found * cleanup and comment * fix alias list in help string * fallback to empty string piped value if null or undefined * fix typo * blur textarea on ctrl+enter execute (and refocus after) * stop executeSlashCommand if parser throws * move /let and /var callbacks into functions * switch textarea to monospace when value starts with slash * add double pipe a pipe breaker * fix /? slash * remove some logging * add "/:name" as shorthand for "/run name" after all * move shit around * fix error message * use testRunShorthandEnd * use parseQuotedValue and parseValue to determine name for "/:" QR labels and set names can include spaces * add some adjustments to make autocomplete work properly some hint in there about "/:" would still be nice * add autocomplete style selector * only strip quotes from subcommand if they are at both ends * fix JSDoc * escaping * allow open quotes on dry run * throwing shit at the wall for /: autocomplete * escapes only for symbols * clean up autocomplete * improve performance * fix scope macros * remove unescaping of pipes * fix macros in scope copy * fix "/? slash" * don't run parser for getNameAt if text has not changed * fix options filter * re-enable blur listener * restore selection on non-replace select * fix for escaping first character of value * add support for {{pipe}} and {{var::}} closures * add index support to var macro * add scoped var macro to macro help * more escape fixes * reduce autocomplete render debounce * cleanup * restore old escape handling and parser flag for strict escaping * fix "no match" autocomplete message * add dummy commands for comments and parser flag * fix type annotations * somewhat safer macro replacements * fix autocomplete select on blank / "no match" * fix cutting off handled part in substitution * add parser flag REPLACE_GETVAR Replaces all {{getvar::}} and {{getglobalvar::}} macros with {{var::}}. Inserts a series of command executors before the command with the macros that: - save {{pipe}} to a var - call /getvar or /getglobalvar to get the variable used in the macro - call /let to save the retrieved variable - return the saved {{pipe}} value This helps to avoid double-substitutions when the var values contain text that could be interpreted as macros. * remove old parser * fix send on enter when no match * deal with pipes in quoted values (loose escaping) * add default parser flags to user settings * allow quoted values in unnamed argument * set parser flag without explicit state to "on" * add click hint on parser error toast * dirty more detailed cmd defs * remove name from unnamed arg * move autocomplete into class and floating with details * replace jQuery's trigger('input') on #send_textarea with native events because jQuery does not dispatch the native event * fix ctrl+space * fix arrow navigation * add comments * fix pointer block * add static fromProps * fix up dummy commands * migrate all commands to addCommandObject * remove commented comment command * fix alias in details * add range as argument type * switch to addCommandObject * switch to addCommandObject * fix height * fix floating details position on left * re-enable blur event * use auto width for full details on floating autocomplete * auto-size floating full details * fix typo * re-enable blur listener * don't prevent enter when selected item is fully typed out * add autocomplete details tooltips * add language to slash command examples * move makeItem into option and command and fix click select * use autocomplete parts in /? slash * fix alias formatting * add language to slash command examples * fix details position on initial input history * small screen styles * replace registerSlashCommand with detailed declarations * put name on first line * add missing returns * fix missing comma * fix alias display in autocomplete list * remove args from help string * move parser settings to its own section * jsdoc * hljs stscript lang * add hljs to autocomplete help examples * add missing import * apply autocomplete colors to stscript codeblocks (hljs) * add fromProps * cache autocomplete elements * towards generic autocomplete * remove unused imports * fix blanks * add return types * re-enable blur * fix blank check * Caption messages by id * add aborting command execution * fix return type * fix chat input font reset * add slash command progress indicator * add missing return * mark registerSlashCommand deprecated * why?? * separate abort logic for commands * remove parsing of quoted values from unnamed arg * add adjustable autocomplete width * revert stop button pulse * add progress and pause/abort to QR editor * add resize event on autocomplete width change * add key= argument to all get vars * refactoring * introduce NamedArgumentAsignment * add TODOs * refactoring * record start and end of named arg assignment * refactoring * prevent duplicate calls to show * refactoring * remove macro ac * add secondary autocomplete and enum descriptions * add syntax highlighting to QR editor * add enum descriptions to /while * add /let key=... to scope variable names * add unnamed argument assignment class and unnamed argument splitting * fix QR editor style * remove dash before autocomplete help text * add autocomplete for unnamed enums * fix remaining dom after holding backslash * fix for unnamed enums * fix autocomplete for /parser-flag * add parser-flag enum help * fix type annotations * fix autocomplete result for /: * add colored autocomplete type icons * collapse second line autocomplete help if empty * mark optional named args in autocomplete * fix when what * remove duplicate debug buttons * dispatch input on autocomplete select * prevent grow from editor syntax layer * add auto-adjust qr editor caret color * remove text-shadow from autocomplete * join value strings in /let and /var * add /abort syntax highlight * fix attempting secondary result when there is none * rename settings headers and split autocomplete / stscript * add parser flag tooltips * add tooltips to chat width stops * fix typo * return clone of help item * fix enum string * don't make optional notice for autocomplete arguments smaller * avoid scrollbar in chat input * add rudimentary macro autocomplete * strip macro from helptext * finally remove closure delimiters around root * cleanup * fix index stuff for removed closure delimiters * fix type hint * add child commands to progress indicator * include sub-separator in macro autocomplete * remove all mentions of interruptsGeneration and purge * remove unused imports * fix syntax highlight with newline at end of input * cleanup select pointer events * coalesce onProgress call * add regex to STscript syntax highlighting * fix closure end * fix autocomplete type icon alignment * adjustments for small screens * fix removing wrong element * add missing "at=" arg to /sys, /comment, /sendas * add font scale setting for autocomplete * add target=_blank for parser flag links * fix for searching enums * remove REGEXP_MODE from hljs just causes trouble * fix autocomplete in closures * fix typo * fix type hint * Get rid of scroll bar on load * Add type hint for /send name argument. Fix 'at' types * Add 'negative' arg hint to /sd command * reenable blur event * Allow /summarize to process any text * Compact layout of script toggles * Expand CSS by default * fix double ranger indicator and adjust to narrow container * make custom css input fill available vertical space * reduce scroll lag * use default cursor on scrollbar * Clean-up module loading in index.html * fix tab indent with hljs --------- Co-authored-by: Cohee <18619528+Cohee1207@users.noreply.github.com>
2025-06-05 21:59:27 +02:00 · 2024-05-12 15:15:05 -04:00
parent c7d75b7789
commit 1d75b98393
57 changed files with 8777 additions and 736 deletions
--- a/public/scripts/extensions/memory/index.js
+++ b/public/scripts/extensions/memory/index.js
@@ -16,11 +16,14 @@ import {
    getMaxContextSize,
 } from '../../../script.js';
 import { is_group_generating, selected_group } from '../../group-chats.js';
-import { registerSlashCommand } from '../../slash-commands.js';
 import { loadMovingUIState } from '../../power-user.js';
 import { dragElement } from '../../RossAscends-mods.js';
 import { getTextTokens, getTokenCountAsync, tokenizers } from '../../tokenizers.js';
 import { debounce_timeout } from '../../constants.js';
+import { SlashCommandParser } from '../../slash-commands/SlashCommandParser.js';
+import { SlashCommand } from '../../slash-commands/SlashCommand.js';
+import { ARGUMENT_TYPE, SlashCommandArgument, SlashCommandNamedArgument } from '../../slash-commands/SlashCommandArgument.js';
+import { resolveVariable } from '../../variables.js';
 export { MODULE_NAME };

 const MODULE_NAME = '1_memory';
@@ -416,7 +419,7 @@ async function forceSummarizeChat() {
    console.log(`Skipping WIAN? ${skipWIAN}`);
    if (!context.chatId) {
        toastr.warning('No chat selected');
-        return;
+        return '';
    }

    toastr.info('Summarizing chat...', 'Please wait');
@@ -424,7 +427,42 @@ async function forceSummarizeChat() {

    if (!value) {
        toastr.warning('Failed to summarize chat');
-        return;
+        return '';
+    }
+
+    return value;
+}
+
+/**
+ * Callback for the summarize command.
+ * @param {object} args Command arguments
+ * @param {string} text Text to summarize
+ */
+async function summarizeCallback(args, text) {
+    text = text.trim();
+
+    // Using forceSummarizeChat to summarize the current chat
+    if (!text) {
+        return await forceSummarizeChat();
+    }
+
+    const source = args.source || extension_settings.memory.source;
+    const prompt = substituteParams((resolveVariable(args.prompt) || extension_settings.memory.prompt)?.replace(/{{words}}/gi, extension_settings.memory.promptWords));
+
+    try {
+        switch (source) {
+            case summary_sources.extras:
+                return await callExtrasSummarizeAPI(text);
+            case summary_sources.main:
+                return await generateRaw(text, '', false, false, prompt, extension_settings.memory.overrideResponseLength);
+            default:
+                toastr.warning('Invalid summarization source specified');
+                return '';
+        }
+    } catch (error) {
+        toastr.error(String(error), 'Failed to summarize text');
+        console.log(error);
+        return '';
    }
 }

@@ -668,37 +706,18 @@ async function summarizeChatExtras(context) {
    // perform the summarization API call
    try {
        inApiCall = true;
-        const url = new URL(getApiUrl());
-        url.pathname = '/api/summarize';
+        const summary = await callExtrasSummarizeAPI(resultingString);
+        const newContext = getContext();

-        const apiResult = await doExtrasFetch(url, {
-            method: 'POST',
-            headers: {
-                'Content-Type': 'application/json',
-                'Bypass-Tunnel-Reminder': 'bypass',
-            },
-            body: JSON.stringify({
-                text: resultingString,
-                params: {},
-            }),
-        });
-
-        if (apiResult.ok) {
-            const data = await apiResult.json();
-            const summary = data.summary;
-
-            const newContext = getContext();
-
-            // something changed during summarization request
-            if (newContext.groupId !== context.groupId
-                || newContext.chatId !== context.chatId
-                || (!newContext.groupId && (newContext.characterId !== context.characterId))) {
-                console.log('Context changed, summary discarded');
-                return;
-            }
-
-            setMemoryContext(summary, true);
+        // something changed during summarization request
+        if (newContext.groupId !== context.groupId
+            || newContext.chatId !== context.chatId
+            || (!newContext.groupId && (newContext.characterId !== context.characterId))) {
+            console.log('Context changed, summary discarded');
+            return;
        }
+
+        setMemoryContext(summary, true);
    }
    catch (error) {
        console.log(error);
@@ -708,6 +727,40 @@ async function summarizeChatExtras(context) {
    }
 }

+/**
+ * Call the Extras API to summarize the provided text.
+ * @param {string} text Text to summarize
+ * @returns {Promise<string>} Summarized text
+ */
+async function callExtrasSummarizeAPI(text) {
+    if (!modules.includes('summarize')) {
+        throw new Error('Summarize module is not enabled in Extras API');
+    }
+
+    const url = new URL(getApiUrl());
+    url.pathname = '/api/summarize';
+
+    const apiResult = await doExtrasFetch(url, {
+        method: 'POST',
+        headers: {
+            'Content-Type': 'application/json',
+            'Bypass-Tunnel-Reminder': 'bypass',
+        },
+        body: JSON.stringify({
+            text: text,
+            params: {},
+        }),
+    });
+
+    if (apiResult.ok) {
+        const data = await apiResult.json();
+        const summary = data.summary;
+        return summary;
+    }
+
+    throw new Error('Extras API call failed');
+}
+
 function onMemoryRestoreClick() {
    const context = getContext();
    const content = $('#memory_contents').val();
@@ -865,5 +918,16 @@ jQuery(async function () {
    eventSource.on(event_types.MESSAGE_EDITED, onChatEvent);
    eventSource.on(event_types.MESSAGE_SWIPED, onChatEvent);
    eventSource.on(event_types.CHAT_CHANGED, onChatEvent);
-    registerSlashCommand('summarize', forceSummarizeChat, [], '– forces the summarization of the current chat using the Main API', true, true);
+    SlashCommandParser.addCommandObject(SlashCommand.fromProps({
+        name: 'summarize',
+        callback: summarizeCallback,
+        namedArgumentList: [
+            new SlashCommandNamedArgument('source', 'API to use for summarization', [ARGUMENT_TYPE.STRING], false, false, '', ['main', 'extras']),
+            new SlashCommandNamedArgument('prompt', 'prompt to use for summarization', [ARGUMENT_TYPE.STRING, ARGUMENT_TYPE.VARIABLE_NAME], false, false, ''),
+        ],
+        unnamedArgumentList: [
+            new SlashCommandArgument('text to summarize', [ARGUMENT_TYPE.STRING], false, false, ''),
+        ],
+        helpString: 'Summarizes the given text. If no text is provided, the current chat will be summarized. Can specify the source and the prompt to use.',
+    }));
 });