STscript Parser Rewrite (#1965)

* set isForced to true on input

* make floating auto-complete follow horizontal scrolling

* add callable closure vars

* changes to /let and /var for callable closures

* fix error message

* fix scope for closure arguments

* if should return the pipe result from closures

* use /run to call closures and no arguments on immediate closures

* throw exception from QRs window-function if no match

* when to show autocomplete vs info only

* autocomplete positioning

* autocomplete styling

* add theming to autocomplete (theme, dark, light)

* improve autocomplete show/hide logic and editor selection

* use blur tint color instead of chat tint color and use blur setting

* cleanup and docs

* use scope macros for QR args

* add enter to select autocomplete

* fix no executor found

* cleanup and comment

* fix alias list in help string

* fallback to empty string piped value if null or undefined

* fix typo

* blur textarea on ctrl+enter execute (and refocus after)

* stop executeSlashCommand if parser throws

* move /let and /var callbacks into functions

* switch textarea to monospace when value starts with slash

* add double pipe a pipe breaker

* fix /? slash

* remove some logging

* add "/:name" as shorthand for "/run name" after all

* move shit around

* fix error message

* use testRunShorthandEnd

* use parseQuotedValue and parseValue to determine name for "/:"

QR labels and set names can include spaces

* add some adjustments to make autocomplete work properly

some hint in there about "/:" would still be nice

* add autocomplete style  selector

* only strip quotes from subcommand if they are at both ends

* fix JSDoc

* escaping

* allow open quotes on dry run

* throwing shit at the wall for /: autocomplete

* escapes only for symbols

* clean up autocomplete

* improve performance

* fix scope macros

* remove unescaping of pipes

* fix macros in scope copy

* fix "/? slash"

* don't run parser for getNameAt if text has not changed

* fix options filter

* re-enable blur listener

* restore selection on non-replace select

* fix for escaping first character of value

* add support for {{pipe}} and {{var::}} closures

* add index support to var macro

* add scoped var macro to macro help

* more escape fixes

* reduce autocomplete render debounce

* cleanup

* restore old escape handling and parser flag for strict escaping

* fix "no match" autocomplete message

* add dummy commands for comments and parser flag

* fix type annotations

* somewhat safer macro replacements

* fix autocomplete select on blank / "no match"

* fix cutting off handled part in substitution

* add parser flag REPLACE_GETVAR

Replaces all {{getvar::}} and {{getglobalvar::}} macros with {{var::}}.
Inserts a series of command executors before the command with the macros that:
- save {{pipe}} to a var
- call /getvar or /getglobalvar to get the variable used in the macro
- call /let to save the retrieved variable
- return the saved {{pipe}} value

This helps to avoid double-substitutions when the var values contain text that could be interpreted as macros.

* remove old parser

* fix send on enter when no match

* deal with pipes in quoted values (loose escaping)

* add default parser flags to user settings

* allow quoted values in unnamed argument

* set parser flag without explicit state to "on"

* add click hint on parser error toast

* dirty more detailed cmd defs

* remove name from unnamed arg

* move autocomplete into class and floating with details

* replace jQuery's trigger('input') on #send_textarea with native events because jQuery does not dispatch the native event

* fix ctrl+space

* fix arrow navigation

* add comments

* fix pointer block

* add static fromProps

* fix up dummy commands

* migrate all commands to addCommandObject

* remove commented comment command

* fix alias in details

* add range as argument type

* switch to addCommandObject

* switch to addCommandObject

* fix height

* fix floating details position on left

* re-enable blur event

* use auto width for full details on floating autocomplete

* auto-size floating full details

* fix typo

* re-enable blur listener

* don't prevent enter when selected item is fully typed out

* add autocomplete details tooltips

* add language to slash command examples

* move makeItem into option and command and fix click select

* use autocomplete parts in /? slash

* fix alias formatting

* add language to slash command examples

* fix details position on initial input history

* small screen styles

* replace registerSlashCommand with detailed declarations

* put name on first line

* add missing returns

* fix missing comma

* fix alias display in autocomplete list

* remove args from help string

* move parser settings to its own section

* jsdoc

* hljs stscript lang

* add hljs to autocomplete help examples

* add missing import

* apply autocomplete colors to stscript codeblocks (hljs)

* add fromProps

* cache autocomplete elements

* towards generic autocomplete

* remove unused imports

* fix blanks

* add return types

* re-enable blur

* fix blank check

* Caption messages by id

* add aborting command execution

* fix return type

* fix chat input font reset

* add slash command progress indicator

* add missing return

* mark registerSlashCommand deprecated

* why??

* separate abort logic for commands

* remove parsing of quoted values from unnamed arg

* add adjustable autocomplete width

* revert stop button pulse

* add progress and pause/abort to QR editor

* add resize event on autocomplete width change

* add key= argument to all get vars

* refactoring

* introduce NamedArgumentAsignment

* add TODOs

* refactoring

* record start and end of named arg assignment

* refactoring

* prevent duplicate calls to show

* refactoring

* remove macro ac

* add secondary autocomplete and enum descriptions

* add syntax highlighting to QR editor

* add enum descriptions to /while

* add /let key=... to scope variable names

* add unnamed argument assignment class and unnamed argument splitting

* fix QR editor style

* remove dash before autocomplete help text

* add autocomplete for unnamed enums

* fix remaining dom after holding backslash

* fix for unnamed enums

* fix autocomplete for /parser-flag

* add parser-flag enum help

* fix type annotations

* fix autocomplete result for /:

* add colored autocomplete type icons

* collapse second line autocomplete help if empty

* mark optional named args in autocomplete

* fix when what

* remove duplicate debug buttons

* dispatch input on autocomplete select

* prevent grow from editor syntax layer

* add auto-adjust qr editor caret color

* remove text-shadow from autocomplete

* join value strings in /let and /var

* add /abort syntax highlight

* fix attempting secondary result when there is none

* rename settings headers and split autocomplete / stscript

* add parser flag tooltips

* add tooltips to chat width stops

* fix typo

* return clone of help item

* fix enum string

* don't make optional notice for autocomplete arguments smaller

* avoid scrollbar in chat input

* add rudimentary macro autocomplete

* strip macro from helptext

* finally remove closure delimiters around root

* cleanup

* fix index stuff for removed closure delimiters

* fix type hint

* add child commands to progress indicator

* include sub-separator in macro autocomplete

* remove all mentions of interruptsGeneration and purge

* remove unused imports

* fix syntax highlight with newline at end of input

* cleanup select pointer events

* coalesce onProgress call

* add regex to STscript syntax highlighting

* fix closure end

* fix autocomplete type icon alignment

* adjustments for small screens

* fix removing wrong element

* add missing "at=" arg to /sys, /comment, /sendas

* add font scale setting for autocomplete

* add target=_blank for parser flag links

* fix for searching enums

* remove REGEXP_MODE from hljs
just causes trouble

* fix autocomplete in closures

* fix typo

* fix type hint

* Get rid of scroll bar on load

* Add type hint for /send name argument. Fix 'at' types

* Add 'negative' arg hint to /sd command

* reenable blur event

* Allow /summarize to process any text

* Compact layout of script toggles

* Expand CSS by default

* fix double ranger indicator and adjust to narrow container

* make custom css input fill available vertical space

* reduce scroll lag

* use default cursor on scrollbar

* Clean-up module loading in index.html

* fix tab indent with hljs

---------

Co-authored-by: Cohee <18619528+Cohee1207@users.noreply.github.com>
This commit is contained in:
Len
2024-05-12 15:15:05 -04:00
committed by GitHub
parent c7d75b7789
commit 1d75b98393
57 changed files with 8777 additions and 736 deletions

View File

@ -35,7 +35,6 @@ import {
selectInstructPreset,
} from './instruct-mode.js';
import { registerSlashCommand } from './slash-commands.js';
import { getTagsList, tag_map, tags } from './tags.js';
import { tokenizers } from './tokenizers.js';
import { BIAS_CACHE } from './logit-bias.js';
@ -43,6 +42,10 @@ import { renderTemplateAsync } from './templates.js';
import { countOccurrences, debounce, delay, download, getFileText, isOdd, onlyUnique, resetScrollHeight, shuffle, sortMoments, stringToRange, timestampToMoment } from './utils.js';
import { FILTER_TYPES } from './filters.js';
import { PARSER_FLAG, SlashCommandParser } from './slash-commands/SlashCommandParser.js';
import { SlashCommand } from './slash-commands/SlashCommand.js';
import { ARGUMENT_TYPE, SlashCommandArgument } from './slash-commands/SlashCommandArgument.js';
import { AUTOCOMPLETE_WIDTH } from './autocomplete/AutoComplete.js';
export {
loadPowerUserSettings,
@ -253,6 +256,23 @@ let power_user = {
zoomed_avatar_magnification: false,
show_tag_filters: false,
aux_field: 'character_version',
stscript: {
matching: 'fuzzy',
autocomplete: {
style: 'theme',
font: {
scale: 0.8,
},
width: {
left: AUTOCOMPLETE_WIDTH.CHAT,
right: AUTOCOMPLETE_WIDTH.CHAT,
},
},
parser: {
/**@type {Object.<PARSER_FLAG,boolean>} */
flags: {},
},
},
restore_user_input: true,
reduced_motion: false,
compact_input_area: true,
@ -1430,11 +1450,32 @@ function getExampleMessagesBehavior() {
}
function loadPowerUserSettings(settings, data) {
const defaultStscript = JSON.parse(JSON.stringify(power_user.stscript));
// Load from settings.json
if (settings.power_user !== undefined) {
Object.assign(power_user, settings.power_user);
}
if (power_user.stscript === undefined) {
power_user.stscript = defaultStscript;
} else {
if (power_user.stscript.autocomplete === undefined) {
power_user.stscript.autocomplete = defaultStscript.autocomplete;
} else {
if (power_user.stscript.autocomplete.width === undefined) {
power_user.stscript.autocomplete.width = defaultStscript.autocomplete.width;
}
if (power_user.stscript.autocomplete.font === undefined) {
power_user.stscript.autocomplete.font = defaultStscript.autocomplete.font;
}
}
if (power_user.stscript.parser === undefined) {
power_user.stscript.parser = defaultStscript.parser;
} else if (power_user.stscript.parser.flags === undefined) {
power_user.stscript.parser.flags = defaultStscript.parser.flags;
}
}
if (data.themes !== undefined) {
themes = data.themes;
}
@ -1576,6 +1617,20 @@ function loadPowerUserSettings(settings, data) {
$('#chat_width_slider').val(power_user.chat_width);
$('#token_padding').val(power_user.token_padding);
$('#aux_field').val(power_user.aux_field);
$('#stscript_matching').val(power_user.stscript.matching ?? 'fuzzy');
$('#stscript_autocomplete_style').val(power_user.stscript.autocomplete_style ?? 'theme');
document.body.setAttribute('data-stscript-style', power_user.stscript.autocomplete_style);
$('#stscript_parser_flag_strict_escaping').prop('checked', power_user.stscript.parser.flags[PARSER_FLAG.STRICT_ESCAPING] ?? false);
$('#stscript_parser_flag_replace_getvar').prop('checked', power_user.stscript.parser.flags[PARSER_FLAG.REPLACE_GETVAR] ?? false);
$('#stscript_autocomplete_font_scale').val(power_user.stscript.autocomplete.font.scale ?? defaultStscript.autocomplete.font.scale);
$('#stscript_autocomplete_font_scale_counter').val(power_user.stscript.autocomplete.font.scale ?? defaultStscript.autocomplete.font.scale);
document.body.style.setProperty('--ac-font-scale', power_user.stscript.autocomplete.font.scale ?? defaultStscript.autocomplete.font.scale.toString());
$('#stscript_autocomplete_width_left').val(power_user.stscript.autocomplete.width.left ?? AUTOCOMPLETE_WIDTH.CHAT);
document.querySelector('#stscript_autocomplete_width_left').dispatchEvent(new Event('input', { bubbles:true }));
$('#stscript_autocomplete_width_right').val(power_user.stscript.autocomplete.width.right ?? AUTOCOMPLETE_WIDTH.CHAT);
document.querySelector('#stscript_autocomplete_width_right').dispatchEvent(new Event('input', { bubbles:true }));
$('#restore_user_input').prop('checked', power_user.restore_user_input);
$('#chat_truncation').val(power_user.chat_truncation);
@ -3591,6 +3646,64 @@ $(document).ready(() => {
saveSettingsDebounced();
});
$('#stscript_matching').on('change', function () {
const value = $(this).find(':selected').val();
power_user.stscript.matching = String(value);
saveSettingsDebounced();
});
$('#stscript_autocomplete_style').on('change', function () {
const value = $(this).find(':selected').val();
power_user.stscript.autocomplete_style = String(value);
document.body.setAttribute('data-stscript-style', power_user.stscript.autocomplete_style);
saveSettingsDebounced();
});
$('#stscript_autocomplete_font_scale').on('input', function () {
const value = $(this).val();
$('#stscript_autocomplete_font_scale_counter').val(value);
power_user.stscript.autocomplete.font.scale = Number(value);
document.body.style.setProperty('--ac-font-scale', value.toString());
window.dispatchEvent(new Event('resize', { bubbles:true }));
saveSettingsDebounced();
});
$('#stscript_autocomplete_font_scale_counter').on('input', function () {
const value = $(this).val();
$('#stscript_autocomplete_font_scale').val(value);
power_user.stscript.autocomplete.font.scale = Number(value);
document.body.style.setProperty('--ac-font-scale', value.toString());
window.dispatchEvent(new Event('resize', { bubbles:true }));
saveSettingsDebounced();
});
$('#stscript_autocomplete_width_left').on('input', function () {
const value = $(this).val();
power_user.stscript.autocomplete.width.left = Number(value);
/**@type {HTMLElement}*/(this.closest('.doubleRangeInputContainer')).style.setProperty('--value', value.toString());
window.dispatchEvent(new Event('resize', { bubbles:true }));
saveSettingsDebounced();
});
$('#stscript_autocomplete_width_right').on('input', function () {
const value = $(this).val();
power_user.stscript.autocomplete.width.right = Number(value);
/**@type {HTMLElement}*/(this.closest('.doubleRangeInputContainer')).style.setProperty('--value', value.toString());
window.dispatchEvent(new Event('resize', { bubbles:true }));
saveSettingsDebounced();
});
$('#stscript_parser_flag_strict_escaping').on('click', function () {
const value = $(this).prop('checked');
power_user.stscript.parser.flags[PARSER_FLAG.STRICT_ESCAPING] = value;
saveSettingsDebounced();
});
$('#stscript_parser_flag_replace_getvar').on('click', function () {
const value = $(this).prop('checked');
power_user.stscript.parser.flags[PARSER_FLAG.REPLACE_GETVAR] = value;
saveSettingsDebounced();
});
$('#restore_user_input').on('input', function () {
power_user.restore_user_input = !!$(this).prop('checked');
saveSettingsDebounced();
@ -3669,13 +3782,84 @@ $(document).ready(() => {
browser_has_focus = false;
});
registerSlashCommand('vn', toggleWaifu, [], ' swaps Visual Novel Mode On/Off', false, true);
registerSlashCommand('newchat', doNewChat, [], ' start a new chat with current character', true, true);
registerSlashCommand('random', doRandomChat, [], '<span class="monospace">(optional tag name)</span> start a new chat with a random character. If an argument is provided, only considers characters that have the specified tag.', true, true);
registerSlashCommand('delmode', doDelMode, ['del'], '<span class="monospace">(optional number)</span> enter message deletion mode, and auto-deletes last N messages if numeric argument is provided', true, true);
registerSlashCommand('cut', doMesCut, [], '<span class="monospace">(number or range)</span> cuts the specified message or continuous chunk from the chat, e.g. <tt>/cut 0-10</tt>. Ranges are inclusive! Returns the text of cut messages separated by a newline.', true, true);
registerSlashCommand('resetpanels', doResetPanels, ['resetui'], ' resets UI panels to original state.', true, true);
registerSlashCommand('bgcol', setAvgBG, [], ' WIP test of auto-bg avg coloring', true, true);
registerSlashCommand('theme', setThemeCallback, [], '<span class="monospace">(name)</span> sets a UI theme by name', true, true);
registerSlashCommand('movingui', setmovingUIPreset, [], '<span class="monospace">(name)</span> activates a movingUI preset by name', true, true);
SlashCommandParser.addCommandObject(SlashCommand.fromProps({ name: 'vn',
callback: toggleWaifu,
helpString: 'Swaps Visual Novel Mode On/Off',
}));
SlashCommandParser.addCommandObject(SlashCommand.fromProps({ name: 'newchat',
callback: doNewChat,
helpString: 'Start a new chat with the current character',
}));
SlashCommandParser.addCommandObject(SlashCommand.fromProps({ name: 'random',
callback: doRandomChat,
unnamedArgumentList: [
new SlashCommandArgument(
'optional tag name', [ARGUMENT_TYPE.STRING], false,
),
],
helpString: 'Start a new chat with a random character. If an argument is provided, only considers characters that have the specified tag.',
}));
SlashCommandParser.addCommandObject(SlashCommand.fromProps({ name: 'delmode',
callback: doDelMode,
aliases: ['del'],
unnamedArgumentList: [
new SlashCommandArgument(
'optional number', [ARGUMENT_TYPE.NUMBER], false,
),
],
helpString: 'Enter message deletion mode, and auto-deletes last N messages if numeric argument is provided.',
}));
SlashCommandParser.addCommandObject(SlashCommand.fromProps({ name: 'cut',
callback: doMesCut,
returns: 'the text of cut messages separated by a newline',
unnamedArgumentList: [
new SlashCommandArgument(
'number or range', [ARGUMENT_TYPE.NUMBER, ARGUMENT_TYPE.RANGE], true,
),
],
helpString: `
<div>
Cuts the specified message or continuous chunk from the chat.
</div>
<div>
Ranges are inclusive!
</div>
<div>
<strong>Example:</strong>
<ul>
<li>
<pre><code>/cut 0-10</code></pre>
</li>
</ul>
</div>
`,
aliases: [],
}));
SlashCommandParser.addCommandObject(SlashCommand.fromProps({ name: 'resetpanels',
callback: doResetPanels,
helpString: 'resets UI panels to original state',
aliases: ['resetui'],
}));
SlashCommandParser.addCommandObject(SlashCommand.fromProps({ name: 'bgcol',
callback: setAvgBG,
helpString: ' WIP test of auto-bg avg coloring',
}));
SlashCommandParser.addCommandObject(SlashCommand.fromProps({ name: 'theme',
callback: setThemeCallback,
unnamedArgumentList: [
new SlashCommandArgument(
'name', [ARGUMENT_TYPE.STRING], true,
),
],
helpString: 'sets a UI theme by name',
}));
SlashCommandParser.addCommandObject(SlashCommand.fromProps({ name: 'movingui',
callback: setmovingUIPreset,
unnamedArgumentList: [
new SlashCommandArgument(
'name', [ARGUMENT_TYPE.STRING], true,
),
],
helpString: 'activates a movingUI preset by name',
}));
});