rixty/SillyTavern
1
0
Fork 0
mirror of https://github.com/SillyTavern/SillyTavern.git synced 2025-06-05 21:59:27 +02:00
Code Issues Packages Projects Releases Wiki Activity

Fix LLAMA tokenization. Add case-sensitive WI matching

Browse Source
This commit is contained in:
SillyLossy
2023-05-21 00:40:13 +03:00
parent 1253d04561
commit 285d3e3d4a
4 changed files with 68 additions and 39 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
public/index.html
View File

@ -1458,7 +1458,7 @@
</div>
</div>
<div class="flex1 range-block">
<div class="flex1 range-block flex-container flexFlowColumn">
<label title="Entries can activate other entries by mentioning their keywords" class="checkbox_label">
<input id="world_info_recursive" type="checkbox" />
<span>
@ -1468,6 +1468,15 @@
</a>
</span>
</label>
<label title="Lookup for the entry keys in the context will respect the case" class="checkbox_label">
<input id="world_info_case_sensitive" type="checkbox" />
<span>
Case-sensitive keys
<a href="/notes#casesensitivekeys" class="notes-link" target="_blank">
<span class="note-link-span">?</span>
</a>
</span>
</label>
</div>
</div>
@ -2195,7 +2204,7 @@
<div id="context_editor_template" class="template_element">
<div class="context_editor">
<h3>Context Template Editor</h3>
<div class="inline-drawer wide100p">
<div class="inline-drawer-toggle inline-drawer-header">
Substitution Parameters
@ -2632,4 +2641,4 @@
</script>
</body>
</html>
</html>

52
public/notes/content.md
View File

@ -16,7 +16,7 @@ Usually it all takes 200-350 tokens.
For most Kobold's models the easiest way is to use a free form for description, and in each sentence it is desirable to specify the name of the character.
The entire description should be in one line without hyphenation.
The entire description should be in one line without hyphenation.
For example:
@ -96,15 +96,15 @@ Another example:
### First message
The First Message is an important thing that sets exactly how and in what style the character will communicate.
The First Message is an important thing that sets exactly how and in what style the character will communicate.
It is desirable that the character's first message be long, so that later it would be less likely that the character would respond in with very short messages.
It is desirable that the character's first message be long, so that later it would be less likely that the character would respond in with very short messages.
You can also use asterisks ** to describe the character's actions.
For example:
`*I noticed you came inside, I walked up and stood right in front of you* Welcome. I'm glad to see you here. *I said with toothy smug sunny smile looking you straight in the eye* What brings you...`
`*I noticed you came inside, I walked up and stood right in front of you* Welcome. I'm glad to see you here. *I said with toothy smug sunny smile looking you straight in the eye* What brings you...`
### Examples of dialogue
@ -116,13 +116,13 @@ Example:
```
<START>
{{user}}: Hi Aqua, I heard you like to spend time in the pub.
{{char}}: *excitedly* Oh my goodness, yes! I just love spending time at the pub! It's so much fun to talk to all the adventurers and hear about their exciting adventures! And you are?
{{user}}: I'm a new here and I wanted to ask for your advice.
{{char}}: *giggles* Oh, advice! I love giving advice! And in gratitude for that, treat me to a drink! *gives signals to the bartender*
{{user}}: Hi Aqua, I heard you like to spend time in the pub.
{{char}}: *excitedly* Oh my goodness, yes! I just love spending time at the pub! It's so much fun to talk to all the adventurers and hear about their exciting adventures! And you are?
{{user}}: I'm a new here and I wanted to ask for your advice.
{{char}}: *giggles* Oh, advice! I love giving advice! And in gratitude for that, treat me to a drink! *gives signals to the bartender*
<START>
{{user}}: Hello
{{user}}: Hello
{{char}}: *excitedly* Hello there, dear! Are you new to Axel? Don't worry, I, Aqua the goddess of water, am here to help you! Do you need any assistance? And may I say, I look simply radiant today! *strikes a pose and looks at you with puppy eyes*
```
@ -134,7 +134,7 @@ Circumstances and context of the dialogue.
_A list of tags that are replaced when sending to generate:_
1. {{user}} and &lt;USER&gt; are replaced by the User's Name
1. {{user}} and &lt;USER&gt; are replaced by the User's Name
2. {{char}} and &lt;BOT&gt; are replaced by the Character's Name
3. {{time}} is replaced with the current system time.
4. {{date}} is replaced with the current system date.
@ -163,7 +163,7 @@ _It is important to note that while World Info helps guide the AI towards your d
#### Key
A list of keywords that trigger the activation of a World Info entry.
A list of keywords that trigger the activation of a World Info entry. Keys are not case-sensitive by default (this is [configurable](#casesensitivekeys)).
#### Secondary Key
@ -234,6 +234,14 @@ Content: Rufus is a dog.
**Both** of them will be pulled into the context if the message text mentions **just Bessie**.
### Case-sensitive keys
**To get pulled into the context, entry keys need to match the case as they are defined in the World Info entry.**
This is useful when your keys are common words or parts of common words.
For example, when this setting is active, keys 'rose' and 'Rose' will be treated differently, depending on the inputs.
## KoboldAI
### Basic Settings
@ -258,7 +266,7 @@ The maximum amount of tokens that the AI will generate to respond. One word is a
#### Context size
How much will the AI remember. Context size also affects the speed of generation.
How much will the AI remember. Context size also affects the speed of generation.
_Important_: The setting of Context Size in SillyTavern GUI overrides the setting for KoboldAI GUI
@ -323,10 +331,10 @@ They are created by training the AI with a special type of prompt using a collec
To get a NovelAI API key, follow these instructions:
1. Go to the NovelAI website and Login.
2. Create a new story, or open an existing story.
3. Open the Network Tools on your web browser. (For Chrome or Firefox, you do this by pressing Ctrl+Shift+I, then switching to the Network tab.)
4. Generate something. You should see two requests to [api.novelai.net/ai/generate-stream](http://api.novelai.net/ai/generate-stream), which might look something like this:
1. Go to the NovelAI website and Login.
2. Create a new story, or open an existing story.
3. Open the Network Tools on your web browser. (For Chrome or Firefox, you do this by pressing Ctrl+Shift+I, then switching to the Network tab.)
4. Generate something. You should see two requests to [api.novelai.net/ai/generate-stream](http://api.novelai.net/ai/generate-stream), which might look something like this:
![1.png](1.png)
@ -340,7 +348,7 @@ The long string (after "Bearer", not including it) is your API key.
### Settings
The files with the settings are here (SillyTavern\public\NovelAI Settings).
The files with the settings are here (SillyTavern\public\NovelAI Settings).
You can also manually add your own settings files.
#### Temperature
@ -510,12 +518,12 @@ Overrides the default separators controlled by "Disable example chats formatting
#### Disable example chats formatting
`<START>` won't be added at the beginning of each example message block.
`<START>` won't be added at the beginning of each example message block.
_(If custom separator is not set)_
#### Disable chat start formatting
`<START>` won't be added between the character card and the chat log.
`<START>` won't be added between the character card and the chat log.
_(If custom separator is not set)_
#### Always add character's name to prompt
@ -538,12 +546,12 @@ Has no effect.
#### Disable example chats formatting
`This is how **Character** should talk` won't be added at the beginning of each example message block.
`This is how **Character** should talk` won't be added at the beginning of each example message block.
_(If custom separator is not set)_
#### Disable chat start formatting
`Then the roleplay chat between **User** and **Character** begins` won't be added between the character card and the chat log.
`Then the roleplay chat between **User** and **Character** begins` won't be added between the character card and the chat log.
_(If custom separator is not set)_
#### Always add character's name to prompt
@ -552,7 +560,7 @@ Appends character's name to the prompt to force the model to complete the messag
```
** OTHER CONTEXT HERE **
Character:
Character:
```
## Group Chats

11
public/script.js
View File

@ -296,7 +296,7 @@ const system_messages = {
<li><tt>{​{text}​}</tt> - sets a one-time behavioral bias for the AI. Resets when you send the next message.
</li>
</ul>
Hotkeys/Keybinds:
Hotkeys/Keybinds:
<ul>
<li><tt>Up</tt> = Edit last message in chat</li>
<li><tt>Ctrl+Up</tt> = Edit last USER message in chat</li>
@ -458,7 +458,7 @@ function getTokenCount(str, padding = undefined) {
jQuery.ajax({
async: false,
type: 'POST', //
url: `/ tokenize_llama`,
url: `/tokenize_llama`,
data: JSON.stringify({ text: str }),
dataType: "json",
contentType: "application/json",
@ -2434,7 +2434,7 @@ function getMaxContextSize() {
if (novel_tier === 1) {
this_max_context = 1024;
} else {
this_max_context = 2048 - 60; //fix for fat tokens
this_max_context = 2048 - 60; //fix for fat tokens
if (nai_settings.model_novel == 'krake-v2') {
this_max_context -= 160;
}
@ -2715,7 +2715,7 @@ function promptItemize(itemizedPrompts, requestedMesId) {
<div class="flex1" style="color: gold;">World Info:</div>
<div class="">${worldInfoStringTokens}</div>
</div>
<div class="wide100p flex-container">
<div class="wide100p flex-container">
<div class="flex1" style="color: palegreen;">Chat History:</div>
<div class=""> ${ActualChatHistoryTokens}</div>
</div>
@ -2802,7 +2802,7 @@ function promptItemize(itemizedPrompts, requestedMesId) {
<div class="flex1" style="color: gold;">World Info:</div>
<div class="">${worldInfoStringTokens}</div>
</div>
<div class="wide100p flex-container">
<div class="wide100p flex-container">
<div class="flex1" style="color: palegreen;">Chat History:</div>
<div class=""> ${ActualChatHistoryTokens}</div>
</div>
@ -3852,6 +3852,7 @@ async function saveSettings(type) {
world_info_depth: world_info_depth,
world_info_budget: world_info_budget,
world_info_recursive: world_info_recursive,
world_info_case_sensitive: world_info_case_sensitive,
textgenerationwebui_settings: textgenerationwebui_settings,
swipes: swipes,
horde_settings: horde_settings,

29
public/scripts/world-info.js
View File

@ -7,6 +7,7 @@ export {
world_info_budget,
world_info_depth,
world_info_recursive,
world_info_case_sensitive,
world_names,
imported_world_name,
checkWorldInfo,
@ -23,6 +24,7 @@ let world_info_depth = 2;
let world_info_budget = 128;
let is_world_edit_open = false;
let world_info_recursive = false;
let world_info_case_sensitive = false;
let imported_world_name = "";
const saveWorldDebounced = debounce(async () => await _save(), 500);
const saveSettingsDebounced = debounce(() => saveSettings(), 500);
@ -51,6 +53,8 @@ function setWorldInfoSettings(settings, data) {
world_info_budget = Number(settings.world_info_budget);
if (settings.world_info_recursive !== undefined)
world_info_recursive = Boolean(settings.world_info_recursive);
if (settings.world_info_case_sensitive !== undefined)
world_info_case_sensitive = Boolean(settings.world_info_case_sensitive);
$("#world_info_depth_counter").text(world_info_depth);
$("#world_info_depth").val(world_info_depth);
@ -59,6 +63,7 @@ function setWorldInfoSettings(settings, data) {
$("#world_info_budget").val(world_info_budget);
$("#world_info_recursive").prop('checked', world_info_recursive);
$("#world_info_case_sensitive").prop('checked', world_info_case_sensitive);
world_names = data.world_names?.length ? data.world_names : [];
@ -476,13 +481,18 @@ async function createNewWorldInfo() {
}
}
// Gets a string that respects the case sensitivity setting
function transformString(str) {
return world_info_case_sensitive ? str : str.toLowerCase();
}
function checkWorldInfo(chat) {
if (world_info_data.entries.length == 0) {
return "";
}
const messagesToLookBack = world_info_depth * 2;
let textToScan = chat.slice(0, messagesToLookBack).join("").toLowerCase();
let textToScan = transformString(chat.slice(0, messagesToLookBack).join(""));
let worldInfoBefore = "";
let worldInfoAfter = "";
let needsToScan = true;
@ -507,7 +517,7 @@ function checkWorldInfo(chat) {
if (Array.isArray(entry.key) && entry.key.length) {
primary: for (let key of entry.key) {
const substituted = substituteParams(key);
if (substituted && textToScan.includes(substituted.trim().toLowerCase())) {
if (substituted && textToScan.includes(transformString(substituted.trim()))) {
if (
entry.selective &&
Array.isArray(entry.keysecondary) &&
@ -517,7 +527,7 @@ function checkWorldInfo(chat) {
const secondarySubstituted = substituteParams(keysecondary);
if (
secondarySubstituted &&
textToScan.includes(secondarySubstituted.trim().toLowerCase())
textToScan.includes(transformString(secondarySubstituted.trim()))
) {
activatedNow.add(entry.uid);
break secondary;
@ -557,11 +567,7 @@ function checkWorldInfo(chat) {
}
if (needsToScan) {
textToScan =
newEntries
.map((x) => x.content)
.join("\n")
.toLowerCase() + textToScan;
textToScan = (transformString(newEntries.map(x => x.content).join('\n')) + textToScan);
}
allActivatedEntries = new Set([...allActivatedEntries, ...activatedNow]);
@ -583,7 +589,7 @@ function selectImportedWorldInfo() {
imported_world_name = "";
}
$(document).ready(() => {
jQuery(() => {
$("#world_info").change(async function () {
const selectedWorld = $("#world_info").find(":selected").val();
world_info = null;
@ -688,4 +694,9 @@ $(document).ready(() => {
world_info_recursive = !!$(this).prop('checked');
saveSettingsDebounced();
})
$('#world_info_case_sensitive').on('input', function () {
world_info_case_sensitive = !!$(this).prop('checked');
saveSettingsDebounced();
})
});