rixty
/
KoboldAI-Client
mirror of https://github.com/KoboldAI/KoboldAI-Client.git
1
1
Fork 0
Code Issues Projects Releases Wiki Activity
KoboldAI-Client/aiserver.py

4708 lines
204 KiB
Python
Raw Normal View History

Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#!/usr/bin/python3
#==================================================================#
# KoboldAI
0.17 polish
2022-01-23 01:05:09 +01:00
# Version: 1.17.0
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# By: KoboldAIDev and the KoboldAI Community
#==================================================================#
# External packages
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
import eventlet
Compile TPU backend in background
2022-01-07 19:47:21 +01:00
eventlet.monkey_patch(all=True, thread=False)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
import os
Windows Color Workaround Different approach that activates this in windows, hopefully now Linux compatible.
2021-12-26 22:46:15 +01:00
os.system("")
Set eventlet threadpool size back to 1
2022-01-20 08:10:57 +01:00
os.environ['EVENTLET_THREADPOOL_SIZE'] = '1'
Revert parralism Testing is done, seems to cause issues in the order things happen with the interface.
2022-02-01 18:58:48 +01:00
os.environ['TOKENIZERS_PARALLELISM'] = 'false'
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
from eventlet import tpool
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
from os import path, getcwd
Show message when TPU backend is compiling
2022-01-17 03:09:10 +01:00
import time
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
import re
import json
import collections
import zipfile
Enable `low_cpu_mem_usage`
2021-11-26 00:09:25 +01:00
import packaging
Load model directly in fp16 if using GPU or breakmodel
2021-11-26 16:55:52 +01:00
import contextlib
Show full stack trace for generator errors to help in diagnosing errors
2021-12-15 08:03:08 +01:00
import traceback
Compile TPU backend in background
2022-01-07 19:47:21 +01:00
import threading
Welcome Message API Allows model creators to customize the welcome message using Markdown and Limited HTML Existing United users need to run install_requirements..bat again, you can leave the existing dependencies intact.
2022-01-30 19:47:30 +01:00
import markdown
import bleach
Repetition penalty slope and range
2022-01-24 21:30:38 +01:00
from collections.abc import Iterable
Add TPU support for dynamic WI scan and generation modifiers
2022-01-15 03:39:02 +01:00
from typing import Any, Callable, TypeVar, Tuple, Union, Dict, Set, List
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
import requests
import html
import argparse
import sys
import gc
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
import lupa
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# KoboldAI
import fileops
import gensettings
from utils import debounce
import utils
import structures
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
if lupa.LUA_VERSION[:2] != (5, 4):
print(f"Please install lupa==1.10. You have lupa {lupa.__version__}.", file=sys.stderr)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#==================================================================#
# Variables & Storage
#==================================================================#
# Terminal tags for colored text
class colors:
PURPLE = '\033[95m'
BLUE = '\033[94m'
CYAN = '\033[96m'
GREEN = '\033[92m'
YELLOW = '\033[93m'
RED = '\033[91m'
END = '\033[0m'
UNDERLINE = '\033[4m'
# AI models
modellist = [
["Load a model from its directory", "NeoCustom", ""],
["Load an old GPT-2 model (eg CloverEdition)", "GPT2Custom", ""],
Improved Model Support Changed the model VRAM requirements to what you'd need to comfortably run the model rather than barely (Like with the manual). Will probably revise this in a later commit. More importantly, it now supports models that use </s> which will be required to support XGLM and Fairseq models.
2022-01-28 18:03:30 +01:00
["Skein 6B (Hybrid)", "KoboldAI/GPT-J-6B-Skein", "16GB"],
["Adventure 6B", "KoboldAI/GPT-J-6B-Adventure", "16GB"],
["Lit 6B (NSFW)", "hakurei/lit-6B", "16GB"],
["C1 6B (Chatbot)", "hakurei/c1-6B", "16GB"],
["Picard 2.7B (Novel)", "KoboldAI/GPT-Neo-2.7B-Picard", "8GB"],
["Adventure 2.7B", "KoboldAI/GPT-Neo-2.7B-AID", "8GB"],
["Horni 2.7B (NSFW)", "KoboldAI/GPT-Neo-2.7B-Horni", "8GB"],
["Horni-LN 2.7B (Novel)", "KoboldAI/GPT-Neo-2.7B-Horni-LN", "8GB"],
["Shinen 2.7B (NSFW)", "KoboldAI/GPT-Neo-2.7B-Shinen", "8GB"],
Threading + Memory Sizes Polish effort to suppress a warning and list more accurate VRAM as tested with the full 2048 max tokens.
2022-01-30 13:56:25 +01:00
["GPT-J 6B", "EleutherAI/gpt-j-6B", "16GB"],
Improved Model Support Changed the model VRAM requirements to what you'd need to comfortably run the model rather than barely (Like with the manual). Will probably revise this in a later commit. More importantly, it now supports models that use </s> which will be required to support XGLM and Fairseq models.
2022-01-28 18:03:30 +01:00
["GPT-Neo 2.7B", "EleutherAI/gpt-neo-2.7B", "8GB"],
["GPT-Neo 1.3B", "EleutherAI/gpt-neo-1.3B", "6GB"],
Threading + Memory Sizes Polish effort to suppress a warning and list more accurate VRAM as tested with the full 2048 max tokens.
2022-01-30 13:56:25 +01:00
["GPT-2 XL", "gpt2-xl", "6GB"],
["GPT-2 Large", "gpt2-large", "4GB"],
["GPT-2 Med", "gpt2-medium", "2GB"],
Improved Model Support Changed the model VRAM requirements to what you'd need to comfortably run the model rather than barely (Like with the manual). Will probably revise this in a later commit. More importantly, it now supports models that use </s> which will be required to support XGLM and Fairseq models.
2022-01-28 18:03:30 +01:00
["GPT-2", "gpt2", "2GB"],
More Models Added more models in the menu, all the popular community models are now easily accessible. I also re-ordered the menu from large to small to have it make a bit more sense.
2021-12-27 01:02:05 +01:00
["OpenAI API (requires API key)", "OAI", ""],
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
["InferKit API (requires API key)", "InferKit", ""],
Small model description update
2022-01-11 00:29:35 +01:00
["KoboldAI Server API (Old Google Colab)", "Colab", ""],
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
["Read Only (No AI)", "ReadOnly", ""]
]
# Variables
class vars:
lastact = "" # The last action received from the user
Add Lua API for reading the text the user submitted to the AI
2021-12-12 07:52:42 +01:00
submission = "" # Same as above, but after applying input formatting
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
lastctx = "" # The last context submitted to the generator
model = "" # Model ID string chosen at startup
Model Type support Automatically detect or assume the model type so we do not have to hardcode all the different models people might use. This almost makes the behavior of --model identical to the NeoCustom behavior as far as the CLI is concerned. But only if the model_type is defined in the models config file.
2021-12-23 02:50:06 +01:00
model_type = "" # Model Type (Automatically taken from the model config)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
noai = False # Runs the script without starting up the transformers pipeline
aibusy = False # Stops submissions while the AI is working
max_length = 1024 # Maximum number of tokens to submit per action
ikmax = 3000 # Maximum number of characters to submit to InferKit
genamt = 80 # Amount of text for each action to generate
ikgen = 200 # Number of characters for InferKit to generate
rep_pen = 1.1 # Default generator repetition_penalty
Defaults and Downloads Default settings for the new repetition penalty settings (Better suggestions very much welcome since broader community testing has not been done). Updated the Readme with the link to the offline installer.
2022-02-03 13:13:26 +01:00
rep_pen_slope = 1.0 # Default generator repetition penalty slope
rep_pen_range = 512 # Default generator repetition penalty range
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
temp = 0.5 # Default generator temperature
top_p = 0.9 # Default generator top_p
top_k = 0 # Default generator top_k
tfs = 1.0 # Default generator tfs (tail-free sampling)
numseqs = 1 # Number of sequences to ask the generator to create
gamestarted = False # Whether the game has started (disables UI elements)
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
gamesaved = True # Whether or not current game is saved
Catch Lua errors properly
2021-12-13 08:32:09 +01:00
serverstarted = False # Whether or not the Flask server has started
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
prompt = "" # Prompt
memory = "" # Text submitted to memory field
authornote = "" # Text submitted to Author's Note field
Author's Note Template
2021-12-30 05:43:36 +01:00
authornotetemplate = "[Author's note: <|>]" # Author's note template
setauthornotetemplate = authornotetemplate # Saved author's note template in settings
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
andepth = 3 # How far back in history to append author's note
actions = structures.KoboldStoryRegister() # Actions submitted by user and AI
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
worldinfo = [] # List of World Info key/value objects
Fix numerous problems in the Lua world info API
2021-12-12 01:11:38 +01:00
worldinfo_i = [] # List of World Info key/value objects sans uninitialized entries
Upload bridge.lua, default.lua and some Lua libs base64 inspect json.lua Lua-hashings Lua-nums Moses mt19937ar-lua Penlight Serpent
2021-12-11 01:45:57 +01:00
worldinfo_u = {} # Dictionary of World Info UID - key/value pairs
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
wifolders_d = {} # Dictionary of World Info folder UID-info pairs
wifolders_l = [] # List of World Info folder UIDs
Upload bridge.lua, default.lua and some Lua libs base64 inspect json.lua Lua-hashings Lua-nums Moses mt19937ar-lua Penlight Serpent
2021-12-11 01:45:57 +01:00
wifolders_u = {} # Dictionary of pairs of folder UID - list of WI UID
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
lua_state = None # Lua state of the Lua scripting system
lua_koboldbridge = None # `koboldbridge` from bridge.lua
lua_kobold = None # `kobold` from` bridge.lua
lua_koboldcore = None # `koboldcore` from bridge.lua
Format Lua printed messages and warnings
2021-12-13 07:59:53 +01:00
lua_logname = ... # Name of previous userscript that logged to terminal
Display icons for active userscripts and softprompts Also fixes the userscript menu so that the active userscripts preserve the previously selected order as was originally intended.
2021-12-23 05:33:27 +01:00
lua_running = False # Whether or not Lua is running (i.e. wasn't stopped due to an error)
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
lua_edited = set() # Set of chunk numbers that were edited from a Lua generation modifier
lua_deleted = set() # Set of chunk numbers that were deleted from a Lua generation modifier
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
generated_tkns = 0 # If using a backend that supports Lua generation modifiers, how many tokens have already been generated, otherwise 0
Allow pressing send button again to stop generation
2022-01-10 22:36:15 +01:00
abort = False # Whether or not generation was aborted by clicking on the submit button during generation
Show message when TPU backend is compiling
2022-01-17 03:09:10 +01:00
compiling = False # If using a TPU Colab, this will be set to True when the TPU backend starts compiling and then set to False again
checking = False # Whether or not we are actively checking to see if TPU backend is compiling or not
Save softprompt filename in settings
2021-12-23 19:02:11 +01:00
spfilename = "" # Filename of soft prompt to load, or an empty string if not using a soft prompt
Add Userscripts menu into GUI
2021-12-13 07:03:26 +01:00
userscripts = [] # List of userscripts to load
Display icons for active userscripts and softprompts Also fixes the userscript menu so that the active userscripts preserve the previously selected order as was originally intended.
2021-12-23 05:33:27 +01:00
last_userscripts = [] # List of previous userscript filenames from the previous time userscripts were send via usstatitems
Add a "corescript" setting
2021-12-14 01:28:33 +01:00
corescript = "default.lua" # Filename of corescript to load
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# badwords = [] # Array of str/chr values that should be removed from output
badwordsids = [[13460], [6880], [50256], [42496], [4613], [17414], [22039], [16410], [27], [29], [38430], [37922], [15913], [24618], [28725], [58], [47175], [36937], [26700], [12878], [16471], [37981], [5218], [29795], [13412], [45160], [3693], [49778], [4211], [20598], [36475], [33409], [44167], [32406], [29847], [29342], [42669], [685], [25787], [7359], [3784], [5320], [33994], [33490], [34516], [43734], [17635], [24293], [9959], [23785], [21737], [28401], [18161], [26358], [32509], [1279], [38155], [18189], [26894], [6927], [14610], [23834], [11037], [14631], [26933], [46904], [22330], [25915], [47934], [38214], [1875], [14692], [41832], [13163], [25970], [29565], [44926], [19841], [37250], [49029], [9609], [44438], [16791], [17816], [30109], [41888], [47527], [42924], [23984], [49074], [33717], [31161], [49082], [30138], [31175], [12240], [14804], [7131], [26076], [33250], [3556], [38381], [36338], [32756], [46581], [17912], [49146]] # Tokenized array of badwords used to prevent AI artifacting
Make all WI commands use UIDs instead of nums
2022-01-01 03:22:51 +01:00
deletewi = None # Temporary storage for UID to delete
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
wirmvwhtsp = False # Whether to remove leading whitespace from WI entries
widepth = 3 # How many historical actions to scan for WI hits
mode = "play" # Whether the interface is in play, memory, or edit mode
editln = 0 # Which line was last selected in Edit Mode
Fix an error that occurs when all layers are on second GPU
2021-12-16 01:03:49 +01:00
gpu_device = 0 # Which PyTorch device to use when using pure GPU generation
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
url = "https://api.inferkit.com/v1/models/standard/generate" # InferKit API URL
oaiurl = "" # OpenAI API URL
oaiengines = "https://api.openai.com/v1/engines"
colaburl = "" # Ngrok url for Google Colab mode
apikey = "" # API key to use for InferKit API calls
oaiapikey = "" # API key to use for OpenAI API calls
savedir = getcwd()+"\stories"
hascuda = False # Whether torch has detected CUDA on the system
usegpu = False # Whether to launch pipeline with GPU support
custmodpth = "" # Filesystem location of custom model to run
formatoptns = {'frmttriminc': True, 'frmtrmblln': False, 'frmtrmspch': False, 'frmtadsnsp': False, 'singleline': False} # Container for state of formatting options
importnum = -1 # Selection on import popup list
importjs = {} # Temporary storage for import data
loadselect = "" # Temporary storage for story filename to load
spselect = "" # Temporary storage for soft prompt filename to load
Display icons for active userscripts and softprompts Also fixes the userscript menu so that the active userscripts preserve the previously selected order as was originally intended.
2021-12-23 05:33:27 +01:00
spmeta = None # Metadata of current soft prompt, or None if not using a soft prompt
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
sp = None # Current soft prompt tensor (as a NumPy array)
Add soft prompt support to TPU backend
2021-11-22 00:08:04 +01:00
sp_length = 0 # Length of current soft prompt in tokens, or 0 if not using a soft prompt
Use original TPU backend if possible
2022-01-16 05:31:07 +01:00
has_genmod = False # Whether or not at least one loaded Lua userscript has a generation modifier
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
svowname = "" # Filename that was flagged for overwrite confirm
saveow = False # Whether or not overwrite confirm has been displayed
Added autosave feature. When action is submitted it will save if the save setting is on and if the filename is set.
2022-01-20 13:46:34 +01:00
autosave = False # Whether or not to automatically save after each action
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
genseqs = [] # Temporary storage for generated sequences
recentback = False # Whether Back button was recently used without Submitting or Retrying after
Allow retry button to regenerate random story
2021-12-27 01:52:56 +01:00
recentrng = None # If a new random game was recently generated without Submitting after, this is the topic used (as a string), otherwise this is None
Retrying random story generation now also remembers memory
2022-01-04 20:40:10 +01:00
recentrngm = None # If a new random game was recently generated without Submitting after, this is the memory used (as a string), otherwise this is None
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
useprompt = False # Whether to send the full prompt with every submit action
breakmodel = False # For GPU users, whether to use both system RAM and VRAM to conserve VRAM while offering speedup compared to CPU-only
bmsupported = False # Whether the breakmodel option is supported (GPT-Neo/GPT-J only, currently)
NoBreakmodel variable Adds a Nobreakmodel var that allows Breakmodel to be turned off. This can be done trough commandline or a model config (In case Neo is used by the models config without it being a true Neo model that is compatible with breakmodel). In addition I removed the args.colab check for breakmodel support and instead make args.colab activate nobreakmodel. And I have added a new check so that breakmodel is not even attempted if you do not specify the layers but do launch a model from the command line.
2022-01-30 17:06:15 +01:00
nobreakmodel = False # Something specifically requested Breakmodel to be disabled (For example a models config)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
smandelete = False # Whether stories can be deleted from inside the browser
smanrename = False # Whether stories can be renamed from inside the browser
allowsp = False # Whether we are allowed to use soft prompts (by default enabled if we're using GPT-2, GPT-Neo or GPT-J)
modeldim = -1 # Embedding dimension of your model (e.g. it's 4096 for GPT-J-6B and 2560 for GPT-Neo-2.7B)
laststory = None # Filename (without extension) of most recent story JSON file we loaded
regex_sl = re.compile(r'\n*(?<=.) *\n(.|\n)*') # Pattern for limiting the output to a single line
acregex_ai = re.compile(r'\n* *>(.|\n)*') # Pattern for matching adventure actions from the AI so we can remove them
acregex_ui = re.compile(r'^ *(&gt;.*)$', re.MULTILINE) # Pattern for matching actions in the HTML-escaped story so we can apply colouring, etc (make sure to encase part to format in parentheses)
Simplify the comment regex
2021-11-21 07:09:19 +01:00
comregex_ai = re.compile(r'(?:\n<\|(?:.|\n)*?\|>(?=\n|$))|(?:<\|(?:.|\n)*?\|>\n?)') # Pattern for matching comments to remove them before sending them to the AI
Add syntax for comments <|...|>
2021-11-20 07:27:57 +01:00
comregex_ui = re.compile(r'(&lt;\|(?:.|\n)*?\|&gt;)') # Pattern for matching comments in the editor
Chat Mode The Initial commit for Chat Mode, the nickname part of the UI is missing other than that it should be fully functional. To use Chat Mode effectively you first input a small dialogue (Can be around 6 lines 3 of your own inputs and 3 of the character) formatted as Name : it will then automate the actions needed to chat properly. During this mode single line mode is forced on, and Trim Incomplete Sentences is forced off.
2021-12-26 01:51:32 +01:00
chatmode = False
chatname = "You"
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
adventure = False
Chat Mode The Initial commit for Chat Mode, the nickname part of the UI is missing other than that it should be fully functional. To use Chat Mode effectively you first input a small dialogue (Can be around 6 lines 3 of your own inputs and 3 of the character) formatted as Name : it will then automate the actions needed to chat properly. During this mode single line mode is forced on, and Trim Incomplete Sentences is forced off.
2021-12-26 01:51:32 +01:00
actionmode = 1
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
dynamicscan = False
remote = False
No Prompt Gen Allow people to enter a prompt without generating anything by the AI. Combined with the always add prompt this is a very useful feature that allows people to write world information first, and then do a specific action. This mimics the behavior previously seen in AI Dungeon forks where it prompts for world information and then asks an action and can be particularly useful for people who want the prompt to always be part of the generation.
2021-12-16 12:47:44 +01:00
nopromptgen = False
Add Memory box to Random Story dialog and "Random Story Persist"
2021-12-30 05:15:59 +01:00
rngpersist = False
Use original TPU backend if possible
2022-01-16 05:31:07 +01:00
nogenmod = False
Welcome Message API Allows model creators to customize the welcome message using Markdown and Limited HTML Existing United users need to run install_requirements..bat again, you can leave the existing dependencies intact.
2022-01-30 19:47:30 +01:00
welcome = False # Custom Welcome Text (False is default)
</s> new line mode Needed for Fairseq and XGLM models that do not understand the regular \n .
2022-01-31 18:39:34 +01:00
newlinemode = "n"
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#==================================================================#
# Function to get model selection at startup
#==================================================================#
def getModelSelection():
Threading + Memory Sizes Polish effort to suppress a warning and list more accurate VRAM as tested with the full 2048 max tokens.
2022-01-30 13:56:25 +01:00
print(" # Model VRAM\n =========================================")
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
i = 1
for m in modellist:
print(" {0} - {1}\t\t{2}".format("{:<2}".format(i), m[0].ljust(15), m[2]))
i += 1
print(" ");
modelsel = 0
vars.model = ''
while(vars.model == ''):
modelsel = input("Model #> ")
if(modelsel.isnumeric() and int(modelsel) > 0 and int(modelsel) <= len(modellist)):
More Lua API fixes * Removed `vars.model_orig` * `requirex()` in bridge.lua now maintains a separate module cache for each userscript instead of using the same cache for all userscripts * `vars.lua_deleted` and `vars.lua_edited` are now erased right before running the input modifiers instead of right before each time the generation modifiers are run
2021-12-26 18:49:28 +01:00
vars.model = modellist[int(modelsel)-1][1]
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
else:
print("{0}Please enter a valid selection.{1}".format(colors.RED, colors.END))
# If custom model was selected, get the filesystem location and store it
if(vars.model == "NeoCustom" or vars.model == "GPT2Custom"):
print("{0}Please choose the folder where pytorch_model.bin is located:{1}\n".format(colors.CYAN, colors.END))
modpath = fileops.getdirpath(getcwd(), "Select Model Folder")
if(modpath):
# Save directory to vars
vars.custmodpth = modpath
else:
# Print error and retry model selection
print("{0}Model select cancelled!{1}".format(colors.RED, colors.END))
print("{0}Select an AI model to continue:{1}\n".format(colors.CYAN, colors.END))
getModelSelection()
#==================================================================#
# Return all keys in tokenizer dictionary containing char
#==================================================================#
def gettokenids(char):
keys = []
for key in vocab_keys:
if(key.find(char) != -1):
keys.append(key)
return keys
#==================================================================#
# Return Model Name
#==================================================================#
def getmodelname():
if(args.configname):
modelname = args.configname
return modelname
Integrate TPU backend This commit puts the TPU backend code directly in to the KoboldAI code to make it easier to modify.
2021-11-20 00:06:57 +01:00
if(vars.model in ("NeoCustom", "GPT2Custom", "TPUMeshTransformerGPTJ")):
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
modelname = os.path.basename(os.path.normpath(vars.custmodpth))
return modelname
else:
modelname = vars.model
return modelname
#==================================================================#
# Breakmodel configuration functions
#==================================================================#
def device_list(n_layers, primary=None, selected=None):
device_count = torch.cuda.device_count()
if(device_count < 2):
primary = None
gpu_blocks = breakmodel.gpu_blocks + (device_count - len(breakmodel.gpu_blocks))*[0]
print(f"{colors.YELLOW} DEVICE ID | LAYERS | DEVICE NAME{colors.END}")
for i in range(device_count):
name = torch.cuda.get_device_name(i)
if(len(name) > 47):
name = "..." + name[-44:]
row_color = colors.END
sep_color = colors.YELLOW
print(f"{row_color}{colors.YELLOW + '->' + row_color if i == selected else ' '} {'(primary)' if i == primary else ' '*9} {i:3} {sep_color}|{row_color} {gpu_blocks[i]:3} {sep_color}|{row_color} {name}{colors.END}")
row_color = colors.END
sep_color = colors.YELLOW
print(f"{row_color} {' '*9} N/A {sep_color}|{row_color} {n_layers:3} {sep_color}|{row_color} (CPU){colors.END}")
def device_config(model):
global breakmodel, generator
import breakmodel
num_layers fixes As requested by VE_FORBRYDERNE (Possibly implemented it on to many places, needs testing but since the other one is already broken I am committing it first so I can more easily test)
2021-11-24 03:44:11 +01:00
n_layers = model.config.num_layers if hasattr(model.config, "num_layers") else model.config.n_layer
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if(args.breakmodel_gpulayers is not None):
try:
breakmodel.gpu_blocks = list(map(int, args.breakmodel_gpulayers.split(',')))
assert len(breakmodel.gpu_blocks) <= torch.cuda.device_count()
`--breakmodel_gpublocks` should handle -1 properly now
2022-01-04 20:43:37 +01:00
s = n_layers
for i in range(len(breakmodel.gpu_blocks)):
if(breakmodel.gpu_blocks[i] <= -1):
breakmodel.gpu_blocks[i] = s
break
else:
s -= breakmodel.gpu_blocks[i]
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
assert sum(breakmodel.gpu_blocks) <= n_layers
n_layers -= sum(breakmodel.gpu_blocks)
except:
Breakmodel Fixes Multiple old references and one mistake in my last commit fixed
2022-01-30 17:40:43 +01:00
print("WARNING: --breakmodel_gpulayers is malformatted. Please use the --help option to see correct usage of --breakmodel_gpulayers. Defaulting to all layers on device 0.", file=sys.stderr)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
breakmodel.gpu_blocks = [n_layers]
n_layers = 0
elif(args.breakmodel_layers is not None):
breakmodel.gpu_blocks = [n_layers - max(0, min(n_layers, args.breakmodel_layers))]
n_layers -= sum(breakmodel.gpu_blocks)
elif(args.model is not None):
print("Breakmodel not specified, assuming GPU 0")
breakmodel.gpu_blocks = [n_layers]
n_layers = 0
else:
device_count = torch.cuda.device_count()
if(device_count > 1):
print(colors.CYAN + "\nPlease select one of your GPUs to be your primary GPU.")
print("VRAM usage in your primary GPU will be higher than for your other ones.")
print("It is recommended you make your fastest GPU your primary GPU.")
device_list(n_layers)
while(True):
primaryselect = input("device ID> ")
if(primaryselect.isnumeric() and 0 <= int(primaryselect) < device_count):
breakmodel.primary_device = int(primaryselect)
break
else:
print(f"{colors.RED}Please enter an integer between 0 and {device_count-1}.{colors.END}")
else:
breakmodel.primary_device = 0
print(colors.PURPLE + "\nIf you don't have enough VRAM to run the model on a single GPU")
print("you can split the model between your CPU and your GPU(s), or between")
print("multiple GPUs if you have more than one.")
print("By putting more 'layers' on a GPU or CPU, more computations will be")
print("done on that device and more VRAM or RAM will be required on that device")
print("(roughly proportional to number of layers).")
print("It should be noted that GPUs are orders of magnitude faster than the CPU.")
print(f"This model has{colors.YELLOW} {n_layers} {colors.PURPLE}layers.{colors.END}\n")
for i in range(device_count):
device_list(n_layers, primary=breakmodel.primary_device, selected=i)
print(f"{colors.CYAN}\nHow many of the remaining{colors.YELLOW} {n_layers} {colors.CYAN}layers would you like to put into device {i}?\nYou can also enter -1 to allocate all remaining layers to this device.{colors.END}\n")
while(True):
layerselect = input("# of layers> ")
if((layerselect.isnumeric() or layerselect.strip() == '-1') and -1 <= int(layerselect) <= n_layers):
layerselect = int(layerselect)
layerselect = n_layers if layerselect == -1 else layerselect
breakmodel.gpu_blocks.append(layerselect)
n_layers -= layerselect
break
else:
print(f"{colors.RED}Please enter an integer between -1 and {n_layers}.{colors.END}")
if(n_layers == 0):
break
print(colors.PURPLE + "\nFinal device configuration:")
device_list(n_layers)
Use old GPU-only generation if all layers are on the same GPU Apparently, this mode uses less RAM than breakmodel does.
2021-11-15 00:42:18 +01:00
# If all layers are on the same device, use the old GPU generation mode
Correct `gpu_layers` to `gpu_blocks`
2021-11-15 03:08:49 +01:00
while(len(breakmodel.gpu_blocks) and breakmodel.gpu_blocks[-1] == 0):
breakmodel.gpu_blocks.pop()
num_layers fixes As requested by VE_FORBRYDERNE (Possibly implemented it on to many places, needs testing but since the other one is already broken I am committing it first so I can more easily test)
2021-11-24 03:44:11 +01:00
if(len(breakmodel.gpu_blocks) and breakmodel.gpu_blocks[-1] in (-1, model.config.num_layers if hasattr(model.config, "num_layers") else model.config.n_layer)):
Use old GPU-only generation if all layers are on the same GPU Apparently, this mode uses less RAM than breakmodel does.
2021-11-15 00:42:18 +01:00
vars.breakmodel = False
vars.usegpu = True
Fix an error that occurs when all layers are on second GPU
2021-12-16 01:03:49 +01:00
vars.gpu_device = len(breakmodel.gpu_blocks)-1
model = model.half().to(vars.gpu_device)
Use old GPU-only generation if all layers are on the same GPU Apparently, this mode uses less RAM than breakmodel does.
2021-11-15 00:42:18 +01:00
generator = model.generate
return
Polishing and Optimizations Multiple things have changed, for now models default to half mode even on the official transformers to make sure its as efficient on the GPU as finetune's. GPU selection is streamlined and cache files are now stored inside the KoboldAI folder (for the most part). A new command line parameter to force the models to run at their full size still needs to be added for the few users that would want a quality bump at the cost of ram.
2021-11-18 00:06:57 +01:00
if(not breakmodel.gpu_blocks):
print("Nothing assigned to a GPU, reverting to CPU only mode")
vars.breakmodel = False
vars.usegpu = False
Fix for CPU mode Recent optimizations caused the CPU version to load in an incompatible format, now we convert it back to the correct format after loading it efficiently first.
2021-11-27 05:34:29 +01:00
model = model.to('cpu').float()
Polishing and Optimizations Multiple things have changed, for now models default to half mode even on the official transformers to make sure its as efficient on the GPU as finetune's. GPU selection is streamlined and cache files are now stored inside the KoboldAI folder (for the most part). A new command line parameter to force the models to run at their full size still needs to be added for the few users that would want a quality bump at the cost of ram.
2021-11-18 00:06:57 +01:00
generator = model.generate
return
Use old GPU-only generation if all layers are on the same GPU Apparently, this mode uses less RAM than breakmodel does.
2021-11-15 00:42:18 +01:00
model.half().to('cpu')
gc.collect()
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
model.transformer.wte.to(breakmodel.primary_device)
model.transformer.ln_f.to(breakmodel.primary_device)
if(hasattr(model, 'lm_head')):
model.lm_head.to(breakmodel.primary_device)
Breakmodel support for GPTJModel
2021-11-26 00:09:16 +01:00
if(hasattr(model.transformer, 'wpe')):
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
model.transformer.wpe.to(breakmodel.primary_device)
gc.collect()
GPTNeoModel.forward = breakmodel.new_forward
Breakmodel support for GPTJModel
2021-11-26 00:09:16 +01:00
if("GPTJModel" in globals()):
GPTJModel.forward = breakmodel.new_forward
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
generator = model.generate
In breakmodel mode, move layers to GPU as soon as model loads Rather than during the first generation.
2021-11-25 17:44:41 +01:00
breakmodel.move_hidden_layers(model.transformer)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
NoBreakmodel variable Adds a Nobreakmodel var that allows Breakmodel to be turned off. This can be done trough commandline or a model config (In case Neo is used by the models config without it being a true Neo model that is compatible with breakmodel). In addition I removed the args.colab check for breakmodel support and instead make args.colab activate nobreakmodel. And I have added a new check so that breakmodel is not even attempted if you do not specify the layers but do launch a model from the command line.
2022-01-30 17:06:15 +01:00
#==================================================================#
# Allow the models to override some settings
#==================================================================#
def loadmodelsettings():
try:
model_js_config = str(model_config).partition(' ')[2]
js = json.loads(model_js_config)
except Exception as e:
try:
model_js_config = open(vars.custmodpth + "/config.json", "r")
except Exception as e:
model_js_config = open(vars.custmodpth.replace('/', '_') + "/config.json", "r")
js = json.load(model_js_config)
if("badwordsids" in js):
vars.badwordsids = js["badwordsids"]
if("nobreakmodel" in js):
vars.nobreakmodel = js["nobreakmodel"]
if("temp" in js):
vars.temp = js["temp"]
if("top_p" in js):
vars.top_p = js["top_p"]
if("top_k" in js):
vars.top_k = js["top_k"]
if("tfs" in js):
vars.tfs = js["tfs"]
if("rep_pen" in js):
vars.rep_pen = js["rep_pen"]
if("rep_pen_slope" in js):
vars.rep_pen_slope = js["rep_pen_slope"]
if("rep_pen_range" in js):
vars.rep_pen_range = js["rep_pen_range"]
if("adventure" in js):
vars.adventure = js["adventure"]
if("chatmode" in js):
vars.chatmode = js["chatmode"]
if("dynamicscan" in js):
vars.dynamicscan = js["dynamicscan"]
if("formatoptns" in js):
vars.formatoptns = js["formatoptns"]
Welcome Message API Allows model creators to customize the welcome message using Markdown and Limited HTML Existing United users need to run install_requirements..bat again, you can leave the existing dependencies intact.
2022-01-30 19:47:30 +01:00
if("welcome" in js):
vars.welcome = js["welcome"]
</s> new line mode Needed for Fairseq and XGLM models that do not understand the regular \n .
2022-01-31 18:39:34 +01:00
if("newlinemode" in js):
vars.newlinemode = js["newlinemode"]
NoBreakmodel variable Adds a Nobreakmodel var that allows Breakmodel to be turned off. This can be done trough commandline or a model config (In case Neo is used by the models config without it being a true Neo model that is compatible with breakmodel). In addition I removed the args.colab check for breakmodel support and instead make args.colab activate nobreakmodel. And I have added a new check so that breakmodel is not even attempted if you do not specify the layers but do launch a model from the command line.
2022-01-30 17:06:15 +01:00
if("antemplate" in js):
vars.setauthornotetemplate = js["antemplate"]
if(not vars.gamestarted):
vars.authornotetemplate = vars.setauthornotetemplate
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#==================================================================#
# Startup
#==================================================================#
# Parsing Parameters
parser = argparse.ArgumentParser(description="KoboldAI Server")
parser.add_argument("--remote", action='store_true', help="Optimizes KoboldAI for Remote Play")
Ngrok Support Not recommended for home users due to DDoS risks, but might make Colab tunnels more reliable.
2021-11-29 18:11:14 +01:00
parser.add_argument("--ngrok", action='store_true', help="Optimizes KoboldAI for Remote Play using Ngrok")
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
parser.add_argument("--model", help="Specify the Model Type to skip the Menu")
parser.add_argument("--path", help="Specify the Path for local models (For model NeoCustom or GPT2Custom)")
parser.add_argument("--cpu", action='store_true', help="By default unattended launches are on the GPU use this option to force CPU usage.")
parser.add_argument("--breakmodel", action='store_true', help=argparse.SUPPRESS)
parser.add_argument("--breakmodel_layers", type=int, help=argparse.SUPPRESS)
Breakmodel Fixes Multiple old references and one mistake in my last commit fixed
2022-01-30 17:40:43 +01:00
parser.add_argument("--breakmodel_gpulayers", type=str, help="If using a model that supports hybrid generation, this is a comma-separated list that specifies how many layers to put on each GPU device. For example to put 8 layers on device 0, 9 layers on device 1 and 11 layers on device 2, use --beakmodel_gpulayers 8,9,11")
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
parser.add_argument("--override_delete", action='store_true', help="Deleting stories from inside the browser is disabled if you are using --remote and enabled otherwise. Using this option will instead allow deleting stories if using --remote and prevent deleting stories otherwise.")
parser.add_argument("--override_rename", action='store_true', help="Renaming stories from inside the browser is disabled if you are using --remote and enabled otherwise. Using this option will instead allow renaming stories if using --remote and prevent renaming stories otherwise.")
parser.add_argument("--configname", help="Force a fixed configuration name to aid with config management.")
Colab Optimizations Breakmodel is useless on Colab, so for the sake of efficiency if --colab is present we will always assume a model is incompatible. The same applies to the conversion, colab's are discarded so converting the model to a .bin file only wastes time since the HDD isn't fast. Finally we automatically set all the useful variables for Colab, so that in the future this can be removed from ckds and other scripts. Lastly ckds has been adapted not to copy the examples folder and to add the new --colab parameter. Local players are much better off running the old --remote command.
2022-01-16 00:56:03 +01:00
parser.add_argument("--colab", action='store_true', help="Optimize for Google Colab.")
NoBreakmodel variable Adds a Nobreakmodel var that allows Breakmodel to be turned off. This can be done trough commandline or a model config (In case Neo is used by the models config without it being a true Neo model that is compatible with breakmodel). In addition I removed the args.colab check for breakmodel support and instead make args.colab activate nobreakmodel. And I have added a new check so that breakmodel is not even attempted if you do not specify the layers but do launch a model from the command line.
2022-01-30 17:06:15 +01:00
parser.add_argument("--nobreakmodel", action='store_true', help="Disables Breakmodel support completely.")
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
Accept command line arguments in `KOBOLDAI_ARGS` environment var So that you can use gunicorn or whatever with command-line arguments by passing the arguments in an environment variable.
2022-01-04 23:11:14 +01:00
args: argparse.Namespace = None
if(os.environ.get("KOBOLDAI_ARGS") is not None):
import shlex
args = parser.parse_args(shlex.split(os.environ["KOBOLDAI_ARGS"]))
else:
args = parser.parse_args()
Colab Optimizations Breakmodel is useless on Colab, so for the sake of efficiency if --colab is present we will always assume a model is incompatible. The same applies to the conversion, colab's are discarded so converting the model to a .bin file only wastes time since the HDD isn't fast. Finally we automatically set all the useful variables for Colab, so that in the future this can be removed from ckds and other scripts. Lastly ckds has been adapted not to copy the examples folder and to add the new --colab parameter. Local players are much better off running the old --remote command.
2022-01-16 00:56:03 +01:00
More Lua API fixes * Removed `vars.model_orig` * `requirex()` in bridge.lua now maintains a separate module cache for each userscript instead of using the same cache for all userscripts * `vars.lua_deleted` and `vars.lua_edited` are now erased right before running the input modifiers instead of right before each time the generation modifiers are run
2021-12-26 18:49:28 +01:00
vars.model = args.model;
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
Colab Optimizations Breakmodel is useless on Colab, so for the sake of efficiency if --colab is present we will always assume a model is incompatible. The same applies to the conversion, colab's are discarded so converting the model to a .bin file only wastes time since the HDD isn't fast. Finally we automatically set all the useful variables for Colab, so that in the future this can be removed from ckds and other scripts. Lastly ckds has been adapted not to copy the examples folder and to add the new --colab parameter. Local players are much better off running the old --remote command.
2022-01-16 00:56:03 +01:00
if args.colab:
args.remote = True;
args.override_rename = True;
args.override_delete = True;
NoBreakmodel variable Adds a Nobreakmodel var that allows Breakmodel to be turned off. This can be done trough commandline or a model config (In case Neo is used by the models config without it being a true Neo model that is compatible with breakmodel). In addition I removed the args.colab check for breakmodel support and instead make args.colab activate nobreakmodel. And I have added a new check so that breakmodel is not even attempted if you do not specify the layers but do launch a model from the command line.
2022-01-30 17:06:15 +01:00
args.nobreakmodel = True;
if args.nobreakmodel:
vars.nobreakmodel = True;
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if args.remote:
vars.remote = True;
Ngrok Support Not recommended for home users due to DDoS risks, but might make Colab tunnels more reliable.
2021-11-29 18:11:14 +01:00
if args.ngrok:
vars.remote = True;
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
vars.smandelete = vars.remote == args.override_delete
vars.smanrename = vars.remote == args.override_rename
# Select a model to run
if args.model:
print("Welcome to KoboldAI!\nYou have selected the following Model:", vars.model)
if args.path:
print("You have selected the following path for your Model :", args.path)
vars.custmodpth = args.path;
vars.colaburl = args.path + "/request"; # Lets just use the same parameter to keep it simple
else:
Threading + Memory Sizes Polish effort to suppress a warning and list more accurate VRAM as tested with the full 2048 max tokens.
2022-01-30 13:56:25 +01:00
print("{0}Welcome to the KoboldAI Server!\nListed RAM is the optimal VRAM and CPU ram can be up to twice the amount.\nMost models can run at less VRAM with reduced max tokens or less layers on the GPU.\nSelect an AI model to continue:{1}\n".format(colors.CYAN, colors.END))
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
getModelSelection()
# If transformers model was selected & GPU available, ask to use CPU or GPU
Integrate TPU backend This commit puts the TPU backend code directly in to the KoboldAI code to make it easier to modify.
2021-11-20 00:06:57 +01:00
if(not vars.model in ["InferKit", "Colab", "OAI", "ReadOnly", "TPUMeshTransformerGPTJ"]):
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
vars.allowsp = True
# Test for GPU support
import torch
Model Type support Automatically detect or assume the model type so we do not have to hardcode all the different models people might use. This almost makes the behavior of --model identical to the NeoCustom behavior as far as the CLI is concerned. But only if the model_type is defined in the models config file.
2021-12-23 02:50:06 +01:00
# Make model path the same as the model name to make this consistent with the other loading method if it isn't a known model type
# This code is not just a workaround for below, it is also used to make the behavior consistent with other loading methods - Henk717
if(not vars.model in ["NeoCustom", "GPT2Custom"]):
Replace model path differently The path correction was applied to soon and broke online loading, applying the replace where it is relevant instead.
2021-12-23 03:05:53 +01:00
vars.custmodpth = vars.model
Replace NeoCustom NeoCustom is now obsolete beyond the file selection and the CLI. So after the CLI we adapt the input to a generic model and then use the improved generic routine to handle it. This saves duplicate efforts of maintaining an almost identical routine now that models are handled by their type and not their name.
2021-12-23 17:10:02 +01:00
elif(vars.model == "NeoCustom"):
vars.model = os.path.basename(os.path.normpath(vars.custmodpth))
Model Type support Automatically detect or assume the model type so we do not have to hardcode all the different models people might use. This almost makes the behavior of --model identical to the NeoCustom behavior as far as the CLI is concerned. But only if the model_type is defined in the models config file.
2021-12-23 02:50:06 +01:00
# Get the model_type from the config or assume a model type if it isn't present
from transformers import AutoConfig
Path loading improvements This fixes a few scenario's of my commit yesterday, models that have a / are now first loaded from the corrected directory if it exists before we fall back to its original name to make sure it loads the config from the correct location. Cache dir fixes and a improved routine for the path loaded models that mimics the NeoCustom option fixing models that have no model_type specified. Because GPT2 doesn't work well with this option and should exclusively be used with the GPT2Custom and GPT-J models should have a model_type we assume its a Neo model when not specified.
2021-12-23 14:40:35 +01:00
if(os.path.isdir(vars.custmodpth.replace('/', '_'))):
try:
model_config = AutoConfig.from_pretrained(vars.custmodpth.replace('/', '_'), cache_dir="cache/")
vars.model_type = model_config.model_type
except ValueError as e:
vars.model_type = "not_found"
else:
try:
model_config = AutoConfig.from_pretrained(vars.custmodpth, cache_dir="cache/")
vars.model_type = model_config.model_type
except ValueError as e:
vars.model_type = "not_found"
if(vars.model_type == "not_found" and vars.model == "NeoCustom"):
Model Type support Automatically detect or assume the model type so we do not have to hardcode all the different models people might use. This almost makes the behavior of --model identical to the NeoCustom behavior as far as the CLI is concerned. But only if the model_type is defined in the models config file.
2021-12-23 02:50:06 +01:00
vars.model_type = "gpt_neo"
Path loading improvements This fixes a few scenario's of my commit yesterday, models that have a / are now first loaded from the corrected directory if it exists before we fall back to its original name to make sure it loads the config from the correct location. Cache dir fixes and a improved routine for the path loaded models that mimics the NeoCustom option fixing models that have no model_type specified. Because GPT2 doesn't work well with this option and should exclusively be used with the GPT2Custom and GPT-J models should have a model_type we assume its a Neo model when not specified.
2021-12-23 14:40:35 +01:00
elif(vars.model_type == "not_found" and vars.model == "GPT2Custom"):
Model Type support Automatically detect or assume the model type so we do not have to hardcode all the different models people might use. This almost makes the behavior of --model identical to the NeoCustom behavior as far as the CLI is concerned. But only if the model_type is defined in the models config file.
2021-12-23 02:50:06 +01:00
vars.model_type = "gpt2"
Path loading improvements This fixes a few scenario's of my commit yesterday, models that have a / are now first loaded from the corrected directory if it exists before we fall back to its original name to make sure it loads the config from the correct location. Cache dir fixes and a improved routine for the path loaded models that mimics the NeoCustom option fixing models that have no model_type specified. Because GPT2 doesn't work well with this option and should exclusively be used with the GPT2Custom and GPT-J models should have a model_type we assume its a Neo model when not specified.
2021-12-23 14:40:35 +01:00
elif(vars.model_type == "not_found"):
Model Type support Automatically detect or assume the model type so we do not have to hardcode all the different models people might use. This almost makes the behavior of --model identical to the NeoCustom behavior as far as the CLI is concerned. But only if the model_type is defined in the models config file.
2021-12-23 02:50:06 +01:00
print("WARNING: No model type detected, assuming Neo (If this is a GPT2 model use the other menu option or --model GPT2Custom)")
vars.model_type = "gpt_neo"
NoBreakmodel variable Adds a Nobreakmodel var that allows Breakmodel to be turned off. This can be done trough commandline or a model config (In case Neo is used by the models config without it being a true Neo model that is compatible with breakmodel). In addition I removed the args.colab check for breakmodel support and instead make args.colab activate nobreakmodel. And I have added a new check so that breakmodel is not even attempted if you do not specify the layers but do launch a model from the command line.
2022-01-30 17:06:15 +01:00
loadmodelsettings()
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
print("{0}Looking for GPU support...{1}".format(colors.PURPLE, colors.END), end="")
vars.hascuda = torch.cuda.is_available()
NoBreakmodel variable Adds a Nobreakmodel var that allows Breakmodel to be turned off. This can be done trough commandline or a model config (In case Neo is used by the models config without it being a true Neo model that is compatible with breakmodel). In addition I removed the args.colab check for breakmodel support and instead make args.colab activate nobreakmodel. And I have added a new check so that breakmodel is not even attempted if you do not specify the layers but do launch a model from the command line.
2022-01-30 17:06:15 +01:00
vars.bmsupported = vars.model_type in ("gpt_neo", "gptj") and not vars.nobreakmodel
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if(args.breakmodel is not None and args.breakmodel):
Breakmodel Fixes Multiple old references and one mistake in my last commit fixed
2022-01-30 17:40:43 +01:00
print("WARNING: --breakmodel is no longer supported. Breakmodel mode is now automatically enabled when --breakmodel_gpulayers is used (see --help for details).", file=sys.stderr)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if(args.breakmodel_layers is not None):
Breakmodel Fixes Multiple old references and one mistake in my last commit fixed
2022-01-30 17:40:43 +01:00
print("WARNING: --breakmodel_layers is deprecated. Use --breakmodel_gpulayers instead (see --help for details).", file=sys.stderr)
if(args.model and vars.bmsupported and not args.breakmodel_gpulayers and not args.breakmodel_layers):
NoBreakmodel variable Adds a Nobreakmodel var that allows Breakmodel to be turned off. This can be done trough commandline or a model config (In case Neo is used by the models config without it being a true Neo model that is compatible with breakmodel). In addition I removed the args.colab check for breakmodel support and instead make args.colab activate nobreakmodel. And I have added a new check so that breakmodel is not even attempted if you do not specify the layers but do launch a model from the command line.
2022-01-30 17:06:15 +01:00
print("WARNING: Model launched without the --breakmodel_gpulayers argument, defaulting to GPU only mode.", file=sys.stderr)
--model without breakmodel disables bmsupported Last commit it only did a warning, now it will turn bmsupported off so that the GPU routine is used.
2022-01-30 17:16:35 +01:00
vars.bmsupported = False
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if(not vars.bmsupported and (args.breakmodel_gpulayers is not None or args.breakmodel_layers is not None)):
Breakmodel Fixes Multiple old references and one mistake in my last commit fixed
2022-01-30 17:40:43 +01:00
print("WARNING: This model does not support hybrid generation. --breakmodel_gpulayers will be ignored.", file=sys.stderr)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if(vars.hascuda):
print("{0}FOUND!{1}".format(colors.GREEN, colors.END))
else:
print("{0}NOT FOUND!{1}".format(colors.YELLOW, colors.END))
if args.model:
if(vars.hascuda):
genselected = True
vars.usegpu = True
vars.breakmodel = False
if(vars.bmsupported):
vars.usegpu = False
vars.breakmodel = True
if(args.cpu):
vars.usegpu = False
vars.breakmodel = False
elif(vars.hascuda):
if(vars.bmsupported):
Polishing and Optimizations Multiple things have changed, for now models default to half mode even on the official transformers to make sure its as efficient on the GPU as finetune's. GPU selection is streamlined and cache files are now stored inside the KoboldAI folder (for the most part). A new command line parameter to force the models to run at their full size still needs to be added for the few users that would want a quality bump at the cost of ram.
2021-11-18 00:06:57 +01:00
genselected = True
vars.usegpu = False
vars.breakmodel = True
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
else:
print(" 1 - GPU\n 2 - CPU\n")
Polishing and Optimizations Multiple things have changed, for now models default to half mode even on the official transformers to make sure its as efficient on the GPU as finetune's. GPU selection is streamlined and cache files are now stored inside the KoboldAI folder (for the most part). A new command line parameter to force the models to run at their full size still needs to be added for the few users that would want a quality bump at the cost of ram.
2021-11-18 00:06:57 +01:00
genselected = False
else:
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
genselected = False
if(vars.hascuda):
while(genselected == False):
genselect = input("Mode> ")
if(genselect == ""):
vars.breakmodel = False
vars.usegpu = True
genselected = True
elif(genselect.isnumeric() and int(genselect) == 1):
if(vars.bmsupported):
vars.breakmodel = True
vars.usegpu = False
genselected = True
else:
vars.breakmodel = False
vars.usegpu = True
genselected = True
elif(genselect.isnumeric() and int(genselect) == 2):
vars.breakmodel = False
vars.usegpu = False
genselected = True
else:
print("{0}Please enter a valid selection.{1}".format(colors.RED, colors.END))
# Ask for API key if InferKit was selected
if(vars.model == "InferKit"):
Replace slashes in model name with underscores
2021-11-17 21:36:36 +01:00
if(not path.exists("settings/" + getmodelname().replace('/', '_') + ".settings")):
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# If the client settings file doesn't exist, create it
print("{0}Please enter your InferKit API key:{1}\n".format(colors.CYAN, colors.END))
vars.apikey = input("Key> ")
# Write API key to file
os.makedirs('settings', exist_ok=True)
Replace slashes in model name with underscores
2021-11-17 21:36:36 +01:00
file = open("settings/" + getmodelname().replace('/', '_') + ".settings", "w")
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
try:
js = {"apikey": vars.apikey}
file.write(json.dumps(js, indent=3))
finally:
file.close()
else:
# Otherwise open it up
Replace slashes in model name with underscores
2021-11-17 21:36:36 +01:00
file = open("settings/" + getmodelname().replace('/', '_') + ".settings", "r")
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Check if API key exists
js = json.load(file)
if("apikey" in js and js["apikey"] != ""):
# API key exists, grab it and close the file
vars.apikey = js["apikey"]
file.close()
else:
# Get API key, add it to settings object, and write it to disk
print("{0}Please enter your InferKit API key:{1}\n".format(colors.CYAN, colors.END))
vars.apikey = input("Key> ")
js["apikey"] = vars.apikey
# Write API key to file
Replace slashes in model name with underscores
2021-11-17 21:36:36 +01:00
file = open("settings/" + getmodelname().replace('/', '_') + ".settings", "w")
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
try:
file.write(json.dumps(js, indent=3))
finally:
file.close()
# Ask for API key if OpenAI was selected
if(vars.model == "OAI"):
Replace slashes in model name with underscores
2021-11-17 21:36:36 +01:00
if(not path.exists("settings/" + getmodelname().replace('/', '_') + ".settings")):
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# If the client settings file doesn't exist, create it
print("{0}Please enter your OpenAI API key:{1}\n".format(colors.CYAN, colors.END))
vars.oaiapikey = input("Key> ")
# Write API key to file
os.makedirs('settings', exist_ok=True)
Replace slashes in model name with underscores
2021-11-17 21:36:36 +01:00
file = open("settings/" + getmodelname().replace('/', '_') + ".settings", "w")
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
try:
js = {"oaiapikey": vars.oaiapikey}
file.write(json.dumps(js, indent=3))
finally:
file.close()
else:
# Otherwise open it up
Replace slashes in model name with underscores
2021-11-17 21:36:36 +01:00
file = open("settings/" + getmodelname().replace('/', '_') + ".settings", "r")
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Check if API key exists
js = json.load(file)
if("oaiapikey" in js and js["oaiapikey"] != ""):
# API key exists, grab it and close the file
vars.oaiapikey = js["oaiapikey"]
file.close()
else:
# Get API key, add it to settings object, and write it to disk
print("{0}Please enter your OpenAI API key:{1}\n".format(colors.CYAN, colors.END))
vars.oaiapikey = input("Key> ")
js["oaiapikey"] = vars.oaiapikey
# Write API key to file
Replace slashes in model name with underscores
2021-11-17 21:36:36 +01:00
file = open("settings/" + getmodelname().replace('/', '_') + ".settings", "w")
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
try:
file.write(json.dumps(js, indent=3))
finally:
file.close()
# Get list of models from OAI
print("{0}Retrieving engine list...{1}".format(colors.PURPLE, colors.END), end="")
req = requests.get(
vars.oaiengines,
headers = {
'Authorization': 'Bearer '+vars.oaiapikey
}
)
if(req.status_code == 200):
print("{0}OK!{1}".format(colors.GREEN, colors.END))
print("{0}Please select an engine to use:{1}\n".format(colors.CYAN, colors.END))
engines = req.json()["data"]
# Print list of engines
i = 0
for en in engines:
print(" {0} - {1} ({2})".format(i, en["id"], "\033[92mready\033[0m" if en["ready"] == True else "\033[91mnot ready\033[0m"))
i += 1
# Get engine to use
print("")
engselected = False
while(engselected == False):
engine = input("Engine #> ")
if(engine.isnumeric() and int(engine) < len(engines)):
vars.oaiurl = "https://api.openai.com/v1/engines/{0}/completions".format(engines[int(engine)]["id"])
engselected = True
else:
print("{0}Please enter a valid selection.{1}".format(colors.RED, colors.END))
else:
# Something went wrong, print the message and quit since we can't initialize an engine
print("{0}ERROR!{1}".format(colors.RED, colors.END))
print(req.json())
quit()
# Ask for ngrok url if Google Colab was selected
if(vars.model == "Colab"):
if(vars.colaburl == ""):
Server clarification To prevent confusion with users who have not used KoboldAI for a while, or who are following old tutorials I have added a disclaimer that informs people that most Colab links should not be used with this feature and instead opened in the browser.
2022-01-11 00:35:20 +01:00
print("{0}NOTE: For the modern KoboldAI Colab's you open the links directly in your browser.\nThis option is only for the KoboldAI Server API, not all features are supported in this mode.\n".format(colors.YELLOW, colors.END))
print("{0}Enter the URL of the server (For example a trycloudflare link):{1}\n".format(colors.CYAN, colors.END))
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
vars.colaburl = input("URL> ") + "/request"
if(vars.model == "ReadOnly"):
vars.noai = True
# Set logging level to reduce chatter from Flask
import logging
log = logging.getLogger('werkzeug')
log.setLevel(logging.ERROR)
# Start flask & SocketIO
print("{0}Initializing Flask... {1}".format(colors.PURPLE, colors.END), end="")
Show message when TPU backend is compiling
2022-01-17 03:09:10 +01:00
from flask import Flask, render_template, Response, request, copy_current_request_context
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
from flask_socketio import SocketIO, emit
app = Flask(__name__)
app.config['SECRET KEY'] = 'secret!'
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
socketio = SocketIO(app, async_method="eventlet")
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
print("{0}OK!{1}".format(colors.GREEN, colors.END))
# Start transformers and create pipeline
Integrate TPU backend This commit puts the TPU backend code directly in to the KoboldAI code to make it easier to modify.
2021-11-20 00:06:57 +01:00
if(not vars.model in ["InferKit", "Colab", "OAI", "ReadOnly", "TPUMeshTransformerGPTJ"]):
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if(not vars.noai):
print("{0}Initializing transformers, please wait...{1}".format(colors.PURPLE, colors.END))
AutoTokenizer Futureproofing for future tokenizers, for now this is not needed since everything uses GPT2. But when that changes we want to be prepared. Not all models have a proper tokenizer config, so if we can't find one we fall back to GPT2.
2021-12-25 00:44:26 +01:00
from transformers import StoppingCriteria, GPT2TokenizerFast, GPT2LMHeadModel, GPTNeoForCausalLM, GPTNeoModel, AutoModelForCausalLM, AutoTokenizer
Remember to actually import `GPTJModel`
2021-11-26 18:38:52 +01:00
try:
from transformers import GPTJModel
except:
pass
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
import transformers.generation_utils
Enable `low_cpu_mem_usage`
2021-11-26 00:09:25 +01:00
from transformers import __version__ as transformers_version
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Patch transformers to use our soft prompt
def patch_causallm(cls):
old_forward = cls.forward
def new_causallm_forward(self, *args, **kwargs):
input_ids = kwargs.get('input_ids').to(self.device)
assert input_ids is not None
kwargs['input_ids'] = None
if(vars.sp is not None):
shifted_input_ids = input_ids - self.config.vocab_size
input_ids.clamp_(max=self.config.vocab_size-1)
inputs_embeds = self.transformer.wte(input_ids)
if(vars.sp is not None):
vars.sp = vars.sp.to(inputs_embeds.dtype).to(inputs_embeds.device)
inputs_embeds = torch.where(
(shifted_input_ids >= 0)[..., None],
vars.sp[shifted_input_ids.clamp(min=0)],
inputs_embeds,
)
kwargs['inputs_embeds'] = inputs_embeds
return old_forward(self, *args, **kwargs)
cls.forward = new_causallm_forward
for cls in (GPT2LMHeadModel, GPTNeoForCausalLM):
patch_causallm(cls)
try:
from transformers import GPTJForCausalLM
patch_causallm(GPTJForCausalLM)
except:
pass
Move TFS warper code into aiserver.py
2021-11-24 19:36:54 +01:00
# Patch transformers to use our custom logit warpers
Add individual configuration files for userscripts
2021-12-22 08:59:31 +01:00
from transformers import LogitsProcessorList, LogitsWarper, LogitsProcessor, TopKLogitsWarper, TopPLogitsWarper, TemperatureLogitsWarper, RepetitionPenaltyLogitsProcessor
Repetition penalty slope and range
2022-01-24 21:30:38 +01:00
from warpers import AdvancedRepetitionPenaltyLogitsProcessor, TailFreeLogitsWarper
Make transformers warpers dynamically update their parameters So that if you change, e.g., `top_p`, from a Lua generation modifier or from the settings menu during generation, the rest of the generation will use the new setting value instead of retaining the settings it had when generation began.
2021-12-22 04:12:24 +01:00
Make `dynamic_processor_wrap` execute warper conditionally The top-k warper doesn't work properly with an argument of 0, so there is now the ability to not execute the warper if a condition is not met.
2021-12-23 05:46:25 +01:00
def dynamic_processor_wrap(cls, field_name, var_name, cond=None):
Make transformers warpers dynamically update their parameters So that if you change, e.g., `top_p`, from a Lua generation modifier or from the settings menu during generation, the rest of the generation will use the new setting value instead of retaining the settings it had when generation began.
2021-12-22 04:12:24 +01:00
old_call = cls.__call__
def new_call(self, *args, **kwargs):
Repetition penalty slope and range
2022-01-24 21:30:38 +01:00
if(not isinstance(field_name, str) and isinstance(field_name, Iterable)):
conds = []
for f, v in zip(field_name, var_name):
conds.append(getattr(vars, v))
setattr(self, f, conds[-1])
else:
conds = getattr(vars, var_name)
setattr(self, field_name, conds)
Make `dynamic_processor_wrap` execute warper conditionally The top-k warper doesn't work properly with an argument of 0, so there is now the ability to not execute the warper if a condition is not met.
2021-12-23 05:46:25 +01:00
assert len(args) == 2
Repetition penalty slope and range
2022-01-24 21:30:38 +01:00
if(cond is None or cond(conds)):
Make `dynamic_processor_wrap` execute warper conditionally The top-k warper doesn't work properly with an argument of 0, so there is now the ability to not execute the warper if a condition is not met.
2021-12-23 05:46:25 +01:00
return old_call(self, *args, **kwargs)
return args[1]
Make transformers warpers dynamically update their parameters So that if you change, e.g., `top_p`, from a Lua generation modifier or from the settings menu during generation, the rest of the generation will use the new setting value instead of retaining the settings it had when generation began.
2021-12-22 04:12:24 +01:00
cls.__call__ = new_call
Repetition penalty slope and range
2022-01-24 21:30:38 +01:00
dynamic_processor_wrap(AdvancedRepetitionPenaltyLogitsProcessor, ("penalty", "penalty_slope", "penalty_range"), ("rep_pen", "rep_pen_slope", "rep_pen_range"), cond=lambda x: x[0] != 1.0)
Make `dynamic_processor_wrap` execute warper conditionally The top-k warper doesn't work properly with an argument of 0, so there is now the ability to not execute the warper if a condition is not met.
2021-12-23 05:46:25 +01:00
dynamic_processor_wrap(TopKLogitsWarper, "top_k", "top_k", cond=lambda x: x > 0)
dynamic_processor_wrap(TopPLogitsWarper, "top_p", "top_p", cond=lambda x: x < 1.0)
Repetition penalty slope and range
2022-01-24 21:30:38 +01:00
dynamic_processor_wrap(TailFreeLogitsWarper, "tfs", "tfs", cond=lambda x: x < 1.0)
Make `dynamic_processor_wrap` execute warper conditionally The top-k warper doesn't work properly with an argument of 0, so there is now the ability to not execute the warper if a condition is not met.
2021-12-23 05:46:25 +01:00
dynamic_processor_wrap(TemperatureLogitsWarper, "temperature", "temp", cond=lambda x: x != 1.0)
Repetition penalty slope and range
2022-01-24 21:30:38 +01:00
RepetitionPenaltyLogitsProcessor.__init__ = AdvancedRepetitionPenaltyLogitsProcessor.__init__
RepetitionPenaltyLogitsProcessor.__call__ = AdvancedRepetitionPenaltyLogitsProcessor.__call__
Make transformers warpers dynamically update their parameters So that if you change, e.g., `top_p`, from a Lua generation modifier or from the settings menu during generation, the rest of the generation will use the new setting value instead of retaining the settings it had when generation began.
2021-12-22 04:12:24 +01:00
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
class LuaLogitsProcessor(LogitsProcessor):
Add Lua API for editing logits during generation TPU backend not supported yet.
2021-12-12 22:18:45 +01:00
def __init__(self):
pass
def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor) -> torch.FloatTensor:
assert scores.ndim == 2
Add API for generated tokens and output text
2021-12-13 01:27:20 +01:00
assert input_ids.ndim == 2
Add Lua API for editing logits during generation TPU backend not supported yet.
2021-12-12 22:18:45 +01:00
self.regeneration_required = False
self.halt = False
Add API for generated tokens and output text
2021-12-13 01:27:20 +01:00
Add Lua API for editing logits during generation TPU backend not supported yet.
2021-12-12 22:18:45 +01:00
scores_shape = scores.shape
Change `self.scores` to `scores` in aiserver.py
2021-12-13 17:18:01 +01:00
scores_list = scores.tolist()
Add Lua API for editing logits during generation TPU backend not supported yet.
2021-12-12 22:18:45 +01:00
vars.lua_koboldbridge.logits = vars.lua_state.table()
for r, row in enumerate(scores_list):
vars.lua_koboldbridge.logits[r+1] = vars.lua_state.table(*row)
vars.lua_koboldbridge.vocab_size = scores_shape[-1]
Add API for generated tokens and output text
2021-12-13 01:27:20 +01:00
Add Lua API for editing logits during generation TPU backend not supported yet.
2021-12-12 22:18:45 +01:00
execute_genmod()
Add API for generated tokens and output text
2021-12-13 01:27:20 +01:00
Add Lua API for editing logits during generation TPU backend not supported yet.
2021-12-12 22:18:45 +01:00
scores = torch.tensor(
tuple(tuple(row.values()) for row in vars.lua_koboldbridge.logits.values()),
device=scores.device,
dtype=scores.dtype,
)
assert scores.shape == scores_shape
Add API for generated tokens and output text
2021-12-13 01:27:20 +01:00
Add Lua API for editing logits during generation TPU backend not supported yet.
2021-12-12 22:18:45 +01:00
return scores
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
def new_get_logits_processor(*args, **kwargs) -> LogitsProcessorList:
processors = new_get_logits_processor.old_get_logits_processor(*args, **kwargs)
processors.insert(0, LuaLogitsProcessor())
return processors
new_get_logits_processor.old_get_logits_processor = transformers.generation_utils.GenerationMixin._get_logits_processor
transformers.generation_utils.GenerationMixin._get_logits_processor = new_get_logits_processor
Move TFS warper code into aiserver.py
2021-11-24 19:36:54 +01:00
Make `dynamic_processor_wrap` execute warper conditionally The top-k warper doesn't work properly with an argument of 0, so there is now the ability to not execute the warper if a condition is not met.
2021-12-23 05:46:25 +01:00
def new_get_logits_warper(beams: int = 1,) -> LogitsProcessorList:
Move TFS warper code into aiserver.py
2021-11-24 19:36:54 +01:00
warper_list = LogitsProcessorList()
Make `dynamic_processor_wrap` execute warper conditionally The top-k warper doesn't work properly with an argument of 0, so there is now the ability to not execute the warper if a condition is not met.
2021-12-23 05:46:25 +01:00
warper_list.append(TopKLogitsWarper(top_k=1, min_tokens_to_keep=1 + (beams > 1)))
warper_list.append(TopPLogitsWarper(top_p=0.5, min_tokens_to_keep=1 + (beams > 1)))
warper_list.append(TailFreeLogitsWarper(tfs=0.5, min_tokens_to_keep=1 + (beams > 1)))
warper_list.append(TemperatureLogitsWarper(temperature=0.5))
Move TFS warper code into aiserver.py
2021-11-24 19:36:54 +01:00
return warper_list
def new_sample(self, *args, **kwargs):
assert kwargs.pop("logits_warper", None) is not None
kwargs["logits_warper"] = new_get_logits_warper(
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
beams=1,
Move TFS warper code into aiserver.py
2021-11-24 19:36:54 +01:00
)
return new_sample.old_sample(self, *args, **kwargs)
new_sample.old_sample = transformers.generation_utils.GenerationMixin.sample
transformers.generation_utils.GenerationMixin.sample = new_sample
Allow bad words filter to ban <|endoftext|> token The official transformers bad words filter doesn't allow this by default. Finetune's version does allow this by default, however.
2021-11-27 17:40:34 +01:00
# Allow bad words filter to ban <|endoftext|> token
import transformers.generation_logits_process
def new_init(self, bad_words_ids: List[List[int]], eos_token_id: int):
return new_init.old_init(self, bad_words_ids, -1)
new_init.old_init = transformers.generation_logits_process.NoBadWordsLogitsProcessor.__init__
transformers.generation_logits_process.NoBadWordsLogitsProcessor.__init__ = new_init
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Sets up dynamic world info scanner
class DynamicWorldInfoScanCriteria(StoppingCriteria):
def __init__(
self,
tokenizer,
Support for multiple gens per action with dynamic scan
2021-11-17 22:17:59 +01:00
excluded_world_info: List[Set],
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
):
Enable generation modifiers for transformers backend only
2021-12-11 22:28:25 +01:00
self.regeneration_required = False
Complete the Lua generation halting API
2021-12-12 18:52:03 +01:00
self.halt = False
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
self.tokenizer = tokenizer
self.excluded_world_info = excluded_world_info
def __call__(
self,
input_ids: torch.LongTensor,
scores: torch.FloatTensor,
**kwargs,
) -> bool:
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
vars.generated_tkns += 1
if(vars.lua_koboldbridge.generated_cols and vars.generated_tkns != vars.lua_koboldbridge.generated_cols):
raise RuntimeError(f"Inconsistency detected between KoboldAI Python and Lua backends ({vars.generated_tkns} != {vars.lua_koboldbridge.generated_cols})")
Allow pressing send button again to stop generation
2022-01-10 22:36:15 +01:00
if(vars.abort or vars.generated_tkns >= vars.genamt):
Prevent dynamic scanning from generating too many tokens
2021-12-15 05:39:04 +01:00
self.regeneration_required = False
self.halt = False
return True
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
assert input_ids.ndim == 2
Support for multiple gens per action with dynamic scan
2021-11-17 22:17:59 +01:00
assert len(self.excluded_world_info) == input_ids.shape[0]
Fix Lua regeneration system
2021-12-14 01:17:18 +01:00
self.regeneration_required = vars.lua_koboldbridge.regeneration_required
self.halt = not vars.lua_koboldbridge.generating
vars.lua_koboldbridge.regeneration_required = False
for i in range(vars.numseqs):
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
vars.lua_koboldbridge.generated[i+1][vars.generated_tkns] = int(input_ids[i, -1].item())
Enable generation modifiers for transformers backend only
2021-12-11 22:28:25 +01:00
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if(not vars.dynamicscan):
Make sure stopping criteria still work with dynamic scan off
2021-12-14 00:10:51 +01:00
return self.regeneration_required or self.halt
Fix a bug with dynamic WI scan when using a soft prompt The problem was that when a soft prompt is being used, the dynamic scanning criteria searches a different set of tokens for world info keys than the `_generate()` function, which results in generation loops when a world info key appears in the former set of tokens but not the latter.
2022-01-10 21:52:49 +01:00
tail = input_ids[..., -vars.generated_tkns:]
Support for multiple gens per action with dynamic scan
2021-11-17 22:17:59 +01:00
for i, t in enumerate(tail):
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
decoded = tokenizer.decode(t)
Fix a bug with dynamic WI scan when using a soft prompt The problem was that when a soft prompt is being used, the dynamic scanning criteria searches a different set of tokens for world info keys than the `_generate()` function, which results in generation loops when a world info key appears in the former set of tokens but not the latter.
2022-01-10 21:52:49 +01:00
_, found = checkworldinfo(decoded, force_use_txt=True, actions=vars._actions)
Support for multiple gens per action with dynamic scan
2021-11-17 22:17:59 +01:00
found -= self.excluded_world_info[i]
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if(len(found) != 0):
Enable generation modifiers for transformers backend only
2021-12-11 22:28:25 +01:00
self.regeneration_required = True
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
break
Complete the Lua generation halting API
2021-12-12 18:52:03 +01:00
return self.regeneration_required or self.halt
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
old_get_stopping_criteria = transformers.generation_utils.GenerationMixin._get_stopping_criteria
def new_get_stopping_criteria(self, *args, **kwargs):
stopping_criteria = old_get_stopping_criteria(self, *args, **kwargs)
global tokenizer
self.kai_scanner = DynamicWorldInfoScanCriteria(
tokenizer=tokenizer,
excluded_world_info=self.kai_scanner_excluded_world_info,
)
Prevent dynamic scanning from generating too many tokens
2021-12-15 05:39:04 +01:00
stopping_criteria.insert(0, self.kai_scanner)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
return stopping_criteria
transformers.generation_utils.GenerationMixin._get_stopping_criteria = new_get_stopping_criteria
def get_hidden_size_from_model(model):
try:
return int(model.transformer.hidden_size)
except:
Fix hidden size detection for GPTJForCausalLM
2021-11-15 17:56:02 +01:00
try:
return int(model.transformer.embed_dim)
except:
return int(model.lm_head.in_features)
Enable `low_cpu_mem_usage`
2021-11-26 00:09:25 +01:00
def maybe_low_cpu_mem_usage() -> Dict[str, Any]:
if(packaging.version.parse(transformers_version) < packaging.version.parse("4.11.0")):
print(f"\nWARNING: Please upgrade to transformers 4.11.0 for lower RAM usage. You have transformers {transformers_version}.", file=sys.stderr)
return {}
return {"low_cpu_mem_usage": True}
Load model directly in fp16 if using GPU or breakmodel
2021-11-26 16:55:52 +01:00
@contextlib.contextmanager
def maybe_use_float16(always_use=False):
if(always_use or (vars.hascuda and (vars.usegpu or vars.breakmodel))):
original_dtype = torch.get_default_dtype()
torch.set_default_dtype(torch.float16)
yield True
torch.set_default_dtype(original_dtype)
else:
yield False
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# If custom GPT2 model was chosen
Replace NeoCustom NeoCustom is now obsolete beyond the file selection and the CLI. So after the CLI we adapt the input to a generic model and then use the improved generic routine to handle it. This saves duplicate efforts of maintaining an almost identical routine now that models are handled by their type and not their name.
2021-12-23 17:10:02 +01:00
if(vars.model == "GPT2Custom"):
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
model_config = open(vars.custmodpth + "/config.json", "r")
js = json.load(model_config)
Load model directly in fp16 if using GPU or breakmodel
2021-11-26 16:55:52 +01:00
with(maybe_use_float16()):
Disable `low_cpu_mem_usage` when using GPT-2 Attempting to use transformers 4.11.0's experimental `low_cpu_mem_usage` feature with GPT-2 models usually results in the output repeating a token over and over or otherwise containing an incoherent response.
2021-12-21 01:54:19 +01:00
model = GPT2LMHeadModel.from_pretrained(vars.custmodpth, cache_dir="cache/")
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
tokenizer = GPT2TokenizerFast.from_pretrained(vars.custmodpth, cache_dir="cache/")
Hidden Size fix for GPT2 Custom Replaced the JS Hidden Size load with the newer function to fix these models
2021-11-14 16:40:04 +01:00
vars.modeldim = get_hidden_size_from_model(model)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Is CUDA available? If so, use GPU, otherwise fall back to CPU
if(vars.hascuda and vars.usegpu):
Fix an error that occurs when all layers are on second GPU
2021-12-16 01:03:49 +01:00
model = model.half().to(vars.gpu_device)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
generator = model.generate
else:
Fix float16 models Forcefully convert float16 models to work on the CPU
2021-12-16 00:31:51 +01:00
model = model.to('cpu').float()
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
generator = model.generate
Replace NeoCustom NeoCustom is now obsolete beyond the file selection and the CLI. So after the CLI we adapt the input to a generic model and then use the improved generic routine to handle it. This saves duplicate efforts of maintaining an almost identical routine now that models are handled by their type and not their name.
2021-12-23 17:10:02 +01:00
# Use the Generic implementation
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
else:
Disable `low_cpu_mem_usage` when using GPT-2 Attempting to use transformers 4.11.0's experimental `low_cpu_mem_usage` feature with GPT-2 models usually results in the output repeating a token over and over or otherwise containing an incoherent response.
2021-12-21 01:54:19 +01:00
lowmem = maybe_low_cpu_mem_usage()
# We must disable low_cpu_mem_usage (by setting lowmem to {}) if
# using a GPT-2 model because GPT-2 is not compatible with this
# feature yet
Use `vars.model_type` to check for GPT-2 models
2022-01-17 19:13:54 +01:00
if(vars.model_type == "gpt2"):
Disable `low_cpu_mem_usage` when using GPT-2 Attempting to use transformers 4.11.0's experimental `low_cpu_mem_usage` feature with GPT-2 models usually results in the output repeating a token over and over or otherwise containing an incoherent response.
2021-12-21 01:54:19 +01:00
lowmem = {}
Use `vars.model_type` to check for GPT-2 models
2022-01-17 19:13:54 +01:00
Compatibility Fixes Rather than coding a vars.custmodpath or vars.model in all the other parts of the code I opted to just set vars.custmodpath instead to make the behavior more consistent now that it always loads from the same location.
2021-12-23 00:36:08 +01:00
# Download model from Huggingface if it does not exist, otherwise load locally
Replace NeoCustom NeoCustom is now obsolete beyond the file selection and the CLI. So after the CLI we adapt the input to a generic model and then use the improved generic routine to handle it. This saves duplicate efforts of maintaining an almost identical routine now that models are handled by their type and not their name.
2021-12-23 17:10:02 +01:00
if(os.path.isdir(vars.custmodpth)):
with(maybe_use_float16()):
AutoTokenizer Futureproofing for future tokenizers, for now this is not needed since everything uses GPT2. But when that changes we want to be prepared. Not all models have a proper tokenizer config, so if we can't find one we fall back to GPT2.
2021-12-25 00:44:26 +01:00
try:
tokenizer = AutoTokenizer.from_pretrained(vars.custmodpth, cache_dir="cache/")
except ValueError as e:
tokenizer = GPT2TokenizerFast.from_pretrained(vars.custmodpth, cache_dir="cache/")
Replace NeoCustom NeoCustom is now obsolete beyond the file selection and the CLI. So after the CLI we adapt the input to a generic model and then use the improved generic routine to handle it. This saves duplicate efforts of maintaining an almost identical routine now that models are handled by their type and not their name.
2021-12-23 17:10:02 +01:00
try:
Restore Lowmem Accidentally got replaced in one of my test runs
2021-12-23 18:50:01 +01:00
model = AutoModelForCausalLM.from_pretrained(vars.custmodpth, cache_dir="cache/", **lowmem)
Replace NeoCustom NeoCustom is now obsolete beyond the file selection and the CLI. So after the CLI we adapt the input to a generic model and then use the improved generic routine to handle it. This saves duplicate efforts of maintaining an almost identical routine now that models are handled by their type and not their name.
2021-12-23 17:10:02 +01:00
except ValueError as e:
Restore Lowmem Accidentally got replaced in one of my test runs
2021-12-23 18:50:01 +01:00
model = GPTNeoForCausalLM.from_pretrained(vars.custmodpth, cache_dir="cache/", **lowmem)
Replace NeoCustom NeoCustom is now obsolete beyond the file selection and the CLI. So after the CLI we adapt the input to a generic model and then use the improved generic routine to handle it. This saves duplicate efforts of maintaining an almost identical routine now that models are handled by their type and not their name.
2021-12-23 17:10:02 +01:00
elif(os.path.isdir(vars.model.replace('/', '_'))):
New download machanism Automatically converts Huggingface cache models to full models on (down)load. WARNING: Does wipe old cache/ dir inside the KoboldAI folder, make a backup before you run these models if you are bandwith constraint.
2021-12-16 01:40:04 +01:00
with(maybe_use_float16()):
AutoTokenizer Futureproofing for future tokenizers, for now this is not needed since everything uses GPT2. But when that changes we want to be prepared. Not all models have a proper tokenizer config, so if we can't find one we fall back to GPT2.
2021-12-25 00:44:26 +01:00
try:
Path Fixes Fixes the tokenizer cache being hit when we already have a local model
2021-12-27 01:56:59 +01:00
tokenizer = AutoTokenizer.from_pretrained(vars.model.replace('/', '_'), cache_dir="cache/")
AutoTokenizer Futureproofing for future tokenizers, for now this is not needed since everything uses GPT2. But when that changes we want to be prepared. Not all models have a proper tokenizer config, so if we can't find one we fall back to GPT2.
2021-12-25 00:44:26 +01:00
except ValueError as e:
Path Fixes Fixes the tokenizer cache being hit when we already have a local model
2021-12-27 01:56:59 +01:00
tokenizer = GPT2TokenizerFast.from_pretrained(vars.model.replace('/', '_'), cache_dir="cache/")
Path loading improvements This fixes a few scenario's of my commit yesterday, models that have a / are now first loaded from the corrected directory if it exists before we fall back to its original name to make sure it loads the config from the correct location. Cache dir fixes and a improved routine for the path loaded models that mimics the NeoCustom option fixing models that have no model_type specified. Because GPT2 doesn't work well with this option and should exclusively be used with the GPT2Custom and GPT-J models should have a model_type we assume its a Neo model when not specified.
2021-12-23 14:40:35 +01:00
try:
Restore Lowmem Accidentally got replaced in one of my test runs
2021-12-23 18:50:01 +01:00
model = AutoModelForCausalLM.from_pretrained(vars.model.replace('/', '_'), cache_dir="cache/", **lowmem)
Path loading improvements This fixes a few scenario's of my commit yesterday, models that have a / are now first loaded from the corrected directory if it exists before we fall back to its original name to make sure it loads the config from the correct location. Cache dir fixes and a improved routine for the path loaded models that mimics the NeoCustom option fixing models that have no model_type specified. Because GPT2 doesn't work well with this option and should exclusively be used with the GPT2Custom and GPT-J models should have a model_type we assume its a Neo model when not specified.
2021-12-23 14:40:35 +01:00
except ValueError as e:
Restore Lowmem Accidentally got replaced in one of my test runs
2021-12-23 18:50:01 +01:00
model = GPTNeoForCausalLM.from_pretrained(vars.model.replace('/', '_'), cache_dir="cache/", **lowmem)
New download machanism Automatically converts Huggingface cache models to full models on (down)load. WARNING: Does wipe old cache/ dir inside the KoboldAI folder, make a backup before you run these models if you are bandwith constraint.
2021-12-16 01:40:04 +01:00
else:
AutoTokenizer Futureproofing for future tokenizers, for now this is not needed since everything uses GPT2. But when that changes we want to be prepared. Not all models have a proper tokenizer config, so if we can't find one we fall back to GPT2.
2021-12-25 00:44:26 +01:00
try:
Path Fixes Fixes the tokenizer cache being hit when we already have a local model
2021-12-27 01:56:59 +01:00
tokenizer = AutoTokenizer.from_pretrained(vars.model, cache_dir="cache/")
AutoTokenizer Futureproofing for future tokenizers, for now this is not needed since everything uses GPT2. But when that changes we want to be prepared. Not all models have a proper tokenizer config, so if we can't find one we fall back to GPT2.
2021-12-25 00:44:26 +01:00
except ValueError as e:
Path Fixes Fixes the tokenizer cache being hit when we already have a local model
2021-12-27 01:56:59 +01:00
tokenizer = GPT2TokenizerFast.from_pretrained(vars.model, cache_dir="cache/")
New download machanism Automatically converts Huggingface cache models to full models on (down)load. WARNING: Does wipe old cache/ dir inside the KoboldAI folder, make a backup before you run these models if you are bandwith constraint.
2021-12-16 01:40:04 +01:00
with(maybe_use_float16()):
Replace NeoCustom NeoCustom is now obsolete beyond the file selection and the CLI. So after the CLI we adapt the input to a generic model and then use the improved generic routine to handle it. This saves duplicate efforts of maintaining an almost identical routine now that models are handled by their type and not their name.
2021-12-23 17:10:02 +01:00
try:
Restore Lowmem Accidentally got replaced in one of my test runs
2021-12-23 18:50:01 +01:00
model = AutoModelForCausalLM.from_pretrained(vars.model, cache_dir="cache/", **lowmem)
Replace NeoCustom NeoCustom is now obsolete beyond the file selection and the CLI. So after the CLI we adapt the input to a generic model and then use the improved generic routine to handle it. This saves duplicate efforts of maintaining an almost identical routine now that models are handled by their type and not their name.
2021-12-23 17:10:02 +01:00
except ValueError as e:
Restore Lowmem Accidentally got replaced in one of my test runs
2021-12-23 18:50:01 +01:00
model = GPTNeoForCausalLM.from_pretrained(vars.model, cache_dir="cache/", **lowmem)
Colab Optimizations Breakmodel is useless on Colab, so for the sake of efficiency if --colab is present we will always assume a model is incompatible. The same applies to the conversion, colab's are discarded so converting the model to a .bin file only wastes time since the HDD isn't fast. Finally we automatically set all the useful variables for Colab, so that in the future this can be removed from ckds and other scripts. Lastly ckds has been adapted not to copy the examples folder and to add the new --colab parameter. Local players are much better off running the old --remote command.
2022-01-16 00:56:03 +01:00
if not args.colab:
model = model.half()
import shutil
shutil.rmtree("cache/")
model.save_pretrained(vars.model.replace('/', '_'))
tokenizer.save_pretrained(vars.model.replace('/', '_'))
Model Type support Automatically detect or assume the model type so we do not have to hardcode all the different models people might use. This almost makes the behavior of --model identical to the NeoCustom behavior as far as the CLI is concerned. But only if the model_type is defined in the models config file.
2021-12-23 02:50:06 +01:00
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if(vars.hascuda):
if(vars.usegpu):
vars.modeldim = get_hidden_size_from_model(model)
Fix an error that occurs when all layers are on second GPU
2021-12-16 01:03:49 +01:00
model = model.half().to(vars.gpu_device)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
generator = model.generate
elif(vars.breakmodel): # Use both RAM and VRAM (breakmodel)
vars.modeldim = get_hidden_size_from_model(model)
device_config(model)
else:
New download machanism Automatically converts Huggingface cache models to full models on (down)load. WARNING: Does wipe old cache/ dir inside the KoboldAI folder, make a backup before you run these models if you are bandwith constraint.
2021-12-16 01:40:04 +01:00
model = model.to('cpu').float()
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
vars.modeldim = get_hidden_size_from_model(model)
generator = model.generate
else:
New download machanism Automatically converts Huggingface cache models to full models on (down)load. WARNING: Does wipe old cache/ dir inside the KoboldAI folder, make a backup before you run these models if you are bandwith constraint.
2021-12-16 01:40:04 +01:00
model.to('cpu').float()
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
vars.modeldim = get_hidden_size_from_model(model)
generator = model.generate
# Suppress Author's Note by flagging square brackets (Old implementation)
#vocab = tokenizer.get_vocab()
#vocab_keys = vocab.keys()
#vars.badwords = gettokenids("[")
#for key in vars.badwords:
# vars.badwordsids.append([vocab[key]])
print("{0}OK! {1} pipeline created!{2}".format(colors.GREEN, vars.model, colors.END))
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
else:
from transformers import GPT2TokenizerFast
Path loading improvements This fixes a few scenario's of my commit yesterday, models that have a / are now first loaded from the corrected directory if it exists before we fall back to its original name to make sure it loads the config from the correct location. Cache dir fixes and a improved routine for the path loaded models that mimics the NeoCustom option fixing models that have no model_type specified. Because GPT2 doesn't work well with this option and should exclusively be used with the GPT2Custom and GPT-J models should have a model_type we assume its a Neo model when not specified.
2021-12-23 14:40:35 +01:00
tokenizer = GPT2TokenizerFast.from_pretrained("gpt2", cache_dir="cache/")
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
else:
Compile TPU backend in background
2022-01-07 19:47:21 +01:00
def tpumtjgetsofttokens():
soft_tokens = None
if(vars.sp is None):
global np
if 'np' not in globals():
import numpy as np
tensor = np.zeros((1, tpu_mtj_backend.params["d_model"]), dtype=np.float32)
rows = tensor.shape[0]
padding_amount = tpu_mtj_backend.params["seq"] - (tpu_mtj_backend.params["seq"] % -tpu_mtj_backend.params["cores_per_replica"]) - rows
tensor = np.pad(tensor, ((0, padding_amount), (0, 0)))
tensor = tensor.reshape(
tpu_mtj_backend.params["cores_per_replica"],
-1,
tpu_mtj_backend.params["d_model"],
)
xmaps for moving things onto TPU
2022-01-13 03:45:30 +01:00
vars.sp = tpu_mtj_backend.shard_xmap(tensor)
Compile TPU backend in background
2022-01-07 19:47:21 +01:00
soft_tokens = np.arange(
tpu_mtj_backend.params["n_vocab"] + tpu_mtj_backend.params["n_vocab_padding"],
tpu_mtj_backend.params["n_vocab"] + tpu_mtj_backend.params["n_vocab_padding"] + vars.sp_length,
dtype=np.uint32
)
return soft_tokens
Fix TPU generation modifier
2022-01-15 05:00:06 +01:00
def tpumtjgenerate_warper_callback(scores) -> "np.array":
Add TPU support for dynamic WI scan and generation modifiers
2022-01-15 03:39:02 +01:00
scores_shape = scores.shape
scores_list = scores.tolist()
vars.lua_koboldbridge.logits = vars.lua_state.table()
for r, row in enumerate(scores_list):
vars.lua_koboldbridge.logits[r+1] = vars.lua_state.table(*row)
vars.lua_koboldbridge.vocab_size = scores_shape[-1]
execute_genmod()
scores = np.array(
tuple(tuple(row.values()) for row in vars.lua_koboldbridge.logits.values()),
dtype=scores.dtype,
)
assert scores.shape == scores_shape
Fix TPU generation modifier
2022-01-15 05:00:06 +01:00
return scores
def tpumtjgenerate_stopping_callback(generated, n_generated, excluded_world_info) -> Tuple[List[set], bool, bool]:
vars.generated_tkns += 1
assert len(excluded_world_info) == len(generated)
regeneration_required = vars.lua_koboldbridge.regeneration_required
Dynamic TPU backend should support dynamic warpers and abort button
2022-01-17 20:10:32 +01:00
halt = vars.abort or not vars.lua_koboldbridge.generating or vars.generated_tkns >= vars.genamt
Fix TPU generation modifier
2022-01-15 05:00:06 +01:00
vars.lua_koboldbridge.regeneration_required = False
global past
for i in range(vars.numseqs):
vars.lua_koboldbridge.generated[i+1][vars.generated_tkns] = int(generated[i, tpu_mtj_backend.params["seq"] + n_generated - 1].item())
Add TPU support for dynamic WI scan and generation modifiers
2022-01-15 03:39:02 +01:00
if(not vars.dynamicscan or halt):
return excluded_world_info, regeneration_required, halt
for i, t in enumerate(generated):
decoded = tokenizer.decode(past[i]) + tokenizer.decode(t[tpu_mtj_backend.params["seq"] : tpu_mtj_backend.params["seq"] + n_generated])
Use `vars._actions` in `tpumtjgenerate` and its callbacks
2022-01-17 19:24:11 +01:00
_, found = checkworldinfo(decoded, force_use_txt=True, actions=vars._actions)
Add TPU support for dynamic WI scan and generation modifiers
2022-01-15 03:39:02 +01:00
found -= excluded_world_info[i]
if(len(found) != 0):
regeneration_required = True
break
return excluded_world_info, regeneration_required, halt
Show message when TPU backend is compiling
2022-01-17 03:09:10 +01:00
def tpumtjgenerate_compiling_callback() -> None:
print(colors.GREEN + "TPU backend compilation triggered" + colors.END)
vars.compiling = True
def tpumtjgenerate_stopped_compiling_callback() -> None:
vars.compiling = False
Dynamic TPU backend should support dynamic warpers and abort button
2022-01-17 20:10:32 +01:00
def tpumtjgenerate_settings_callback() -> dict:
return {
"top_p": float(vars.top_p),
"temp": float(vars.temp),
"top_k": int(vars.top_k),
"tfs": float(vars.tfs),
"repetition_penalty": float(vars.rep_pen),
Repetition penalty slope and range
2022-01-24 21:30:38 +01:00
"rpslope": float(vars.rep_pen_slope),
"rprange": int(vars.rep_pen_range),
Dynamic TPU backend should support dynamic warpers and abort button
2022-01-17 20:10:32 +01:00
}
Show message when TPU backend is compiling
2022-01-17 03:09:10 +01:00
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# If we're running Colab or OAI, we still need a tokenizer.
if(vars.model == "Colab"):
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
from transformers import GPT2TokenizerFast
Path loading improvements This fixes a few scenario's of my commit yesterday, models that have a / are now first loaded from the corrected directory if it exists before we fall back to its original name to make sure it loads the config from the correct location. Cache dir fixes and a improved routine for the path loaded models that mimics the NeoCustom option fixing models that have no model_type specified. Because GPT2 doesn't work well with this option and should exclusively be used with the GPT2Custom and GPT-J models should have a model_type we assume its a Neo model when not specified.
2021-12-23 14:40:35 +01:00
tokenizer = GPT2TokenizerFast.from_pretrained("EleutherAI/gpt-neo-2.7B", cache_dir="cache/")
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
elif(vars.model == "OAI"):
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
from transformers import GPT2TokenizerFast
Path loading improvements This fixes a few scenario's of my commit yesterday, models that have a / are now first loaded from the corrected directory if it exists before we fall back to its original name to make sure it loads the config from the correct location. Cache dir fixes and a improved routine for the path loaded models that mimics the NeoCustom option fixing models that have no model_type specified. Because GPT2 doesn't work well with this option and should exclusively be used with the GPT2Custom and GPT-J models should have a model_type we assume its a Neo model when not specified.
2021-12-23 14:40:35 +01:00
tokenizer = GPT2TokenizerFast.from_pretrained("gpt2", cache_dir="cache/")
Integrate TPU backend This commit puts the TPU backend code directly in to the KoboldAI code to make it easier to modify.
2021-11-20 00:06:57 +01:00
# Load the TPU backend if requested
elif(vars.model == "TPUMeshTransformerGPTJ"):
print("{0}Initializing Mesh Transformer JAX, please wait...{1}".format(colors.PURPLE, colors.END))
assert vars.model == "TPUMeshTransformerGPTJ" and vars.custmodpth and os.path.isdir(vars.custmodpth)
import tpu_mtj_backend
Badwords unification TPU's no longer use hardcoded badwords but instead use the var
2022-01-29 18:09:53 +01:00
tpu_mtj_backend.vars = vars
Add TPU support for dynamic WI scan and generation modifiers
2022-01-15 03:39:02 +01:00
tpu_mtj_backend.warper_callback = tpumtjgenerate_warper_callback
Fix TPU generation modifier
2022-01-15 05:00:06 +01:00
tpu_mtj_backend.stopping_callback = tpumtjgenerate_stopping_callback
Show message when TPU backend is compiling
2022-01-17 03:09:10 +01:00
tpu_mtj_backend.compiling_callback = tpumtjgenerate_compiling_callback
tpu_mtj_backend.stopped_compiling_callback = tpumtjgenerate_stopped_compiling_callback
Fix settings callback, and `genout.shape[-1]` in `tpumtjgenerate()`
2022-01-17 20:52:29 +01:00
tpu_mtj_backend.settings_callback = tpumtjgenerate_settings_callback
Integrate TPU backend This commit puts the TPU backend code directly in to the KoboldAI code to make it easier to modify.
2021-11-20 00:06:57 +01:00
tpu_mtj_backend.load_model(vars.custmodpth)
Add soft prompt support to TPU backend
2021-11-22 00:08:04 +01:00
vars.allowsp = True
vars.modeldim = int(tpu_mtj_backend.params["d_model"])
Integrate TPU backend This commit puts the TPU backend code directly in to the KoboldAI code to make it easier to modify.
2021-11-20 00:06:57 +01:00
tokenizer = tpu_mtj_backend.tokenizer
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Set up Flask routes
@app.route('/')
@app.route('/index')
def index():
return render_template('index.html')
@app.route('/download')
def download():
save_format = request.args.get("format", "json").strip().lower()
if(save_format == "plaintext"):
txt = vars.prompt + "".join(vars.actions.values())
save = Response(txt)
filename = path.basename(vars.savedir)
if filename[-5:] == ".json":
filename = filename[:-5]
save.headers.set('Content-Disposition', 'attachment', filename='%s.txt' % filename)
return(save)
# Build json to write
js = {}
js["gamestarted"] = vars.gamestarted
js["prompt"] = vars.prompt
js["memory"] = vars.memory
js["authorsnote"] = vars.authornote
Author's Note Template
2021-12-30 05:43:36 +01:00
js["anotetemplate"] = vars.authornotetemplate
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
js["actions"] = tuple(vars.actions.values())
js["worldinfo"] = []
# Extract only the important bits of WI
for wi in vars.worldinfo:
if(wi["constant"] or wi["key"] != ""):
js["worldinfo"].append({
"key": wi["key"],
"keysecondary": wi["keysecondary"],
"content": wi["content"],
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
"comment": wi["comment"],
"folder": wi["folder"],
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
"selective": wi["selective"],
"constant": wi["constant"]
})
save = Response(json.dumps(js, indent=3))
filename = path.basename(vars.savedir)
if filename[-5:] == ".json":
filename = filename[:-5]
save.headers.set('Content-Disposition', 'attachment', filename='%s.json' % filename)
return(save)
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
Add Userscripts menu into GUI
2021-12-13 07:03:26 +01:00
#============================ LUA API =============================#
if(path.exists("settings/" + getmodelname().replace('/', '_') + ".settings")):
file = open("settings/" + getmodelname().replace('/', '_') + ".settings", "r")
js = json.load(file)
if("userscripts" in js):
vars.userscripts = []
for userscript in js["userscripts"]:
if type(userscript) is not str:
continue
userscript = userscript.strip()
if len(userscript) != 0 and all(q not in userscript for q in ("..", ":")) and all(userscript[0] not in q for q in ("/", "\\")) and os.path.exists(fileops.uspath(userscript)):
vars.userscripts.append(userscript)
Add a "corescript" setting
2021-12-14 01:28:33 +01:00
if("corescript" in js and type(js["corescript"]) is str and all(q not in js["corescript"] for q in ("..", ":")) and all(js["corescript"][0] not in q for q in ("/", "\\"))):
vars.corescript = js["corescript"]
else:
vars.corescript = "default.lua"
Add Userscripts menu into GUI
2021-12-13 07:03:26 +01:00
file.close()
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
Format Lua printed messages and warnings
2021-12-13 07:59:53 +01:00
def lua_log_format_name(name):
return f"[{name}]" if type(name) is str else "CORE"
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
_bridged = {}
Fix the type hint for `bridged_kwarg` decorator
2022-01-05 02:48:34 +01:00
F = TypeVar("F", bound=Callable)
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
def bridged_kwarg(name=None):
Fix the type hint for `bridged_kwarg` decorator
2022-01-05 02:48:34 +01:00
def _bridged_kwarg(f: F):
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
_bridged[name if name is not None else f.__name__[4:] if f.__name__[:4] == "lua_" else f.__name__] = f
return f
return _bridged_kwarg
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
#==================================================================#
# Event triggered when a userscript is loaded
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Add Userscripts menu into GUI
2021-12-13 07:03:26 +01:00
def load_callback(filename, modulename):
Format Lua printed messages and warnings
2021-12-13 07:59:53 +01:00
print(colors.GREEN + f"Loading Userscript [{modulename}] <{filename}>" + colors.END)
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
#==================================================================#
# Load all Lua scripts
#==================================================================#
def load_lua_scripts():
Format Lua printed messages and warnings
2021-12-13 07:59:53 +01:00
print(colors.GREEN + "Loading Core Script" + colors.END)
Add Userscripts menu into GUI
2021-12-13 07:03:26 +01:00
filenames = []
modulenames = []
descriptions = []
lst = fileops.getusfiles(long_desc=True)
filenames_dict = {ob["filename"]: i for i, ob in enumerate(lst)}
for filename in vars.userscripts:
if filename in filenames_dict:
i = filenames_dict[filename]
filenames.append(filename)
modulenames.append(lst[i]["modulename"])
descriptions.append(lst[i]["description"])
Use original TPU backend if possible
2022-01-16 05:31:07 +01:00
vars.has_genmod = False
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
try:
vars.lua_koboldbridge.obliterate_multiverse()
Add a "corescript" setting
2021-12-14 01:28:33 +01:00
tpool.execute(vars.lua_koboldbridge.load_corescript, vars.corescript)
Use original TPU backend if possible
2022-01-16 05:31:07 +01:00
vars.has_genmod = tpool.execute(vars.lua_koboldbridge.load_userscripts, filenames, modulenames, descriptions)
Display icons for active userscripts and softprompts Also fixes the userscript menu so that the active userscripts preserve the previously selected order as was originally intended.
2021-12-23 05:33:27 +01:00
vars.lua_running = True
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
except lupa.LuaError as e:
Use original TPU backend if possible
2022-01-16 05:31:07 +01:00
try:
vars.lua_koboldbridge.obliterate_multiverse()
except:
pass
Display icons for active userscripts and softprompts Also fixes the userscript menu so that the active userscripts preserve the previously selected order as was originally intended.
2021-12-23 05:33:27 +01:00
vars.lua_running = False
Catch Lua errors properly
2021-12-13 08:32:09 +01:00
if(vars.serverstarted):
Fix typos in error messages
2022-01-14 04:33:55 +01:00
emit('from_server', {'cmd': 'errmsg', 'data': 'Lua script error; please check console.'}, broadcast=True)
Display icons for active userscripts and softprompts Also fixes the userscript menu so that the active userscripts preserve the previously selected order as was originally intended.
2021-12-23 05:33:27 +01:00
sendUSStatItems()
Catch Lua errors properly
2021-12-13 08:32:09 +01:00
print("{0}{1}{2}".format(colors.RED, "***LUA ERROR***: ", colors.END), end="", file=sys.stderr)
Remove escape characters from all error messages
2021-12-13 17:47:34 +01:00
print("{0}{1}{2}".format(colors.RED, str(e).replace("\033", ""), colors.END), file=sys.stderr)
Catch Lua errors properly
2021-12-13 08:32:09 +01:00
print("{0}{1}{2}".format(colors.YELLOW, "Lua engine stopped; please open 'Userscripts' and press Load to reinitialize scripts.", colors.END), file=sys.stderr)
if(vars.serverstarted):
set_aibusy(0)
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
Format Lua printed messages and warnings
2021-12-13 07:59:53 +01:00
#==================================================================#
# Print message that originates from the userscript with the given name
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Format Lua printed messages and warnings
2021-12-13 07:59:53 +01:00
def lua_print(msg):
if(vars.lua_logname != vars.lua_koboldbridge.logging_name):
vars.lua_logname = vars.lua_koboldbridge.logging_name
print(colors.BLUE + lua_log_format_name(vars.lua_logname) + ":" + colors.END, file=sys.stderr)
print(colors.PURPLE + msg.replace("\033", "") + colors.END)
#==================================================================#
# Print warning that originates from the userscript with the given name
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Format Lua printed messages and warnings
2021-12-13 07:59:53 +01:00
def lua_warn(msg):
if(vars.lua_logname != vars.lua_koboldbridge.logging_name):
vars.lua_logname = vars.lua_koboldbridge.logging_name
print(colors.BLUE + lua_log_format_name(vars.lua_logname) + ":" + colors.END, file=sys.stderr)
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
print(colors.YELLOW + msg.replace("\033", "") + colors.END)
Format Lua printed messages and warnings
2021-12-13 07:59:53 +01:00
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
#==================================================================#
# Decode tokens into a string using current tokenizer
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
def lua_decode(tokens):
Fix the Lua tokenizer API
2021-12-12 03:24:34 +01:00
tokens = list(tokens.values())
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
assert type(tokens) is list
Fix the Lua tokenizer API
2021-12-12 03:24:34 +01:00
if("tokenizer" not in globals()):
from transformers import GPT2TokenizerFast
global tokenizer
Path loading improvements This fixes a few scenario's of my commit yesterday, models that have a / are now first loaded from the corrected directory if it exists before we fall back to its original name to make sure it loads the config from the correct location. Cache dir fixes and a improved routine for the path loaded models that mimics the NeoCustom option fixing models that have no model_type specified. Because GPT2 doesn't work well with this option and should exclusively be used with the GPT2Custom and GPT-J models should have a model_type we assume its a Neo model when not specified.
2021-12-23 14:40:35 +01:00
tokenizer = GPT2TokenizerFast.from_pretrained("gpt2", cache_dir="cache/")
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
return tokenizer.decode(tokens)
#==================================================================#
# Encode string into list of token IDs using current tokenizer
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
def lua_encode(string):
assert type(string) is str
Fix the Lua tokenizer API
2021-12-12 03:24:34 +01:00
if("tokenizer" not in globals()):
from transformers import GPT2TokenizerFast
global tokenizer
Path loading improvements This fixes a few scenario's of my commit yesterday, models that have a / are now first loaded from the corrected directory if it exists before we fall back to its original name to make sure it loads the config from the correct location. Cache dir fixes and a improved routine for the path loaded models that mimics the NeoCustom option fixing models that have no model_type specified. Because GPT2 doesn't work well with this option and should exclusively be used with the GPT2Custom and GPT-J models should have a model_type we assume its a Neo model when not specified.
2021-12-23 14:40:35 +01:00
tokenizer = GPT2TokenizerFast.from_pretrained("gpt2", cache_dir="cache/")
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
return tokenizer.encode(string, max_length=int(4e9), truncation=True)
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
#==================================================================#
# Computes context given a submission, Lua array of entry UIDs and a Lua array
# of folder UIDs
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Add option to `compute_context()` to not scan story
2022-01-05 01:26:59 +01:00
def lua_compute_context(submission, entries, folders, kwargs):
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
assert type(submission) is str
Add option to `compute_context()` to not scan story
2022-01-05 01:26:59 +01:00
if(kwargs is None):
kwargs = vars.lua_state.table()
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
actions = vars._actions if vars.lua_koboldbridge.userstate == "genmod" else vars.actions
allowed_entries = None
allowed_folders = None
if(entries is not None):
allowed_entries = set()
i = 1
while(entries[i] is not None):
allowed_entries.add(int(entries[i]))
i += 1
if(folders is not None):
allowed_folders = set()
i = 1
while(folders[i] is not None):
allowed_folders.add(int(folders[i]))
i += 1
Add option to `compute_context()` to not scan story
2022-01-05 01:26:59 +01:00
winfo, mem, anotetxt, _ = calcsubmitbudgetheader(
submission,
allowed_entries=allowed_entries,
allowed_folders=allowed_folders,
force_use_txt=True,
scan_story=kwargs["scan_story"] if kwargs["scan_story"] != None else True,
)
txt, _, _ = calcsubmitbudget(
len(actions),
winfo,
mem,
anotetxt,
actions,
)
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
return tokenizer.decode(txt)
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
#==================================================================#
# Get property of a world info entry given its UID and property name
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
def lua_get_attr(uid, k):
assert type(uid) is int and type(k) is str
if(uid in vars.worldinfo_u and k in (
"key",
"keysecondary",
"content",
"comment",
"folder",
"num",
"selective",
"constant",
"uid",
)):
return vars.worldinfo_u[uid][k]
#==================================================================#
# Set property of a world info entry given its UID, property name and new value
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
def lua_set_attr(uid, k, v):
assert type(uid) is int and type(k) is str
assert uid in vars.worldinfo_u and k in (
"key",
"keysecondary",
"content",
"comment",
"selective",
"constant",
)
if(type(vars.worldinfo_u[uid][k]) is int and type(v) is float):
v = int(v)
assert type(vars.worldinfo_u[uid][k]) is type(v)
vars.worldinfo_u[uid][k] = v
Format Lua printed messages and warnings
2021-12-13 07:59:53 +01:00
print(colors.GREEN + f"{lua_log_format_name(vars.lua_koboldbridge.logging_name)} set {k} of world info entry {uid} to {v}" + colors.END)
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
#==================================================================#
# Get property of a world info folder given its UID and property name
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
def lua_folder_get_attr(uid, k):
assert type(uid) is int and type(k) is str
if(uid in vars.wifolders_d and k in (
Fix numerous problems in the Lua world info API
2021-12-12 01:11:38 +01:00
"name",
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
)):
return vars.wifolders_d[uid][k]
#==================================================================#
# Set property of a world info folder given its UID, property name and new value
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
def lua_folder_set_attr(uid, k, v):
assert type(uid) is int and type(k) is str
assert uid in vars.wifolders_d and k in (
Fix numerous problems in the Lua world info API
2021-12-12 01:11:38 +01:00
"name",
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
)
if(type(vars.wifolders_d[uid][k]) is int and type(v) is float):
v = int(v)
assert type(vars.wifolders_d[uid][k]) is type(v)
vars.wifolders_d[uid][k] = v
Format Lua printed messages and warnings
2021-12-13 07:59:53 +01:00
print(colors.GREEN + f"{lua_log_format_name(vars.lua_koboldbridge.logging_name)} set {k} of world info folder {uid} to {v}" + colors.END)
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
#==================================================================#
# Get the "Amount to Generate"
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Fix Lua settings API
2021-12-11 23:01:41 +01:00
def lua_get_genamt():
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
return vars.genamt
#==================================================================#
# Set the "Amount to Generate"
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Fix Lua settings API
2021-12-11 23:01:41 +01:00
def lua_set_genamt(genamt):
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
assert vars.lua_koboldbridge.userstate != "genmod" and type(genamt) in (int, float) and genamt >= 0
Format Lua printed messages and warnings
2021-12-13 07:59:53 +01:00
print(colors.GREEN + f"{lua_log_format_name(vars.lua_koboldbridge.logging_name)} set genamt to {int(genamt)}" + colors.END)
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
vars.genamt = int(genamt)
#==================================================================#
# Get the "Gens Per Action"
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
def lua_get_numseqs():
return vars.numseqs
#==================================================================#
# Set the "Gens Per Action"
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
def lua_set_numseqs(numseqs):
assert type(numseqs) in (int, float) and numseqs >= 1
Format Lua printed messages and warnings
2021-12-13 07:59:53 +01:00
print(colors.GREEN + f"{lua_log_format_name(vars.lua_koboldbridge.logging_name)} set numseqs to {int(numseqs)}" + colors.END)
Lua API fixes for deleting/editing story chunks
2022-01-01 00:28:03 +01:00
vars.numseqs = int(numseqs)
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
#==================================================================#
# Check if a setting exists with the given name
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
def lua_has_setting(setting):
return setting in (
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
"anotedepth",
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
"settemp",
"settopp",
"settopk",
"settfs",
"setreppen",
Repetition penalty slope and range
2022-01-24 21:30:38 +01:00
"setreppenslope",
"setreppenrange",
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
"settknmax",
"setwidepth",
"setuseprompt",
"setadventure",
Chat Mode The Initial commit for Chat Mode, the nickname part of the UI is missing other than that it should be fully functional. To use Chat Mode effectively you first input a small dialogue (Can be around 6 lines 3 of your own inputs and 3 of the character) formatted as Name : it will then automate the actions needed to chat properly. During this mode single line mode is forced on, and Trim Incomplete Sentences is forced off.
2021-12-26 01:51:32 +01:00
"setchatmode",
Allow Lua API to toggle dynamic scan
2021-12-12 07:55:46 +01:00
"setdynamicscan",
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
"setnopromptgen",
Add Memory box to Random Story dialog and "Random Story Persist"
2021-12-30 05:15:59 +01:00
"setrngpersist",
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
"temp",
"topp",
Add Memory box to Random Story dialog and "Random Story Persist"
2021-12-30 05:15:59 +01:00
"top_p",
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
"topk",
Add Memory box to Random Story dialog and "Random Story Persist"
2021-12-30 05:15:59 +01:00
"top_k",
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
"tfs",
"reppen",
Repetition penalty slope and range
2022-01-24 21:30:38 +01:00
"reppenslope",
"reppenrange",
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
"tknmax",
"widepth",
"useprompt",
Chat Mode The Initial commit for Chat Mode, the nickname part of the UI is missing other than that it should be fully functional. To use Chat Mode effectively you first input a small dialogue (Can be around 6 lines 3 of your own inputs and 3 of the character) formatted as Name : it will then automate the actions needed to chat properly. During this mode single line mode is forced on, and Trim Incomplete Sentences is forced off.
2021-12-26 01:51:32 +01:00
"chatmode",
Store chatname Also save the chatname in the settings for later re-use by the user.
2021-12-26 01:55:27 +01:00
"chatname",
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
"adventure",
"dynamicscan",
No Prompt Gen Allow people to enter a prompt without generating anything by the AI. Combined with the always add prompt this is a very useful feature that allows people to write world information first, and then do a specific action. This mimics the behavior previously seen in AI Dungeon forks where it prompts for world information and then asks an action and can be particularly useful for people who want the prompt to always be part of the generation.
2021-12-16 12:47:44 +01:00
"nopromptgen",
Add Memory box to Random Story dialog and "Random Story Persist"
2021-12-30 05:15:59 +01:00
"rngpersist",
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
"frmttriminc",
"frmtrmblln",
"frmtrmspch",
"frmtadsnsp",
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
"frmtsingleline",
"triminc",
"rmblln",
"rmspch",
"adsnsp",
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
"singleline",
)
#==================================================================#
# Return the setting with the given name if it exists
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
def lua_get_setting(setting):
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
if(setting in ("settemp", "temp")): return vars.temp
Add Memory box to Random Story dialog and "Random Story Persist"
2021-12-30 05:15:59 +01:00
if(setting in ("settopp", "topp", "top_p")): return vars.top_p
if(setting in ("settopk", "topk", "top_k")): return vars.top_k
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
if(setting in ("settfs", "tfs")): return vars.tfs
if(setting in ("setreppen", "reppen")): return vars.rep_pen
Repetition penalty slope and range
2022-01-24 21:30:38 +01:00
if(setting in ("setreppenslope", "reppenslope")): return vars.rep_pen_slope
if(setting in ("setreppenrange", "reppenrange")): return vars.rep_pen_range
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
if(setting in ("settknmax", "tknmax")): return vars.max_length
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
if(setting == "anotedepth"): return vars.andepth
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
if(setting in ("setwidepth", "widepth")): return vars.widepth
if(setting in ("setuseprompt", "useprompt")): return vars.useprompt
if(setting in ("setadventure", "adventure")): return vars.adventure
Chat Mode The Initial commit for Chat Mode, the nickname part of the UI is missing other than that it should be fully functional. To use Chat Mode effectively you first input a small dialogue (Can be around 6 lines 3 of your own inputs and 3 of the character) formatted as Name : it will then automate the actions needed to chat properly. During this mode single line mode is forced on, and Trim Incomplete Sentences is forced off.
2021-12-26 01:51:32 +01:00
if(setting in ("setchatmode", "chatmode")): return vars.chatmode
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
if(setting in ("setdynamicscan", "dynamicscan")): return vars.dynamicscan
if(setting in ("setnopromptgen", "nopromptgen")): return vars.nopromptgen
Add Memory box to Random Story dialog and "Random Story Persist"
2021-12-30 05:15:59 +01:00
if(setting in ("setrngpersist", "rngpersist")): return vars.rngpersist
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
if(setting in ("frmttriminc", "triminc")): return vars.formatoptns["frmttriminc"]
if(setting in ("frmtrmblln", "rmblln")): return vars.formatoptns["frmttrmblln"]
if(setting in ("frmtrmspch", "rmspch")): return vars.formatoptns["frmttrmspch"]
if(setting in ("frmtadsnsp", "adsnsp")): return vars.formatoptns["frmtadsnsp"]
if(setting in ("frmtsingleline", "singleline")): return vars.formatoptns["singleline"]
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
#==================================================================#
# Set the setting with the given name if it exists
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
def lua_set_setting(setting, v):
actual_type = type(lua_get_setting(setting))
assert v is not None and (actual_type is type(v) or (actual_type is int and type(v) is float))
v = actual_type(v)
Format Lua printed messages and warnings
2021-12-13 07:59:53 +01:00
print(colors.GREEN + f"{lua_log_format_name(vars.lua_koboldbridge.logging_name)} set {setting} to {v}" + colors.END)
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
if(setting in ("setadventure", "adventure") and v):
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
vars.actionmode = 1
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
if(setting in ("settemp", "temp")): vars.temp = v
if(setting in ("settopp", "topp")): vars.top_p = v
if(setting in ("settopk", "topk")): vars.top_k = v
if(setting in ("settfs", "tfs")): vars.tfs = v
if(setting in ("setreppen", "reppen")): vars.rep_pen = v
Repetition penalty slope and range
2022-01-24 21:30:38 +01:00
if(setting in ("setreppenslope", "reppenslope")): vars.rep_pen_slope = v
if(setting in ("setreppenrange", "reppenrange")): vars.rep_pen_range = v
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
if(setting in ("settknmax", "tknmax")): vars.max_length = v; return True
if(setting == "anotedepth"): vars.andepth = v; return True
if(setting in ("setwidepth", "widepth")): vars.widepth = v; return True
if(setting in ("setuseprompt", "useprompt")): vars.useprompt = v; return True
if(setting in ("setadventure", "adventure")): vars.adventure = v
if(setting in ("setdynamicscan", "dynamicscan")): vars.dynamicscan = v
if(setting in ("setnopromptgen", "nopromptgen")): vars.nopromptgen = v
Add Memory box to Random Story dialog and "Random Story Persist"
2021-12-30 05:15:59 +01:00
if(setting in ("setrngpersist", "rngpersist")): vars.rngpersist = v
Chat Mode The Initial commit for Chat Mode, the nickname part of the UI is missing other than that it should be fully functional. To use Chat Mode effectively you first input a small dialogue (Can be around 6 lines 3 of your own inputs and 3 of the character) formatted as Name : it will then automate the actions needed to chat properly. During this mode single line mode is forced on, and Trim Incomplete Sentences is forced off.
2021-12-26 01:51:32 +01:00
if(setting in ("setchatmode", "chatmode")): vars.chatmode = v
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
if(setting in ("frmttriminc", "triminc")): vars.formatoptns["frmttriminc"] = v
if(setting in ("frmtrmblln", "rmblln")): vars.formatoptns["frmttrmblln"] = v
if(setting in ("frmtrmspch", "rmspch")): vars.formatoptns["frmttrmspch"] = v
if(setting in ("frmtadsnsp", "adsnsp")): vars.formatoptns["frmtadsnsp"] = v
if(setting in ("frmtsingleline", "singleline")): vars.formatoptns["singleline"] = v
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
#==================================================================#
# Get contents of memory
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
def lua_get_memory():
return vars.memory
#==================================================================#
# Set contents of memory
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
def lua_set_memory(m):
assert type(m) is str
vars.memory = m
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
#==================================================================#
# Get contents of author's note
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
def lua_get_authorsnote():
return vars.authornote
#==================================================================#
# Set contents of author's note
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
def lua_set_authorsnote(m):
assert type(m) is str
vars.authornote = m
Author's Note Template
2021-12-30 05:43:36 +01:00
#==================================================================#
# Get contents of author's note template
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Author's Note Template
2021-12-30 05:43:36 +01:00
def lua_get_authorsnotetemplate():
return vars.authornotetemplate
#==================================================================#
# Set contents of author's note template
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Author's Note Template
2021-12-30 05:43:36 +01:00
def lua_set_authorsnotetemplate(m):
assert type(m) is str
vars.authornotetemplate = m
Fix Lua settings API
2021-12-11 23:01:41 +01:00
#==================================================================#
# Save settings and send them to client
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Fix Lua settings API
2021-12-11 23:01:41 +01:00
def lua_resend_settings():
settingschanged()
refresh_settings()
Add Lua API for story chunks
2021-12-12 05:44:07 +01:00
#==================================================================#
# Set story chunk text and delete the chunk if the new chunk is empty
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Add Lua API for story chunks
2021-12-12 05:44:07 +01:00
def lua_set_chunk(k, v):
assert type(k) in (int, None) and type(v) is str
assert k >= 0
assert k != 0 or len(v) != 0
if(len(v) == 0):
Format Lua printed messages and warnings
2021-12-13 07:59:53 +01:00
print(colors.GREEN + f"{lua_log_format_name(vars.lua_koboldbridge.logging_name)} deleted story chunk {k}" + colors.END)
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
chunk = int(k)
if(vars.lua_koboldbridge.userstate == "genmod"):
del vars._actions[chunk-1]
Lua API fixes for deleting/editing story chunks
2022-01-01 00:28:03 +01:00
vars.lua_deleted.add(chunk)
if(not hasattr(vars, "_actions") or vars._actions is not vars.actions):
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
del vars.actions[chunk-1]
Add Lua API for story chunks
2021-12-12 05:44:07 +01:00
else:
if(k == 0):
Format Lua printed messages and warnings
2021-12-13 07:59:53 +01:00
print(colors.GREEN + f"{lua_log_format_name(vars.lua_koboldbridge.logging_name)} edited prompt chunk" + colors.END)
Add Lua API for story chunks
2021-12-12 05:44:07 +01:00
else:
Format Lua printed messages and warnings
2021-12-13 07:59:53 +01:00
print(colors.GREEN + f"{lua_log_format_name(vars.lua_koboldbridge.logging_name)} edited story chunk {k}" + colors.END)
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
chunk = int(k)
if(chunk == 0):
if(vars.lua_koboldbridge.userstate == "genmod"):
vars._prompt = v
Lua API fixes for deleting/editing story chunks
2022-01-01 00:28:03 +01:00
vars.lua_edited.add(chunk)
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
vars.prompt = v
else:
if(vars.lua_koboldbridge.userstate == "genmod"):
vars._actions[chunk-1] = v
Lua API fixes for deleting/editing story chunks
2022-01-01 00:28:03 +01:00
vars.lua_edited.add(chunk)
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
vars.actions[chunk-1] = v
Add Lua API for story chunks
2021-12-12 05:44:07 +01:00
Add Lua API for reading model information
2021-12-12 18:09:59 +01:00
#==================================================================#
# Get model type as "gpt-2-xl", "gpt-neo-2.7B", etc.
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Add Lua API for reading model information
2021-12-12 18:09:59 +01:00
def lua_get_modeltype():
if(vars.noai):
return "readonly"
if(vars.model in ("Colab", "OAI", "InferKit")):
return "api"
Update Lua modeltype and model API
2021-12-23 22:35:52 +01:00
if(vars.model not in ("TPUMeshTransformerGPTJ",) and (vars.model in ("GPT2Custom", "NeoCustom") or vars.model_type in ("gpt2", "gpt_neo", "gptj"))):
Add Lua API for reading model information
2021-12-12 18:09:59 +01:00
hidden_size = get_hidden_size_from_model(model)
Update Lua modeltype and model API
2021-12-23 22:35:52 +01:00
if(vars.model in ("gpt2",) or (vars.model_type == "gpt2" and hidden_size == 768)):
Add Lua API for reading model information
2021-12-12 18:09:59 +01:00
return "gpt2"
Update Lua modeltype and model API
2021-12-23 22:35:52 +01:00
if(vars.model in ("gpt2-medium",) or (vars.model_type == "gpt2" and hidden_size == 1024)):
Add Lua API for reading model information
2021-12-12 18:09:59 +01:00
return "gpt2-medium"
Update Lua modeltype and model API
2021-12-23 22:35:52 +01:00
if(vars.model in ("gpt2-large",) or (vars.model_type == "gpt2" and hidden_size == 1280)):
Add Lua API for reading model information
2021-12-12 18:09:59 +01:00
return "gpt2-large"
Update Lua modeltype and model API
2021-12-23 22:35:52 +01:00
if(vars.model in ("gpt2-xl",) or (vars.model_type == "gpt2" and hidden_size == 1600)):
Add Lua API for reading model information
2021-12-12 18:09:59 +01:00
return "gpt2-xl"
Update Lua modeltype and model API
2021-12-23 22:35:52 +01:00
if(vars.model_type == "gpt_neo" and hidden_size == 768):
Add Lua API for reading model information
2021-12-12 18:09:59 +01:00
return "gpt-neo-125M"
Update Lua modeltype and model API
2021-12-23 22:35:52 +01:00
if(vars.model in ("EleutherAI/gpt-neo-1.3B",) or (vars.model_type == "gpt_neo" and hidden_size == 2048)):
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
return "gpt-neo-1.3B"
Update Lua modeltype and model API
2021-12-23 22:35:52 +01:00
if(vars.model in ("EleutherAI/gpt-neo-2.7B",) or (vars.model_type == "gpt_neo" and hidden_size == 2560)):
Add Lua API for reading model information
2021-12-12 18:09:59 +01:00
return "gpt-neo-2.7B"
Update Lua modeltype and model API
2021-12-23 22:35:52 +01:00
if(vars.model in ("EleutherAI/gpt-j-6B",) or (vars.model == "TPUMeshTransformerGPTJ" and tpu_mtj_backend.params["d_model"] == 4096) or (vars.model_type in ("gpt_neo", "gptj") and hidden_size == 4096)):
Add Lua API for reading model information
2021-12-12 18:09:59 +01:00
return "gpt-j-6B"
return "unknown"
#==================================================================#
# Get model backend as "transformers" or "mtj"
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Add Lua API for reading model information
2021-12-12 18:09:59 +01:00
def lua_get_modelbackend():
if(vars.noai):
return "readonly"
if(vars.model in ("Colab", "OAI", "InferKit")):
return "api"
if(vars.model in ("TPUMeshTransformerGPTJ",)):
return "mtj"
return "transformers"
#==================================================================#
# Check whether model is loaded from a custom path
#==================================================================#
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
@bridged_kwarg()
Add Lua API for reading model information
2021-12-12 18:09:59 +01:00
def lua_is_custommodel():
return vars.model in ("GPT2Custom", "NeoCustom", "TPUMeshTransformerGPTJ")
Fix Lua settings API
2021-12-11 23:01:41 +01:00
#==================================================================#
#
#==================================================================#
def execute_inmod():
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(False)
Format Lua printed messages and warnings
2021-12-13 07:59:53 +01:00
vars.lua_logname = ...
More Lua API fixes * Removed `vars.model_orig` * `requirex()` in bridge.lua now maintains a separate module cache for each userscript instead of using the same cache for all userscripts * `vars.lua_deleted` and `vars.lua_edited` are now erased right before running the input modifiers instead of right before each time the generation modifiers are run
2021-12-26 18:49:28 +01:00
vars.lua_edited = set()
vars.lua_deleted = set()
Catch Lua errors properly
2021-12-13 08:32:09 +01:00
try:
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
tpool.execute(vars.lua_koboldbridge.execute_inmod)
Catch Lua errors properly
2021-12-13 08:32:09 +01:00
except lupa.LuaError as e:
vars.lua_koboldbridge.obliterate_multiverse()
Display icons for active userscripts and softprompts Also fixes the userscript menu so that the active userscripts preserve the previously selected order as was originally intended.
2021-12-23 05:33:27 +01:00
vars.lua_running = False
Fix typos in error messages
2022-01-14 04:33:55 +01:00
emit('from_server', {'cmd': 'errmsg', 'data': 'Lua script error; please check console.'}, broadcast=True)
Display icons for active userscripts and softprompts Also fixes the userscript menu so that the active userscripts preserve the previously selected order as was originally intended.
2021-12-23 05:33:27 +01:00
sendUSStatItems()
Catch Lua errors properly
2021-12-13 08:32:09 +01:00
print("{0}{1}{2}".format(colors.RED, "***LUA ERROR***: ", colors.END), end="", file=sys.stderr)
Remove escape characters from all error messages
2021-12-13 17:47:34 +01:00
print("{0}{1}{2}".format(colors.RED, str(e).replace("\033", ""), colors.END), file=sys.stderr)
Catch Lua errors properly
2021-12-13 08:32:09 +01:00
print("{0}{1}{2}".format(colors.YELLOW, "Lua engine stopped; please open 'Userscripts' and press Load to reinitialize scripts.", colors.END), file=sys.stderr)
set_aibusy(0)
Fix Lua settings API
2021-12-11 23:01:41 +01:00
def execute_genmod():
vars.lua_koboldbridge.execute_genmod()
def execute_outmod():
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(False)
Show message when TPU backend is compiling
2022-01-17 03:09:10 +01:00
emit('from_server', {'cmd': 'hidemsg', 'data': ''}, broadcast=True)
Catch Lua errors properly
2021-12-13 08:32:09 +01:00
try:
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
tpool.execute(vars.lua_koboldbridge.execute_outmod)
Catch Lua errors properly
2021-12-13 08:32:09 +01:00
except lupa.LuaError as e:
vars.lua_koboldbridge.obliterate_multiverse()
Display icons for active userscripts and softprompts Also fixes the userscript menu so that the active userscripts preserve the previously selected order as was originally intended.
2021-12-23 05:33:27 +01:00
vars.lua_running = False
Fix typos in error messages
2022-01-14 04:33:55 +01:00
emit('from_server', {'cmd': 'errmsg', 'data': 'Lua script error; please check console.'}, broadcast=True)
Display icons for active userscripts and softprompts Also fixes the userscript menu so that the active userscripts preserve the previously selected order as was originally intended.
2021-12-23 05:33:27 +01:00
sendUSStatItems()
Catch Lua errors properly
2021-12-13 08:32:09 +01:00
print("{0}{1}{2}".format(colors.RED, "***LUA ERROR***: ", colors.END), end="", file=sys.stderr)
Remove escape characters from all error messages
2021-12-13 17:47:34 +01:00
print("{0}{1}{2}".format(colors.RED, str(e).replace("\033", ""), colors.END), file=sys.stderr)
Catch Lua errors properly
2021-12-13 08:32:09 +01:00
print("{0}{1}{2}".format(colors.YELLOW, "Lua engine stopped; please open 'Userscripts' and press Load to reinitialize scripts.", colors.END), file=sys.stderr)
set_aibusy(0)
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
if(vars.lua_koboldbridge.resend_settings_required):
vars.lua_koboldbridge.resend_settings_required = False
lua_resend_settings()
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
for k in vars.lua_edited:
inlineedit(k, vars.actions[k])
for k in vars.lua_deleted:
inlinedelete(k)
Fix Lua settings API
2021-12-11 23:01:41 +01:00
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
#==================================================================#
# Lua runtime startup
#==================================================================#
Compile TPU backend in background
2022-01-07 19:47:21 +01:00
print("", end="", flush=True)
print(colors.PURPLE + "Initializing Lua Bridge... " + colors.END, end="", flush=True)
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
# Set up Lua state
vars.lua_state = lupa.LuaRuntime(unpack_returned_tuples=True)
# Load bridge.lua
bridged = {
Change script directory tree Userscripts have been moved from /scripts/userscripts to /userscripts. Core scripts have been moved from /scripts/corescripts to /cores.
2021-12-12 05:46:30 +01:00
"corescript_path": os.path.join(os.path.dirname(os.path.realpath(__file__)), "cores"),
"userscript_path": os.path.join(os.path.dirname(os.path.realpath(__file__)), "userscripts"),
Add individual configuration files for userscripts
2021-12-22 08:59:31 +01:00
"config_path": os.path.join(os.path.dirname(os.path.realpath(__file__)), "userscripts"),
Add support for multiple library paths in bridge.lua
2021-12-22 20:24:31 +01:00
"lib_paths": vars.lua_state.table(os.path.join(os.path.dirname(os.path.realpath(__file__)), "lualibs"), os.path.join(os.path.dirname(os.path.realpath(__file__)), "extern", "lualibs")),
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
"vars": vars,
}
Easier method of adding kwargs to bridged in aiserver.py
2022-01-05 01:36:21 +01:00
for kwarg in _bridged:
bridged[kwarg] = _bridged[kwarg]
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
try:
vars.lua_kobold, vars.lua_koboldcore, vars.lua_koboldbridge = vars.lua_state.globals().dofile(os.path.join(os.path.dirname(os.path.realpath(__file__)), "bridge.lua"))(
vars.lua_state.globals().python,
bridged,
)
except lupa.LuaError as e:
print(colors.RED + "ERROR!" + colors.END)
Catch Lua errors properly
2021-12-13 08:32:09 +01:00
vars.lua_koboldbridge.obliterate_multiverse()
print("{0}{1}{2}".format(colors.RED, "***LUA ERROR***: ", colors.END), end="", file=sys.stderr)
Remove escape characters from all error messages
2021-12-13 17:47:34 +01:00
print("{0}{1}{2}".format(colors.RED, str(e).replace("\033", ""), colors.END), file=sys.stderr)
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
exit(1)
print(colors.GREEN + "OK!" + colors.END)
# Load scripts
load_lua_scripts()
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#============================ METHODS =============================#
#==================================================================#
# Event triggered when browser SocketIO is loaded and connects to server
#==================================================================#
@socketio.on('connect')
def do_connect():
print("{0}Client connected!{1}".format(colors.GREEN, colors.END))
Chatname Fix Sends the chatname to the client
2021-12-27 18:52:06 +01:00
emit('from_server', {'cmd': 'setchatname', 'data': vars.chatname})
Author's Note Template
2021-12-30 05:43:36 +01:00
emit('from_server', {'cmd': 'setanotetemplate', 'data': vars.authornotetemplate})
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
emit('from_server', {'cmd': 'connected', 'smandelete': vars.smandelete, 'smanrename': vars.smanrename, 'modelname': getmodelname()})
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if(vars.remote):
emit('from_server', {'cmd': 'runs_remotely'})
if(vars.allowsp):
emit('from_server', {'cmd': 'allowsp', 'data': vars.allowsp})
Display icons for active userscripts and softprompts Also fixes the userscript menu so that the active userscripts preserve the previously selected order as was originally intended.
2021-12-23 05:33:27 +01:00
sendUSStatItems()
Save softprompt filename in settings
2021-12-23 19:02:11 +01:00
emit('from_server', {'cmd': 'spstatitems', 'data': {vars.spfilename: vars.spmeta} if vars.allowsp and len(vars.spfilename) else {}}, broadcast=True)
Display icons for active userscripts and softprompts Also fixes the userscript menu so that the active userscripts preserve the previously selected order as was originally intended.
2021-12-23 05:33:27 +01:00
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if(not vars.gamestarted):
setStartState()
sendsettings()
refresh_settings()
vars.laststory = None
emit('from_server', {'cmd': 'setstoryname', 'data': vars.laststory})
sendwi()
emit('from_server', {'cmd': 'setmemory', 'data': vars.memory})
emit('from_server', {'cmd': 'setanote', 'data': vars.authornote})
vars.mode = "play"
else:
# Game in session, send current game data and ready state to browser
refresh_story()
sendsettings()
refresh_settings()
emit('from_server', {'cmd': 'setstoryname', 'data': vars.laststory})
sendwi()
emit('from_server', {'cmd': 'setmemory', 'data': vars.memory})
emit('from_server', {'cmd': 'setanote', 'data': vars.authornote})
if(vars.mode == "play"):
if(not vars.aibusy):
emit('from_server', {'cmd': 'setgamestate', 'data': 'ready'})
else:
emit('from_server', {'cmd': 'setgamestate', 'data': 'wait'})
elif(vars.mode == "edit"):
emit('from_server', {'cmd': 'editmode', 'data': 'true'})
elif(vars.mode == "memory"):
emit('from_server', {'cmd': 'memmode', 'data': 'true'})
elif(vars.mode == "wi"):
emit('from_server', {'cmd': 'wimode', 'data': 'true'})
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
emit('from_server', {'cmd': 'gamesaved', 'data': vars.gamesaved}, broadcast=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#==================================================================#
# Event triggered when browser SocketIO sends data to the server
#==================================================================#
@socketio.on('message')
def get_message(msg):
print("{0}Data received:{1}{2}".format(colors.GREEN, msg, colors.END))
# Submit action
if(msg['cmd'] == 'submit'):
if(vars.mode == "play"):
Fix some of the logic for generation aborting
2022-01-10 23:09:47 +01:00
if(vars.aibusy):
if(msg.get('allowabort', False)):
vars.abort = True
Allow pressing send button again to stop generation
2022-01-10 22:36:15 +01:00
return
vars.abort = False
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
vars.lua_koboldbridge.feedback = None
Show a text box for chat name when Chat Mode is enabled
2021-12-27 04:21:58 +01:00
if(vars.chatmode):
if(type(msg['chatname']) is not str):
raise ValueError("Chatname must be a string")
vars.chatname = msg['chatname']
settingschanged()
Allow unique chatnames in multiplayer No longer update the chatname outside of the config, this will not effect singleplayer tab at all, but it will allow people in multiplayer to chat with their own names.
2022-01-11 21:31:44 +01:00
emit('from_server', {'cmd': 'setchatname', 'data': vars.chatname})
Retrying random story generation now also remembers memory
2022-01-04 20:40:10 +01:00
vars.recentrng = vars.recentrngm = None
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
actionsubmit(msg['data'], actionmode=msg['actionmode'])
elif(vars.mode == "edit"):
editsubmit(msg['data'])
elif(vars.mode == "memory"):
memsubmit(msg['data'])
# Retry Action
elif(msg['cmd'] == 'retry'):
Fix some of the logic for generation aborting
2022-01-10 23:09:47 +01:00
if(vars.aibusy):
if(msg.get('allowabort', False)):
vars.abort = True
Allow pressing send button again to stop generation
2022-01-10 22:36:15 +01:00
return
vars.abort = False
Show a text box for chat name when Chat Mode is enabled
2021-12-27 04:21:58 +01:00
if(vars.chatmode):
if(type(msg['chatname']) is not str):
raise ValueError("Chatname must be a string")
vars.chatname = msg['chatname']
settingschanged()
Allow unique chatnames in multiplayer No longer update the chatname outside of the config, this will not effect singleplayer tab at all, but it will allow people in multiplayer to chat with their own names.
2022-01-11 21:31:44 +01:00
emit('from_server', {'cmd': 'setchatname', 'data': vars.chatname})
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
actionretry(msg['data'])
# Back/Undo Action
elif(msg['cmd'] == 'back'):
actionback()
# EditMode Action (old)
elif(msg['cmd'] == 'edit'):
if(vars.mode == "play"):
vars.mode = "edit"
emit('from_server', {'cmd': 'editmode', 'data': 'true'}, broadcast=True)
elif(vars.mode == "edit"):
vars.mode = "play"
emit('from_server', {'cmd': 'editmode', 'data': 'false'}, broadcast=True)
# EditLine Action (old)
elif(msg['cmd'] == 'editline'):
editrequest(int(msg['data']))
# Inline edit
elif(msg['cmd'] == 'inlineedit'):
inlineedit(msg['chunk'], msg['data'])
elif(msg['cmd'] == 'inlinedelete'):
inlinedelete(msg['data'])
# DeleteLine Action (old)
elif(msg['cmd'] == 'delete'):
deleterequest()
elif(msg['cmd'] == 'memory'):
togglememorymode()
elif(not vars.remote and msg['cmd'] == 'savetofile'):
savetofile()
elif(not vars.remote and msg['cmd'] == 'loadfromfile'):
loadfromfile()
elif(msg['cmd'] == 'loadfromstring'):
loadRequest(json.loads(msg['data']), filename=msg['filename'])
elif(not vars.remote and msg['cmd'] == 'import'):
importRequest()
elif(msg['cmd'] == 'newgame'):
newGameRequest()
elif(msg['cmd'] == 'rndgame'):
Add Memory box to Random Story dialog and "Random Story Persist"
2021-12-30 05:15:59 +01:00
randomGameRequest(msg['data'], memory=msg['memory'])
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
elif(msg['cmd'] == 'settemp'):
vars.temp = float(msg['data'])
emit('from_server', {'cmd': 'setlabeltemp', 'data': msg['data']}, broadcast=True)
settingschanged()
refresh_settings()
elif(msg['cmd'] == 'settopp'):
vars.top_p = float(msg['data'])
emit('from_server', {'cmd': 'setlabeltopp', 'data': msg['data']}, broadcast=True)
settingschanged()
refresh_settings()
elif(msg['cmd'] == 'settopk'):
vars.top_k = int(msg['data'])
emit('from_server', {'cmd': 'setlabeltopk', 'data': msg['data']}, broadcast=True)
settingschanged()
refresh_settings()
elif(msg['cmd'] == 'settfs'):
vars.tfs = float(msg['data'])
emit('from_server', {'cmd': 'setlabeltfs', 'data': msg['data']}, broadcast=True)
settingschanged()
refresh_settings()
elif(msg['cmd'] == 'setreppen'):
vars.rep_pen = float(msg['data'])
emit('from_server', {'cmd': 'setlabelreppen', 'data': msg['data']}, broadcast=True)
settingschanged()
refresh_settings()
Repetition penalty slope and range
2022-01-24 21:30:38 +01:00
elif(msg['cmd'] == 'setreppenslope'):
vars.rep_pen_slope = float(msg['data'])
emit('from_server', {'cmd': 'setlabelreppenslope', 'data': msg['data']}, broadcast=True)
settingschanged()
refresh_settings()
elif(msg['cmd'] == 'setreppenrange'):
vars.rep_pen_range = float(msg['data'])
emit('from_server', {'cmd': 'setlabelreppenrange', 'data': msg['data']}, broadcast=True)
settingschanged()
refresh_settings()
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
elif(msg['cmd'] == 'setoutput'):
vars.genamt = int(msg['data'])
emit('from_server', {'cmd': 'setlabeloutput', 'data': msg['data']}, broadcast=True)
settingschanged()
refresh_settings()
elif(msg['cmd'] == 'settknmax'):
vars.max_length = int(msg['data'])
emit('from_server', {'cmd': 'setlabeltknmax', 'data': msg['data']}, broadcast=True)
settingschanged()
refresh_settings()
elif(msg['cmd'] == 'setikgen'):
vars.ikgen = int(msg['data'])
emit('from_server', {'cmd': 'setlabelikgen', 'data': msg['data']}, broadcast=True)
settingschanged()
refresh_settings()
# Author's Note field update
elif(msg['cmd'] == 'anote'):
Author's Note Template
2021-12-30 05:43:36 +01:00
anotesubmit(msg['data'], template=msg['template'])
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Author's Note depth update
elif(msg['cmd'] == 'anotedepth'):
vars.andepth = int(msg['data'])
emit('from_server', {'cmd': 'setlabelanotedepth', 'data': msg['data']}, broadcast=True)
settingschanged()
refresh_settings()
# Format - Trim incomplete sentences
elif(msg['cmd'] == 'frmttriminc'):
if('frmttriminc' in vars.formatoptns):
vars.formatoptns["frmttriminc"] = msg['data']
settingschanged()
refresh_settings()
elif(msg['cmd'] == 'frmtrmblln'):
if('frmtrmblln' in vars.formatoptns):
vars.formatoptns["frmtrmblln"] = msg['data']
settingschanged()
refresh_settings()
elif(msg['cmd'] == 'frmtrmspch'):
if('frmtrmspch' in vars.formatoptns):
vars.formatoptns["frmtrmspch"] = msg['data']
settingschanged()
refresh_settings()
elif(msg['cmd'] == 'frmtadsnsp'):
if('frmtadsnsp' in vars.formatoptns):
vars.formatoptns["frmtadsnsp"] = msg['data']
settingschanged()
refresh_settings()
elif(msg['cmd'] == 'singleline'):
if('singleline' in vars.formatoptns):
vars.formatoptns["singleline"] = msg['data']
settingschanged()
refresh_settings()
elif(msg['cmd'] == 'importselect'):
vars.importnum = int(msg["data"].replace("import", ""))
elif(msg['cmd'] == 'importcancel'):
emit('from_server', {'cmd': 'popupshow', 'data': False})
vars.importjs = {}
elif(msg['cmd'] == 'importaccept'):
emit('from_server', {'cmd': 'popupshow', 'data': False})
importgame()
elif(msg['cmd'] == 'wi'):
togglewimode()
elif(msg['cmd'] == 'wiinit'):
if(int(msg['data']) < len(vars.worldinfo)):
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(False)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
vars.worldinfo[msg['data']]["init"] = True
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
addwiitem(folder_uid=msg['folder'])
elif(msg['cmd'] == 'wifolderinit'):
addwifolder()
elif(msg['cmd'] == 'wimoveitem'):
movewiitem(msg['destination'], msg['data'])
elif(msg['cmd'] == 'wimovefolder'):
movewifolder(msg['destination'], msg['data'])
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
elif(msg['cmd'] == 'widelete'):
deletewi(msg['data'])
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
elif(msg['cmd'] == 'wifolderdelete'):
deletewifolder(msg['data'])
elif(msg['cmd'] == 'wiexpand'):
assert 0 <= int(msg['data']) < len(vars.worldinfo)
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(False)
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
emit('from_server', {'cmd': 'wiexpand', 'data': msg['data']}, broadcast=True)
elif(msg['cmd'] == 'wiexpandfolder'):
assert 0 <= int(msg['data']) < len(vars.worldinfo)
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(False)
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
emit('from_server', {'cmd': 'wiexpandfolder', 'data': msg['data']}, broadcast=True)
World info folders can now be collapsed by clicking on the folder icon
2021-12-22 19:12:35 +01:00
elif(msg['cmd'] == 'wifoldercollapsecontent'):
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(False)
World info folders can now be collapsed by clicking on the folder icon
2021-12-22 19:12:35 +01:00
vars.wifolders_d[msg['data']]['collapsed'] = True
emit('from_server', {'cmd': 'wifoldercollapsecontent', 'data': msg['data']}, broadcast=True)
elif(msg['cmd'] == 'wifolderexpandcontent'):
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(False)
World info folders can now be collapsed by clicking on the folder icon
2021-12-22 19:12:35 +01:00
vars.wifolders_d[msg['data']]['collapsed'] = False
emit('from_server', {'cmd': 'wifolderexpandcontent', 'data': msg['data']}, broadcast=True)
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
elif(msg['cmd'] == 'wiupdate'):
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(False)
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
num = int(msg['num'])
fields = ("key", "keysecondary", "content", "comment")
for field in fields:
if(field in msg['data'] and type(msg['data'][field]) is str):
vars.worldinfo[num][field] = msg['data'][field]
emit('from_server', {'cmd': 'wiupdate', 'num': msg['num'], 'data': {field: vars.worldinfo[num][field] for field in fields}}, broadcast=True)
elif(msg['cmd'] == 'wifolderupdate'):
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(False)
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
uid = int(msg['uid'])
fields = ("name", "collapsed")
for field in fields:
if(field in msg['data'] and type(msg['data'][field]) is (str if field != "collapsed" else bool)):
vars.wifolders_d[uid][field] = msg['data'][field]
emit('from_server', {'cmd': 'wifolderupdate', 'uid': msg['uid'], 'data': {field: vars.wifolders_d[uid][field] for field in fields}}, broadcast=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
elif(msg['cmd'] == 'wiselon'):
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(False)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
vars.worldinfo[msg['data']]["selective"] = True
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
emit('from_server', {'cmd': 'wiselon', 'data': msg['data']}, broadcast=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
elif(msg['cmd'] == 'wiseloff'):
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(False)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
vars.worldinfo[msg['data']]["selective"] = False
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
emit('from_server', {'cmd': 'wiseloff', 'data': msg['data']}, broadcast=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
elif(msg['cmd'] == 'wiconstanton'):
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(False)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
vars.worldinfo[msg['data']]["constant"] = True
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
emit('from_server', {'cmd': 'wiconstanton', 'data': msg['data']}, broadcast=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
elif(msg['cmd'] == 'wiconstantoff'):
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(False)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
vars.worldinfo[msg['data']]["constant"] = False
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
emit('from_server', {'cmd': 'wiconstantoff', 'data': msg['data']}, broadcast=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
elif(msg['cmd'] == 'sendwilist'):
commitwi(msg['data'])
elif(msg['cmd'] == 'aidgimport'):
importAidgRequest(msg['data'])
elif(msg['cmd'] == 'saveasrequest'):
saveas(msg['data'])
elif(msg['cmd'] == 'saverequest'):
save()
elif(msg['cmd'] == 'loadlistrequest'):
getloadlist()
elif(msg['cmd'] == 'splistrequest'):
getsplist()
Add Userscripts menu into GUI
2021-12-13 07:03:26 +01:00
elif(msg['cmd'] == 'uslistrequest'):
Display icons for active userscripts and softprompts Also fixes the userscript menu so that the active userscripts preserve the previously selected order as was originally intended.
2021-12-23 05:33:27 +01:00
unloaded, loaded = getuslist()
emit('from_server', {'cmd': 'buildus', 'data': {"unloaded": unloaded, "loaded": loaded}})
Add Userscripts menu into GUI
2021-12-13 07:03:26 +01:00
elif(msg['cmd'] == 'usloaded'):
vars.userscripts = []
for userscript in msg['data']:
if type(userscript) is not str:
continue
userscript = userscript.strip()
if len(userscript) != 0 and all(q not in userscript for q in ("..", ":")) and all(userscript[0] not in q for q in ("/", "\\")) and os.path.exists(fileops.uspath(userscript)):
vars.userscripts.append(userscript)
settingschanged()
elif(msg['cmd'] == 'usload'):
load_lua_scripts()
Display icons for active userscripts and softprompts Also fixes the userscript menu so that the active userscripts preserve the previously selected order as was originally intended.
2021-12-23 05:33:27 +01:00
unloaded, loaded = getuslist()
sendUSStatItems()
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
elif(msg['cmd'] == 'loadselect'):
vars.loadselect = msg["data"]
elif(msg['cmd'] == 'spselect'):
vars.spselect = msg["data"]
elif(msg['cmd'] == 'loadrequest'):
loadRequest(fileops.storypath(vars.loadselect))
elif(msg['cmd'] == 'sprequest'):
spRequest(vars.spselect)
Save softprompt filename in settings
2021-12-23 19:02:11 +01:00
emit('from_server', {'cmd': 'spstatitems', 'data': {vars.spfilename: vars.spmeta} if vars.allowsp and len(vars.spfilename) else {}}, broadcast=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
elif(msg['cmd'] == 'deletestory'):
deletesave(msg['data'])
elif(msg['cmd'] == 'renamestory'):
renamesave(msg['data'], msg['newname'])
elif(msg['cmd'] == 'clearoverwrite'):
vars.svowname = ""
vars.saveow = False
elif(msg['cmd'] == 'seqsel'):
selectsequence(msg['data'])
elif(msg['cmd'] == 'setnumseq'):
vars.numseqs = int(msg['data'])
emit('from_server', {'cmd': 'setlabelnumseq', 'data': msg['data']})
settingschanged()
refresh_settings()
elif(msg['cmd'] == 'setwidepth'):
vars.widepth = int(msg['data'])
emit('from_server', {'cmd': 'setlabelwidepth', 'data': msg['data']})
settingschanged()
refresh_settings()
elif(msg['cmd'] == 'setuseprompt'):
vars.useprompt = msg['data']
settingschanged()
refresh_settings()
elif(msg['cmd'] == 'setadventure'):
vars.adventure = msg['data']
Chatmode improvements Blank lines appear often in chatmode so it is best played with blank line removal turned on, this is now forced. Its not compatible with Adventure mode, so they now turn each other off.
2021-12-27 13:32:25 +01:00
vars.chatmode = False
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
settingschanged()
refresh_settings()
Added autosave feature. When action is submitted it will save if the save setting is on and if the filename is set.
2022-01-20 13:46:34 +01:00
elif(msg['cmd'] == 'autosave'):
vars.autosave = msg['data']
settingschanged()
refresh_settings()
Chat Mode The Initial commit for Chat Mode, the nickname part of the UI is missing other than that it should be fully functional. To use Chat Mode effectively you first input a small dialogue (Can be around 6 lines 3 of your own inputs and 3 of the character) formatted as Name : it will then automate the actions needed to chat properly. During this mode single line mode is forced on, and Trim Incomplete Sentences is forced off.
2021-12-26 01:51:32 +01:00
elif(msg['cmd'] == 'setchatmode'):
vars.chatmode = msg['data']
Chatmode improvements Blank lines appear often in chatmode so it is best played with blank line removal turned on, this is now forced. Its not compatible with Adventure mode, so they now turn each other off.
2021-12-27 13:32:25 +01:00
vars.adventure = False
Chat Mode The Initial commit for Chat Mode, the nickname part of the UI is missing other than that it should be fully functional. To use Chat Mode effectively you first input a small dialogue (Can be around 6 lines 3 of your own inputs and 3 of the character) formatted as Name : it will then automate the actions needed to chat properly. During this mode single line mode is forced on, and Trim Incomplete Sentences is forced off.
2021-12-26 01:51:32 +01:00
settingschanged()
refresh_settings()
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
elif(msg['cmd'] == 'setdynamicscan'):
vars.dynamicscan = msg['data']
settingschanged()
refresh_settings()
No Prompt Gen Allow people to enter a prompt without generating anything by the AI. Combined with the always add prompt this is a very useful feature that allows people to write world information first, and then do a specific action. This mimics the behavior previously seen in AI Dungeon forks where it prompts for world information and then asks an action and can be particularly useful for people who want the prompt to always be part of the generation.
2021-12-16 12:47:44 +01:00
elif(msg['cmd'] == 'setnopromptgen'):
vars.nopromptgen = msg['data']
settingschanged()
refresh_settings()
Add Memory box to Random Story dialog and "Random Story Persist"
2021-12-30 05:15:59 +01:00
elif(msg['cmd'] == 'setrngpersist'):
vars.rngpersist = msg['data']
settingschanged()
refresh_settings()
Use original TPU backend if possible
2022-01-16 05:31:07 +01:00
elif(msg['cmd'] == 'setnogenmod'):
vars.nogenmod = msg['data']
settingschanged()
refresh_settings()
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
elif(not vars.remote and msg['cmd'] == 'importwi'):
wiimportrequest()
Display icons for active userscripts and softprompts Also fixes the userscript menu so that the active userscripts preserve the previously selected order as was originally intended.
2021-12-23 05:33:27 +01:00
#==================================================================#
# Send userscripts list to client
#==================================================================#
def sendUSStatItems():
_, loaded = getuslist()
loaded = loaded if vars.lua_running else []
last_userscripts = [e["filename"] for e in loaded]
emit('from_server', {'cmd': 'usstatitems', 'data': loaded, 'flash': last_userscripts != vars.last_userscripts}, broadcast=True)
vars.last_userscripts = last_userscripts
Welcome Message API Allows model creators to customize the welcome message using Markdown and Limited HTML Existing United users need to run install_requirements..bat again, you can leave the existing dependencies intact.
2022-01-30 19:47:30 +01:00
#==================================================================#
# KoboldAI Markup Formatting (Mixture of Markdown and sanitized html)
#==================================================================#
def kml(txt):
KML improvements Don't parse > since that has a different meaning for us, also whitelisting a few more markdown tags so lists work.
2022-01-30 20:07:47 +01:00
txt = txt.replace('>', '&gt;')
txt = bleach.clean(markdown.markdown(txt), tags = ['p', 'em', 'strong', 'code', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'li', 'ul', 'b', 'i', 'a', 'span', 'button'], styles = ['color', 'font-weight'], attributes=['id', 'class', 'style', 'href'])
Welcome Message API Allows model creators to customize the welcome message using Markdown and Limited HTML Existing United users need to run install_requirements..bat again, you can leave the existing dependencies intact.
2022-01-30 19:47:30 +01:00
return txt
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#==================================================================#
# Send start message and tell Javascript to set UI state
#==================================================================#
def setStartState():
Welcome Message API Allows model creators to customize the welcome message using Markdown and Limited HTML Existing United users need to run install_requirements..bat again, you can leave the existing dependencies intact.
2022-01-30 19:47:30 +01:00
if(vars.welcome):
txt = kml(vars.welcome) + "<br/>"
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
else:
Welcome Message API Allows model creators to customize the welcome message using Markdown and Limited HTML Existing United users need to run install_requirements..bat again, you can leave the existing dependencies intact.
2022-01-30 19:47:30 +01:00
txt = "<span>Welcome to <span class=\"color_cyan\">KoboldAI</span>! You are running <span class=\"color_green\">"+getmodelname()+"</span>.<br/>"
if(not vars.noai and not vars.welcome):
txt = txt + "Please load a game or enter a prompt below to begin!</span>"
if(vars.noai):
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
txt = txt + "Please load or import a story to read. There is no AI in this mode."
emit('from_server', {'cmd': 'updatescreen', 'gamestarted': vars.gamestarted, 'data': txt}, broadcast=True)
emit('from_server', {'cmd': 'setgamestate', 'data': 'start'}, broadcast=True)
#==================================================================#
# Transmit applicable settings to SocketIO to build UI sliders/toggles
#==================================================================#
def sendsettings():
# Send settings for selected AI type
if(vars.model != "InferKit"):
for set in gensettings.gensettingstf:
emit('from_server', {'cmd': 'addsetting', 'data': set})
else:
for set in gensettings.gensettingsik:
emit('from_server', {'cmd': 'addsetting', 'data': set})
# Send formatting options
for frm in gensettings.formatcontrols:
emit('from_server', {'cmd': 'addformat', 'data': frm})
# Add format key to vars if it wasn't loaded with client.settings
if(not frm["id"] in vars.formatoptns):
vars.formatoptns[frm["id"]] = False;
#==================================================================#
# Take settings from vars and write them to client settings file
#==================================================================#
def savesettings():
# Build json to write
js = {}
js["apikey"] = vars.apikey
js["andepth"] = vars.andepth
js["temp"] = vars.temp
js["top_p"] = vars.top_p
js["top_k"] = vars.top_k
js["tfs"] = vars.tfs
js["rep_pen"] = vars.rep_pen
Repetition penalty slope and range
2022-01-24 21:30:38 +01:00
js["rep_pen_slope"] = vars.rep_pen_slope
js["rep_pen_range"] = vars.rep_pen_range
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
js["genamt"] = vars.genamt
js["max_length"] = vars.max_length
js["ikgen"] = vars.ikgen
js["formatoptns"] = vars.formatoptns
js["numseqs"] = vars.numseqs
js["widepth"] = vars.widepth
js["useprompt"] = vars.useprompt
js["adventure"] = vars.adventure
Chat Mode The Initial commit for Chat Mode, the nickname part of the UI is missing other than that it should be fully functional. To use Chat Mode effectively you first input a small dialogue (Can be around 6 lines 3 of your own inputs and 3 of the character) formatted as Name : it will then automate the actions needed to chat properly. During this mode single line mode is forced on, and Trim Incomplete Sentences is forced off.
2021-12-26 01:51:32 +01:00
js["chatmode"] = vars.chatmode
Store chatname Also save the chatname in the settings for later re-use by the user.
2021-12-26 01:55:27 +01:00
js["chatname"] = vars.chatname
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
js["dynamicscan"] = vars.dynamicscan
No Prompt Gen Allow people to enter a prompt without generating anything by the AI. Combined with the always add prompt this is a very useful feature that allows people to write world information first, and then do a specific action. This mimics the behavior previously seen in AI Dungeon forks where it prompts for world information and then asks an action and can be particularly useful for people who want the prompt to always be part of the generation.
2021-12-16 12:47:44 +01:00
js["nopromptgen"] = vars.nopromptgen
Add Memory box to Random Story dialog and "Random Story Persist"
2021-12-30 05:15:59 +01:00
js["rngpersist"] = vars.rngpersist
Use original TPU backend if possible
2022-01-16 05:31:07 +01:00
js["nogenmod"] = vars.nogenmod
Added autosave feature. When action is submitted it will save if the save setting is on and if the filename is set.
2022-01-20 13:46:34 +01:00
js["autosave"] = vars.autosave
Save model values Without saving these they get lost after someone saves. So saving them is more important than the model being able to override them after the fact.
2022-02-01 18:37:52 +01:00
js["welcome"] = vars.welcome
js["newlinemode"] = vars.newlinemode
Use original TPU backend if possible
2022-01-16 05:31:07 +01:00
Author's Note Template
2021-12-30 05:43:36 +01:00
js["antemplate"] = vars.setauthornotetemplate
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
Add Userscripts menu into GUI
2021-12-13 07:03:26 +01:00
js["userscripts"] = vars.userscripts
Add a "corescript" setting
2021-12-14 01:28:33 +01:00
js["corescript"] = vars.corescript
Save softprompt filename in settings
2021-12-23 19:02:11 +01:00
js["softprompt"] = vars.spfilename
Add Userscripts menu into GUI
2021-12-13 07:03:26 +01:00
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Write it
if not os.path.exists('settings'):
os.mkdir('settings')
Replace slashes in model name with underscores
2021-11-17 21:36:36 +01:00
file = open("settings/" + getmodelname().replace('/', '_') + ".settings", "w")
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
try:
file.write(json.dumps(js, indent=3))
finally:
file.close()
#==================================================================#
# Read settings from client file JSON and send to vars
#==================================================================#
def loadsettings():
Replace slashes in model name with underscores
2021-11-17 21:36:36 +01:00
if(path.exists("settings/" + getmodelname().replace('/', '_') + ".settings")):
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Read file contents into JSON object
Replace slashes in model name with underscores
2021-11-17 21:36:36 +01:00
file = open("settings/" + getmodelname().replace('/', '_') + ".settings", "r")
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
js = json.load(file)
# Copy file contents to vars
if("apikey" in js):
vars.apikey = js["apikey"]
if("andepth" in js):
vars.andepth = js["andepth"]
if("temp" in js):
vars.temp = js["temp"]
if("top_p" in js):
vars.top_p = js["top_p"]
if("top_k" in js):
vars.top_k = js["top_k"]
if("tfs" in js):
vars.tfs = js["tfs"]
if("rep_pen" in js):
vars.rep_pen = js["rep_pen"]
Repetition penalty slope and range
2022-01-24 21:30:38 +01:00
if("rep_pen_slope" in js):
vars.rep_pen_slope = js["rep_pen_slope"]
if("rep_pen_range" in js):
vars.rep_pen_range = js["rep_pen_range"]
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if("genamt" in js):
vars.genamt = js["genamt"]
if("max_length" in js):
vars.max_length = js["max_length"]
if("ikgen" in js):
vars.ikgen = js["ikgen"]
if("formatoptns" in js):
vars.formatoptns = js["formatoptns"]
if("numseqs" in js):
vars.numseqs = js["numseqs"]
if("widepth" in js):
vars.widepth = js["widepth"]
if("useprompt" in js):
vars.useprompt = js["useprompt"]
if("adventure" in js):
vars.adventure = js["adventure"]
Chat Mode The Initial commit for Chat Mode, the nickname part of the UI is missing other than that it should be fully functional. To use Chat Mode effectively you first input a small dialogue (Can be around 6 lines 3 of your own inputs and 3 of the character) formatted as Name : it will then automate the actions needed to chat properly. During this mode single line mode is forced on, and Trim Incomplete Sentences is forced off.
2021-12-26 01:51:32 +01:00
if("chatmode" in js):
vars.chatmode = js["chatmode"]
Store chatname Also save the chatname in the settings for later re-use by the user.
2021-12-26 01:55:27 +01:00
if("chatname" in js):
Update aiserver.py
2021-12-26 02:01:57 +01:00
vars.chatname = js["chatname"]
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if("dynamicscan" in js):
vars.dynamicscan = js["dynamicscan"]
No Prompt Gen Allow people to enter a prompt without generating anything by the AI. Combined with the always add prompt this is a very useful feature that allows people to write world information first, and then do a specific action. This mimics the behavior previously seen in AI Dungeon forks where it prompts for world information and then asks an action and can be particularly useful for people who want the prompt to always be part of the generation.
2021-12-16 12:47:44 +01:00
if("nopromptgen" in js):
vars.nopromptgen = js["nopromptgen"]
Add Memory box to Random Story dialog and "Random Story Persist"
2021-12-30 05:15:59 +01:00
if("rngpersist" in js):
vars.rngpersist = js["rngpersist"]
Use original TPU backend if possible
2022-01-16 05:31:07 +01:00
if("nogenmod" in js):
vars.nogenmod = js["nogenmod"]
Added autosave feature. When action is submitted it will save if the save setting is on and if the filename is set.
2022-01-20 13:46:34 +01:00
if("autosave" in js):
vars.autosave = js["autosave"]
</s> new line mode Needed for Fairseq and XGLM models that do not understand the regular \n .
2022-01-31 18:39:34 +01:00
if("newlinemode" in js):
vars.newlinemode = js["newlinemode"]
Further Polishing Multiple smaller changes to get 1.17 in shape for its release.
2022-02-01 11:15:44 +01:00
if("welcome" in js):
vars.welcome = js["welcome"]
Author's Note Template
2021-12-30 05:43:36 +01:00
if("antemplate" in js):
vars.setauthornotetemplate = js["antemplate"]
if(not vars.gamestarted):
vars.authornotetemplate = vars.setauthornotetemplate
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
Add Userscripts menu into GUI
2021-12-13 07:03:26 +01:00
if("userscripts" in js):
vars.userscripts = []
for userscript in js["userscripts"]:
if type(userscript) is not str:
continue
userscript = userscript.strip()
if len(userscript) != 0 and all(q not in userscript for q in ("..", ":")) and all(userscript[0] not in q for q in ("/", "\\")) and os.path.exists(fileops.uspath(userscript)):
vars.userscripts.append(userscript)
Add a "corescript" setting
2021-12-14 01:28:33 +01:00
if("corescript" in js and type(js["corescript"]) is str and all(q not in js["corescript"] for q in ("..", ":")) and all(js["corescript"][0] not in q for q in ("/", "\\"))):
vars.corescript = js["corescript"]
else:
vars.corescript = "default.lua"
Add Userscripts menu into GUI
2021-12-13 07:03:26 +01:00
Update aiserver.py Crash fix
2021-12-24 19:57:08 +01:00
if(vars.allowsp and "softprompt" in js and type(js["softprompt"]) is str and all(q not in js["softprompt"] for q in ("..", ":")) and (len(js["softprompt"]) == 0 or all(js["softprompt"][0] not in q for q in ("/", "\\")))):
Fix the behaviour of the `softprompt` setting
2021-12-23 22:14:09 +01:00
spRequest(js["softprompt"])
Save softprompt filename in settings
2021-12-23 19:02:11 +01:00
else:
vars.spfilename = ""
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
file.close()
#==================================================================#
# Don't save settings unless 2 seconds have passed without modification
#==================================================================#
@debounce(2)
def settingschanged():
print("{0}Saving settings!{1}".format(colors.GREEN, colors.END))
savesettings()
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
#==================================================================#
# Set value of gamesaved
#==================================================================#
def setgamesaved(gamesaved):
assert type(gamesaved) is bool
if(gamesaved != vars.gamesaved):
emit('from_server', {'cmd': 'gamesaved', 'data': gamesaved}, broadcast=True)
vars.gamesaved = gamesaved
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#==================================================================#
# Take input text from SocketIO and decide what to do with it
#==================================================================#
Show message when TPU backend is compiling
2022-01-17 03:09:10 +01:00
def check_for_backend_compilation():
if(vars.checking):
return
vars.checking = True
for _ in range(31):
time.sleep(0.06276680299820175)
if(vars.compiling):
emit('from_server', {'cmd': 'warnmsg', 'data': 'Compiling TPU backend&mdash;this usually takes 1&ndash;2 minutes...'}, broadcast=True)
break
vars.checking = False
Allow retry button to regenerate random story
2021-12-27 01:52:56 +01:00
def actionsubmit(data, actionmode=0, force_submit=False, force_prompt_gen=False, disable_recentrng=False):
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Ignore new submissions if the AI is currently busy
if(vars.aibusy):
return
</s> new line mode Needed for Fairseq and XGLM models that do not understand the regular \n .
2022-01-31 18:39:34 +01:00
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
while(True):
set_aibusy(1)
Allow retry button to regenerate random story
2021-12-27 01:52:56 +01:00
if(disable_recentrng):
Retrying random story generation now also remembers memory
2022-01-04 20:40:10 +01:00
vars.recentrng = vars.recentrngm = None
Allow retry button to regenerate random story
2021-12-27 01:52:56 +01:00
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
vars.recentback = False
vars.recentedit = False
vars.actionmode = actionmode
# "Action" mode
if(actionmode == 1):
data = data.strip().lstrip('>')
data = re.sub(r'\n+', ' ', data)
if(len(data)):
data = f"\n\n> {data}\n"
Chat Mode The Initial commit for Chat Mode, the nickname part of the UI is missing other than that it should be fully functional. To use Chat Mode effectively you first input a small dialogue (Can be around 6 lines 3 of your own inputs and 3 of the character) formatted as Name : it will then automate the actions needed to chat properly. During this mode single line mode is forced on, and Trim Incomplete Sentences is forced off.
2021-12-26 01:51:32 +01:00
# "Chat" mode
if(vars.chatmode and vars.gamestarted):
data = re.sub(r'\n+', ' ', data)
if(len(data)):
data = f"\n{vars.chatname} : {data}\n"
Increase newlinemode compatibility Ran into issues with other modes like chatmode and adventure, moved it further down the pipeline and converting </s> back to \n before processing additional formatting. Still has an issue with the html formatting not working, but at least the AI works now.
2022-01-31 19:39:32 +01:00
# </s> mode
if(vars.newlinemode == "s"):
data = data.replace('\n', "</s>")
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
# If we're not continuing, store a copy of the raw input
if(data != ""):
vars.lastact = data
if(not vars.gamestarted):
vars.submission = data
execute_inmod()
data = vars.submission
if(not force_submit and len(data.strip()) == 0):
assert False
# Start the game
vars.gamestarted = True
Fix random story generator when No Prompt Gen is enabled
2021-12-27 01:40:20 +01:00
if(not vars.noai and vars.lua_koboldbridge.generating and (not vars.nopromptgen or force_prompt_gen)):
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
# Save this first action as the prompt
vars.prompt = data
# Clear the startup text from game screen
emit('from_server', {'cmd': 'updatescreen', 'gamestarted': False, 'data': 'Please wait, generating story...'}, broadcast=True)
calcsubmit(data) # Run the first action through the generator
Allow pressing send button again to stop generation
2022-01-10 22:36:15 +01:00
if(not vars.abort and vars.lua_koboldbridge.restart_sequence is not None and len(vars.genseqs) == 0):
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
data = ""
force_submit = True
Allow retry button to regenerate random story
2021-12-27 01:52:56 +01:00
disable_recentrng = True
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
continue
emit('from_server', {'cmd': 'scrolldown', 'data': ''}, broadcast=True)
break
else:
# Save this first action as the prompt
Allow pressing send button again to stop generation
2022-01-10 22:36:15 +01:00
vars.prompt = data if len(data) > 0 else '"'
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
for i in range(vars.numseqs):
vars.lua_koboldbridge.outputs[i+1] = ""
execute_outmod()
Add API for generated tokens and output text
2021-12-13 01:27:20 +01:00
vars.lua_koboldbridge.regeneration_required = False
genout = []
for i in range(vars.numseqs):
genout.append({"generated_text": vars.lua_koboldbridge.outputs[i+1]})
assert type(genout[-1]["generated_text"]) is str
if(len(genout) == 1):
Fix AI output text flash showing up on wrong chunk
2021-12-29 20:23:22 +01:00
genresult(genout[0]["generated_text"], flash=False)
refresh_story()
if(len(vars.actions) > 0):
emit('from_server', {'cmd': 'texteffect', 'data': vars.actions.get_last_key() + 1}, broadcast=True)
Allow pressing send button again to stop generation
2022-01-10 22:36:15 +01:00
if(not vars.abort and vars.lua_koboldbridge.restart_sequence is not None):
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
data = ""
force_submit = True
Allow retry button to regenerate random story
2021-12-27 01:52:56 +01:00
disable_recentrng = True
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
continue
Add API for generated tokens and output text
2021-12-13 01:27:20 +01:00
else:
Allow pressing send button again to stop generation
2022-01-10 22:36:15 +01:00
if(not vars.abort and vars.lua_koboldbridge.restart_sequence is not None and vars.lua_koboldbridge.restart_sequence > 0):
Fix AI output text flash showing up on wrong chunk
2021-12-29 20:23:22 +01:00
genresult(genout[vars.lua_koboldbridge.restart_sequence-1]["generated_text"], flash=False)
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
refresh_story()
data = ""
force_submit = True
Allow retry button to regenerate random story
2021-12-27 01:52:56 +01:00
disable_recentrng = True
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
continue
Add API for generated tokens and output text
2021-12-13 01:27:20 +01:00
genselect(genout)
Fix AI output text flash showing up on wrong chunk
2021-12-29 20:23:22 +01:00
refresh_story()
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
set_aibusy(0)
emit('from_server', {'cmd': 'scrolldown', 'data': ''}, broadcast=True)
break
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
else:
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
# Apply input formatting & scripts before sending to tokenizer
if(vars.actionmode == 0):
data = applyinputformatting(data)
vars.submission = data
execute_inmod()
data = vars.submission
# Dont append submission if it's a blank/continue action
if(data != ""):
# Store the result in the Action log
if(len(vars.prompt.strip()) == 0):
vars.prompt = data
else:
vars.actions.append(data)
update_story_chunk('last')
if(not vars.noai and vars.lua_koboldbridge.generating):
# Off to the tokenizer!
calcsubmit(data)
Allow pressing send button again to stop generation
2022-01-10 22:36:15 +01:00
if(not vars.abort and vars.lua_koboldbridge.restart_sequence is not None and len(vars.genseqs) == 0):
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
data = ""
force_submit = True
Allow retry button to regenerate random story
2021-12-27 01:52:56 +01:00
disable_recentrng = True
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
continue
emit('from_server', {'cmd': 'scrolldown', 'data': ''}, broadcast=True)
break
else:
for i in range(vars.numseqs):
vars.lua_koboldbridge.outputs[i+1] = ""
execute_outmod()
Add API for generated tokens and output text
2021-12-13 01:27:20 +01:00
vars.lua_koboldbridge.regeneration_required = False
genout = []
for i in range(vars.numseqs):
genout.append({"generated_text": vars.lua_koboldbridge.outputs[i+1]})
assert type(genout[-1]["generated_text"]) is str
if(len(genout) == 1):
genresult(genout[0]["generated_text"])
Allow pressing send button again to stop generation
2022-01-10 22:36:15 +01:00
if(not vars.abort and vars.lua_koboldbridge.restart_sequence is not None):
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
data = ""
force_submit = True
Allow retry button to regenerate random story
2021-12-27 01:52:56 +01:00
disable_recentrng = True
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
continue
Add API for generated tokens and output text
2021-12-13 01:27:20 +01:00
else:
Allow pressing send button again to stop generation
2022-01-10 22:36:15 +01:00
if(not vars.abort and vars.lua_koboldbridge.restart_sequence is not None and vars.lua_koboldbridge.restart_sequence > 0):
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
genresult(genout[vars.lua_koboldbridge.restart_sequence-1]["generated_text"])
data = ""
force_submit = True
Allow retry button to regenerate random story
2021-12-27 01:52:56 +01:00
disable_recentrng = True
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
continue
Add API for generated tokens and output text
2021-12-13 01:27:20 +01:00
genselect(genout)
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
set_aibusy(0)
emit('from_server', {'cmd': 'scrolldown', 'data': ''}, broadcast=True)
break
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#==================================================================#
#
#==================================================================#
def actionretry(data):
if(vars.noai):
emit('from_server', {'cmd': 'errmsg', 'data': "Retry function unavailable in Read Only mode."})
return
if(vars.aibusy):
return
Allow retry button to regenerate random story
2021-12-27 01:52:56 +01:00
if(vars.recentrng is not None):
Retrying random story generation now also remembers memory
2022-01-04 20:40:10 +01:00
randomGameRequest(vars.recentrng, memory=vars.recentrngm)
Allow retry button to regenerate random story
2021-12-27 01:52:56 +01:00
return
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Remove last action if possible and resubmit
if(vars.gamestarted if vars.useprompt else len(vars.actions) > 0):
if(not vars.recentback and len(vars.actions) != 0 and len(vars.genseqs) == 0): # Don't pop if we're in the "Select sequence to keep" menu or if there are no non-prompt actions
last_key = vars.actions.get_last_key()
vars.actions.pop()
remove_story_chunk(last_key + 1)
vars.recentback = False
vars.recentedit = False
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
vars.lua_koboldbridge.feedback = None
actionsubmit("", actionmode=vars.actionmode, force_submit=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
elif(not vars.useprompt):
emit('from_server', {'cmd': 'errmsg', 'data': "Please enable \"Always Add Prompt\" to retry with your prompt."})
#==================================================================#
#
#==================================================================#
def actionback():
if(vars.aibusy):
return
# Remove last index of actions and refresh game screen
if(len(vars.genseqs) == 0 and len(vars.actions) > 0):
last_key = vars.actions.get_last_key()
vars.actions.pop()
vars.recentback = True
remove_story_chunk(last_key + 1)
elif(len(vars.genseqs) == 0):
emit('from_server', {'cmd': 'errmsg', 'data': "Cannot delete the prompt."})
else:
vars.genseqs = []
#==================================================================#
#
#==================================================================#
def calcsubmitbudgetheader(txt, **kwargs):
# Scan for WorldInfo matches
winfo, found_entries = checkworldinfo(txt, **kwargs)
# Add a newline to the end of memory
if(vars.memory != "" and vars.memory[-1] != "\n"):
mem = vars.memory + "\n"
else:
mem = vars.memory
# Build Author's Note if set
if(vars.authornote != ""):
Author's Note Template
2021-12-30 05:43:36 +01:00
anotetxt = ("\n" + vars.authornotetemplate + "\n").replace("<|>", vars.authornote)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
else:
anotetxt = ""
return winfo, mem, anotetxt, found_entries
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
def calcsubmitbudget(actionlen, winfo, mem, anotetxt, actions, submission=None, budget_deduction=0):
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
forceanote = False # In case we don't have enough actions to hit A.N. depth
anoteadded = False # In case our budget runs out before we hit A.N. depth
anotetkns = [] # Placeholder for Author's Note tokens
lnanote = 0 # Placeholder for Author's Note length
Fix soft prompt length calculation in `calcsubmitbudget()` In TPU instances, `vars.sp.shape[0]` is not always the actual number of tokens in the soft prompt. We have to use `vars.sp_length` to get an accurate token count.
2022-01-17 19:17:20 +01:00
lnsp = vars.sp_length
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
Make sure tokenizer is initialized when used in read-only mode
2021-12-31 23:13:11 +01:00
if("tokenizer" not in globals()):
from transformers import GPT2TokenizerFast
global tokenizer
tokenizer = GPT2TokenizerFast.from_pretrained("gpt2", cache_dir="cache/")
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Calculate token budget
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
prompttkns = tokenizer.encode(vars.comregex_ai.sub('', vars.prompt), max_length=int(2e9), truncation=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
lnprompt = len(prompttkns)
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
memtokens = tokenizer.encode(mem, max_length=int(2e9), truncation=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
lnmem = len(memtokens)
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
if(lnmem > vars.max_length - lnsp - vars.genamt - budget_deduction):
raise OverflowError("The memory in your story is too long. Please either write a shorter memory text or increase the Max Tokens setting. If you are using a soft prompt, additionally consider using a smaller soft prompt.")
witokens = tokenizer.encode(winfo, max_length=int(2e9), truncation=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
lnwi = len(witokens)
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
if(lnmem + lnwi > vars.max_length - lnsp - vars.genamt - budget_deduction):
raise OverflowError("The current active world info keys take up too many tokens. Please either write shorter world info, decrease World Info Depth or increase the Max Tokens setting. If you are using a soft prompt, additionally consider using a smaller soft prompt.")
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if(anotetxt != ""):
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
anotetkns = tokenizer.encode(anotetxt, max_length=int(2e9), truncation=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
lnanote = len(anotetkns)
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
if(lnmem + lnwi + lnanote > vars.max_length - lnsp - vars.genamt - budget_deduction):
raise OverflowError("The author's note in your story is too long. Please either write a shorter author's note or increase the Max Tokens setting. If you are using a soft prompt, additionally consider using a smaller soft prompt.")
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if(vars.useprompt):
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
budget = vars.max_length - lnsp - lnprompt - lnmem - lnanote - lnwi - vars.genamt - budget_deduction
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
else:
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
budget = vars.max_length - lnsp - lnmem - lnanote - lnwi - vars.genamt - budget_deduction
lnsubmission = len(tokenizer.encode(vars.comregex_ai.sub('', submission), max_length=int(2e9), truncation=True)) if submission is not None else 0
maybe_lnprompt = lnprompt if vars.useprompt and actionlen > 0 else 0
if(lnmem + lnwi + lnanote + maybe_lnprompt + lnsubmission > vars.max_length - lnsp - vars.genamt - budget_deduction):
raise OverflowError("Your submission is too long. Please either write a shorter submission or increase the Max Tokens setting. If you are using a soft prompt, additionally consider using a smaller soft prompt. If you are using the Always Add Prompt setting, turning it off may help.")
assert budget >= 0
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if(actionlen == 0):
# First/Prompt action
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
tokens = memtokens + witokens + anotetkns + prompttkns
assert len(tokens) <= vars.max_length - lnsp - vars.genamt - budget_deduction
ln = len(tokens) + lnsp
return tokens, ln+1, ln+vars.genamt
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
else:
tokens = []
# Check if we have the action depth to hit our A.N. depth
if(anotetxt != "" and actionlen < vars.andepth):
forceanote = True
# Get most recent action tokens up to our budget
n = 0
for key in reversed(actions):
Add syntax for comments <|...|>
2021-11-20 07:27:57 +01:00
chunk = vars.comregex_ai.sub('', actions[key])
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
assert budget >= 0
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if(budget <= 0):
break
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
acttkns = tokenizer.encode(chunk, max_length=int(2e9), truncation=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
tknlen = len(acttkns)
if(tknlen < budget):
tokens = acttkns + tokens
budget -= tknlen
else:
count = budget * -1
tokens = acttkns[count:] + tokens
budget = 0
break
# Inject Author's Note if we've reached the desired depth
if(n == vars.andepth-1):
if(anotetxt != ""):
tokens = anotetkns + tokens # A.N. len already taken from bdgt
anoteadded = True
n += 1
# If we're not using the prompt every time and there's still budget left,
# add some prompt.
if(not vars.useprompt):
if(budget > 0):
prompttkns = prompttkns[-budget:]
else:
prompttkns = []
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Did we get to add the A.N.? If not, do it here
if(anotetxt != ""):
if((not anoteadded) or forceanote):
tokens = memtokens + witokens + anotetkns + prompttkns + tokens
else:
tokens = memtokens + witokens + prompttkns + tokens
else:
# Prepend Memory, WI, and Prompt before action tokens
tokens = memtokens + witokens + prompttkns + tokens
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Send completed bundle to generator
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
assert len(tokens) <= vars.max_length - lnsp - vars.genamt - budget_deduction
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
ln = len(tokens) + lnsp
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
return tokens, ln+1, ln+vars.genamt
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#==================================================================#
# Take submitted text and build the text to be given to generator
#==================================================================#
def calcsubmit(txt):
anotetxt = "" # Placeholder for Author's Note text
forceanote = False # In case we don't have enough actions to hit A.N. depth
anoteadded = False # In case our budget runs out before we hit A.N. depth
actionlen = len(vars.actions)
winfo, mem, anotetxt, found_entries = calcsubmitbudgetheader(txt)
</s> new line mode Needed for Fairseq and XGLM models that do not understand the regular \n .
2022-01-31 18:39:34 +01:00
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# For all transformers models
if(vars.model != "InferKit"):
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
subtxt, min, max = calcsubmitbudget(actionlen, winfo, mem, anotetxt, vars.actions, submission=txt)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if(actionlen == 0):
Integrate TPU backend This commit puts the TPU backend code directly in to the KoboldAI code to make it easier to modify.
2021-11-20 00:06:57 +01:00
if(not vars.model in ["Colab", "OAI", "TPUMeshTransformerGPTJ"]):
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
generate(subtxt, min, max, found_entries=found_entries)
elif(vars.model == "Colab"):
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
sendtocolab(tokenizer.decode(subtxt), min, max)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
elif(vars.model == "OAI"):
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
oairequest(tokenizer.decode(subtxt), min, max)
Integrate TPU backend This commit puts the TPU backend code directly in to the KoboldAI code to make it easier to modify.
2021-11-20 00:06:57 +01:00
elif(vars.model == "TPUMeshTransformerGPTJ"):
tpumtjgenerate(subtxt, min, max, found_entries=found_entries)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
else:
Integrate TPU backend This commit puts the TPU backend code directly in to the KoboldAI code to make it easier to modify.
2021-11-20 00:06:57 +01:00
if(not vars.model in ["Colab", "OAI", "TPUMeshTransformerGPTJ"]):
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
generate(subtxt, min, max, found_entries=found_entries)
elif(vars.model == "Colab"):
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
sendtocolab(tokenizer.decode(subtxt), min, max)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
elif(vars.model == "OAI"):
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
oairequest(tokenizer.decode(subtxt), min, max)
Integrate TPU backend This commit puts the TPU backend code directly in to the KoboldAI code to make it easier to modify.
2021-11-20 00:06:57 +01:00
elif(vars.model == "TPUMeshTransformerGPTJ"):
tpumtjgenerate(subtxt, min, max, found_entries=found_entries)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# For InferKit web API
else:
# Check if we have the action depth to hit our A.N. depth
if(anotetxt != "" and actionlen < vars.andepth):
forceanote = True
if(vars.useprompt):
Remove comments from prompt in WI processor and InferKit mode
2021-11-21 04:23:06 +01:00
budget = vars.ikmax - len(vars.comregex_ai.sub('', vars.prompt)) - len(anotetxt) - len(mem) - len(winfo) - 1
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
else:
budget = vars.ikmax - len(anotetxt) - len(mem) - len(winfo) - 1
subtxt = ""
Remove comments from prompt in WI processor and InferKit mode
2021-11-21 04:23:06 +01:00
prompt = vars.comregex_ai.sub('', vars.prompt)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
n = 0
for key in reversed(vars.actions):
chunk = vars.actions[key]
if(budget <= 0):
break
actlen = len(chunk)
if(actlen < budget):
subtxt = chunk + subtxt
budget -= actlen
else:
count = budget * -1
subtxt = chunk[count:] + subtxt
budget = 0
break
# If we're not using the prompt every time and there's still budget left,
# add some prompt.
if(not vars.useprompt):
if(budget > 0):
Remove comments from prompt in WI processor and InferKit mode
2021-11-21 04:23:06 +01:00
prompt = vars.comregex_ai.sub('', vars.prompt)[-budget:]
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
else:
prompt = ""
# Inject Author's Note if we've reached the desired depth
if(n == vars.andepth-1):
if(anotetxt != ""):
subtxt = anotetxt + subtxt # A.N. len already taken from bdgt
anoteadded = True
n += 1
# Did we get to add the A.N.? If not, do it here
if(anotetxt != ""):
if((not anoteadded) or forceanote):
subtxt = mem + winfo + anotetxt + prompt + subtxt
else:
subtxt = mem + winfo + prompt + subtxt
else:
subtxt = mem + winfo + prompt + subtxt
# Send it!
ikrequest(subtxt)
#==================================================================#
# Send text to generator and deal with output
#==================================================================#
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
def _generate(txt, minimum, maximum, found_entries):
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
gen_in = torch.tensor(txt, dtype=torch.long)[None]
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
if(vars.sp is not None):
soft_tokens = torch.arange(
model.config.vocab_size,
model.config.vocab_size + vars.sp.shape[0],
)
gen_in = torch.cat((soft_tokens[None], gen_in), dim=-1)
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
assert gen_in.shape[-1] + vars.genamt <= vars.max_length
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
if(vars.hascuda and vars.usegpu):
Fix an error that occurs when all layers are on second GPU
2021-12-16 01:03:49 +01:00
gen_in = gen_in.to(vars.gpu_device)
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
elif(vars.hascuda and vars.breakmodel):
gen_in = gen_in.to(breakmodel.primary_device)
else:
gen_in = gen_in.to('cpu')
model.kai_scanner_excluded_world_info = found_entries
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
vars._actions = vars.actions
vars._prompt = vars.prompt
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
if(vars.dynamicscan):
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
vars._actions = vars._actions.copy()
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
with torch.no_grad():
already_generated = 0
numseqs = vars.numseqs
while True:
genout = generator(
gen_in,
do_sample=True,
min_length=minimum,
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
max_length=int(2e9),
Always pass 1.1 as repetition penalty to generator The `dynamic_processor_wrap` makes it so that the repetition penalty is read directly from `vars`, but this only works if the initial repetition sent to `generator` is not equal to 1. So we are now forcing the initial repetition penalty to be something other than 1.
2022-01-04 20:18:58 +01:00
repetition_penalty=1.1,
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
bad_words_ids=vars.badwordsids,
use_cache=True,
num_return_sequences=numseqs
)
already_generated += len(genout[0]) - len(gen_in[0])
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
assert already_generated <= vars.genamt
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
if(model.kai_scanner.halt or not model.kai_scanner.regeneration_required):
break
assert genout.ndim >= 2
assert genout.shape[0] == vars.numseqs
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
if(vars.lua_koboldbridge.generated_cols and vars.generated_tkns != vars.lua_koboldbridge.generated_cols):
raise RuntimeError("Inconsistency detected between KoboldAI Python and Lua backends")
if(already_generated != vars.generated_tkns):
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
raise RuntimeError("WI scanning error")
for r in range(vars.numseqs):
for c in range(already_generated):
assert vars.lua_koboldbridge.generated[r+1][c+1] is not None
Fix a problem with the Lua regeneration API It was an egregious typo that caused tokens to be rearranged on regeneration.
2021-12-15 05:04:03 +01:00
genout[r][genout.shape[-1] - already_generated + c] = vars.lua_koboldbridge.generated[r+1][c+1]
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
encoded = []
for i in range(vars.numseqs):
txt = tokenizer.decode(genout[i, -already_generated:])
Fix a bug with dynamic WI scan when using a soft prompt The problem was that when a soft prompt is being used, the dynamic scanning criteria searches a different set of tokens for world info keys than the `_generate()` function, which results in generation loops when a world info key appears in the former set of tokens but not the latter.
2022-01-10 21:52:49 +01:00
winfo, mem, anotetxt, _found_entries = calcsubmitbudgetheader(txt, force_use_txt=True, actions=vars._actions)
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
found_entries[i].update(_found_entries)
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
txt, _, _ = calcsubmitbudget(len(vars._actions), winfo, mem, anotetxt, vars._actions, submission=txt)
encoded.append(torch.tensor(txt, dtype=torch.long, device=genout.device))
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
max_length = len(max(encoded, key=len))
encoded = torch.stack(tuple(torch.nn.functional.pad(e, (max_length - len(e), 0), value=model.config.pad_token_id or model.config.eos_token_id) for e in encoded))
genout = torch.cat(
(
encoded,
genout[..., -already_generated:],
),
dim=-1
)
if(vars.sp is not None):
soft_tokens = torch.arange(
model.config.vocab_size,
model.config.vocab_size + vars.sp.shape[0],
device=genout.device,
)
genout = torch.cat((soft_tokens.tile(vars.numseqs, 1), genout), dim=-1)
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
assert genout.shape[-1] + vars.genamt - already_generated <= vars.max_length
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
diff = genout.shape[-1] - gen_in.shape[-1]
minimum += diff
maximum += diff
gen_in = genout
numseqs = 1
return genout, already_generated
Support for multiple gens per action with dynamic scan
2021-11-17 22:17:59 +01:00
def generate(txt, minimum, maximum, found_entries=None):
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
vars.generated_tkns = 0
Support for multiple gens per action with dynamic scan
2021-11-17 22:17:59 +01:00
if(found_entries is None):
found_entries = set()
found_entries = tuple(found_entries.copy() for _ in range(vars.numseqs))
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
print("{0}Min:{1}, Max:{2}, Txt:{3}{4}".format(colors.YELLOW, minimum, maximum, tokenizer.decode(txt), colors.END))
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Store context in memory to use it for comparison with generated content
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
vars.lastctx = tokenizer.decode(txt)
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Clear CUDA cache if using GPU
if(vars.hascuda and (vars.usegpu or vars.breakmodel)):
gc.collect()
torch.cuda.empty_cache()
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Submit input text to generator
try:
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
genout, already_generated = tpool.execute(_generate, txt, minimum, maximum, found_entries)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
except Exception as e:
Catch Lua errors properly
2021-12-13 08:32:09 +01:00
if(issubclass(type(e), lupa.LuaError)):
vars.lua_koboldbridge.obliterate_multiverse()
Display icons for active userscripts and softprompts Also fixes the userscript menu so that the active userscripts preserve the previously selected order as was originally intended.
2021-12-23 05:33:27 +01:00
vars.lua_running = False
Fix typos in error messages
2022-01-14 04:33:55 +01:00
emit('from_server', {'cmd': 'errmsg', 'data': 'Lua script error; please check console.'}, broadcast=True)
Display icons for active userscripts and softprompts Also fixes the userscript menu so that the active userscripts preserve the previously selected order as was originally intended.
2021-12-23 05:33:27 +01:00
sendUSStatItems()
Catch Lua errors properly
2021-12-13 08:32:09 +01:00
print("{0}{1}{2}".format(colors.RED, "***LUA ERROR***: ", colors.END), end="", file=sys.stderr)
Remove escape characters from all error messages
2021-12-13 17:47:34 +01:00
print("{0}{1}{2}".format(colors.RED, str(e).replace("\033", ""), colors.END), file=sys.stderr)
Catch Lua errors properly
2021-12-13 08:32:09 +01:00
print("{0}{1}{2}".format(colors.YELLOW, "Lua engine stopped; please open 'Userscripts' and press Load to reinitialize scripts.", colors.END), file=sys.stderr)
else:
Fix typos in error messages
2022-01-14 04:33:55 +01:00
emit('from_server', {'cmd': 'errmsg', 'data': 'Error occurred during generator call; please check console.'}, broadcast=True)
Show full stack trace for generator errors to help in diagnosing errors
2021-12-15 08:03:08 +01:00
print("{0}{1}{2}".format(colors.RED, traceback.format_exc().replace("\033", ""), colors.END), file=sys.stderr)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
set_aibusy(0)
return
Add API for generated tokens and output text
2021-12-13 01:27:20 +01:00
for i in range(vars.numseqs):
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
vars.lua_koboldbridge.generated[i+1][vars.generated_tkns] = int(genout[i, -1].item())
Add API for generated tokens and output text
2021-12-13 01:27:20 +01:00
vars.lua_koboldbridge.outputs[i+1] = tokenizer.decode(genout[i, -already_generated:])
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
Add Lua API for reading the text the user submitted to the AI
2021-12-12 07:52:42 +01:00
execute_outmod()
Add API for generated tokens and output text
2021-12-13 01:27:20 +01:00
if(vars.lua_koboldbridge.regeneration_required):
vars.lua_koboldbridge.regeneration_required = False
genout = []
for i in range(vars.numseqs):
genout.append({"generated_text": vars.lua_koboldbridge.outputs[i+1]})
assert type(genout[-1]["generated_text"]) is str
else:
genout = [{"generated_text": tokenizer.decode(tokens[-already_generated:])} for tokens in genout]
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if(len(genout) == 1):
genresult(genout[0]["generated_text"])
else:
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
if(vars.lua_koboldbridge.restart_sequence is not None and vars.lua_koboldbridge.restart_sequence > 0):
genresult(genout[vars.lua_koboldbridge.restart_sequence-1]["generated_text"])
else:
genselect(genout)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Clear CUDA cache again if using GPU
if(vars.hascuda and (vars.usegpu or vars.breakmodel)):
del genout
gc.collect()
torch.cuda.empty_cache()
set_aibusy(0)
#==================================================================#
# Deal with a single return sequence from generate()
#==================================================================#
Fix AI output text flash showing up on wrong chunk
2021-12-29 20:23:22 +01:00
def genresult(genout, flash=True):
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
print("{0}{1}{2}".format(colors.CYAN, genout, colors.END))
# Format output before continuing
genout = applyoutputformatting(genout)
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
vars.lua_koboldbridge.feedback = genout
if(len(genout) == 0):
return
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Add formatted text to Actions array and refresh the game screen
if(len(vars.prompt.strip()) == 0):
vars.prompt = genout
else:
vars.actions.append(genout)
update_story_chunk('last')
Fix AI output text flash showing up on wrong chunk
2021-12-29 20:23:22 +01:00
if(flash):
emit('from_server', {'cmd': 'texteffect', 'data': vars.actions.get_last_key() + 1 if len(vars.actions) else 0}, broadcast=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#==================================================================#
# Send generator sequences to the UI for selection
#==================================================================#
def genselect(genout):
i = 0
for result in genout:
# Apply output formatting rules to sequences
result["generated_text"] = applyoutputformatting(result["generated_text"])
print("{0}[Result {1}]\n{2}{3}".format(colors.CYAN, i, result["generated_text"], colors.END))
i += 1
# Store sequences in memory until selection is made
vars.genseqs = genout
# Send sequences to UI for selection
emit('from_server', {'cmd': 'genseqs', 'data': genout}, broadcast=True)
#==================================================================#
# Send selected sequence to action log and refresh UI
#==================================================================#
def selectsequence(n):
if(len(vars.genseqs) == 0):
return
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
vars.lua_koboldbridge.feedback = vars.genseqs[int(n)]["generated_text"]
if(len(vars.lua_koboldbridge.feedback) != 0):
vars.actions.append(vars.lua_koboldbridge.feedback)
update_story_chunk('last')
Fix AI output text flash showing up on wrong chunk
2021-12-29 20:23:22 +01:00
emit('from_server', {'cmd': 'texteffect', 'data': vars.actions.get_last_key() + 1 if len(vars.actions) else 0}, broadcast=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
emit('from_server', {'cmd': 'hidegenseqs', 'data': ''}, broadcast=True)
vars.genseqs = []
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
if(vars.lua_koboldbridge.restart_sequence is not None):
Allow retry button to regenerate random story
2021-12-27 01:52:56 +01:00
actionsubmit("", actionmode=vars.actionmode, force_submit=True, disable_recentrng=True)
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#==================================================================#
# Send transformers-style request to ngrok/colab host
#==================================================================#
def sendtocolab(txt, min, max):
# Log request to console
print("{0}Tokens:{1}, Txt:{2}{3}".format(colors.YELLOW, min-1, txt, colors.END))
# Store context in memory to use it for comparison with generated content
vars.lastctx = txt
# Build request JSON data
reqdata = {
'text': txt,
'min': min,
'max': max,
'rep_pen': vars.rep_pen,
Repetition penalty slope and range
2022-01-24 21:30:38 +01:00
'rep_pen_slope': vars.rep_pen_slope,
'rep_pen_range': vars.rep_pen_range,
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
'temperature': vars.temp,
'top_p': vars.top_p,
'top_k': vars.top_k,
'tfs': vars.tfs,
'numseqs': vars.numseqs,
'retfultxt': False
}
# Create request
req = requests.post(
vars.colaburl,
json = reqdata
)
# Deal with the response
if(req.status_code == 200):
js = req.json()["data"]
# Try to be backwards compatible with outdated colab
if("text" in js):
genout = [getnewcontent(js["text"])]
else:
genout = js["seqs"]
Add API for generated tokens and output text
2021-12-13 01:27:20 +01:00
for i in range(vars.numseqs):
vars.lua_koboldbridge.outputs[i+1] = genout[i]
Add Lua API for reading the text the user submitted to the AI
2021-12-12 07:52:42 +01:00
execute_outmod()
Add API for generated tokens and output text
2021-12-13 01:27:20 +01:00
if(vars.lua_koboldbridge.regeneration_required):
vars.lua_koboldbridge.regeneration_required = False
genout = []
for i in range(vars.numseqs):
genout.append(vars.lua_koboldbridge.outputs[i+1])
assert type(genout[-1]) is str
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if(len(genout) == 1):
genresult(genout[0])
else:
# Convert torch output format to transformers
seqs = []
for seq in genout:
seqs.append({"generated_text": seq})
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
if(vars.lua_koboldbridge.restart_sequence is not None and vars.lua_koboldbridge.restart_sequence > 0):
genresult(genout[vars.lua_koboldbridge.restart_sequence-1]["generated_text"])
else:
genselect(genout)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Format output before continuing
#genout = applyoutputformatting(getnewcontent(genout))
# Add formatted text to Actions array and refresh the game screen
#vars.actions.append(genout)
#refresh_story()
Fix AI output text flash showing up on wrong chunk
2021-12-29 20:23:22 +01:00
#emit('from_server', {'cmd': 'texteffect', 'data': vars.actions.get_last_key() + 1 if len(vars.actions) else 0})
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
set_aibusy(0)
else:
errmsg = "Colab API Error: Failed to get a reply from the server. Please check the colab console."
print("{0}{1}{2}".format(colors.RED, errmsg, colors.END))
emit('from_server', {'cmd': 'errmsg', 'data': errmsg}, broadcast=True)
set_aibusy(0)
Integrate TPU backend This commit puts the TPU backend code directly in to the KoboldAI code to make it easier to modify.
2021-11-20 00:06:57 +01:00
#==================================================================#
# Send text to TPU mesh transformer backend
#==================================================================#
def tpumtjgenerate(txt, minimum, maximum, found_entries=None):
Add TPU support for dynamic WI scan and generation modifiers
2022-01-15 03:39:02 +01:00
vars.generated_tkns = 0
Integrate TPU backend This commit puts the TPU backend code directly in to the KoboldAI code to make it easier to modify.
2021-11-20 00:06:57 +01:00
if(found_entries is None):
found_entries = set()
found_entries = tuple(found_entries.copy() for _ in range(vars.numseqs))
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
print("{0}Min:{1}, Max:{2}, Txt:{3}{4}".format(colors.YELLOW, minimum, maximum, tokenizer.decode(txt), colors.END))
Integrate TPU backend This commit puts the TPU backend code directly in to the KoboldAI code to make it easier to modify.
2021-11-20 00:06:57 +01:00
Add TPU support for dynamic WI scan and generation modifiers
2022-01-15 03:39:02 +01:00
vars._actions = vars.actions
vars._prompt = vars.prompt
if(vars.dynamicscan):
vars._actions = vars._actions.copy()
Integrate TPU backend This commit puts the TPU backend code directly in to the KoboldAI code to make it easier to modify.
2021-11-20 00:06:57 +01:00
# Submit input text to generator
try:
Compile TPU backend in background
2022-01-07 19:47:21 +01:00
soft_tokens = tpumtjgetsofttokens()
Add soft prompt support to TPU backend
2021-11-22 00:08:04 +01:00
Add TPU support for dynamic WI scan and generation modifiers
2022-01-15 03:39:02 +01:00
global past
Show message when TPU backend is compiling
2022-01-17 03:09:10 +01:00
socketio.start_background_task(copy_current_request_context(check_for_backend_compilation))
Use original TPU backend if possible
2022-01-16 05:31:07 +01:00
if(vars.dynamicscan or (not vars.nogenmod and vars.has_genmod)):
context = np.tile(np.uint32(txt), (vars.numseqs, 1))
past = np.empty((vars.numseqs, 0), dtype=np.uint32)
while(True):
genout, n_generated, regeneration_required, halt = tpool.execute(
tpu_mtj_backend.infer_dynamic,
context,
gen_len = maximum-minimum+1,
numseqs=vars.numseqs,
soft_embeddings=vars.sp,
soft_tokens=soft_tokens,
excluded_world_info=found_entries,
)
past = np.pad(past, ((0, 0), (0, n_generated)))
for r in range(vars.numseqs):
for c in range(vars.lua_koboldbridge.generated_cols):
assert vars.lua_koboldbridge.generated[r+1][c+1] is not None
past[r, c] = vars.lua_koboldbridge.generated[r+1][c+1]
Dynamic TPU backend should support dynamic warpers and abort button
2022-01-17 20:10:32 +01:00
if(vars.abort or halt or not regeneration_required):
Use original TPU backend if possible
2022-01-16 05:31:07 +01:00
break
print("(regeneration triggered)")
encoded = []
for i in range(vars.numseqs):
txt = tokenizer.decode(past[i])
Use `vars._actions` in `tpumtjgenerate` and its callbacks
2022-01-17 19:24:11 +01:00
winfo, mem, anotetxt, _found_entries = calcsubmitbudgetheader(txt, force_use_txt=True, actions=vars._actions)
Use original TPU backend if possible
2022-01-16 05:31:07 +01:00
found_entries[i].update(_found_entries)
txt, _, _ = calcsubmitbudget(len(vars._actions), winfo, mem, anotetxt, vars._actions, submission=txt)
encoded.append(np.array(txt, dtype=np.uint32))
max_length = len(max(encoded, key=len))
encoded = np.stack(tuple(np.pad(e, (max_length - len(e), 0), constant_values=tpu_mtj_backend.pad_token_id) for e in encoded))
context = np.concatenate(
(
encoded,
past,
),
axis=-1,
)
else:
genout = tpool.execute(
tpu_mtj_backend.infer_static,
np.uint32(txt),
Add TPU support for dynamic WI scan and generation modifiers
2022-01-15 03:39:02 +01:00
gen_len = maximum-minimum+1,
temp=vars.temp,
top_p=vars.top_p,
top_k=vars.top_k,
tfs=vars.tfs,
numseqs=vars.numseqs,
repetition_penalty=vars.rep_pen,
Repetition penalty slope and range
2022-01-24 21:30:38 +01:00
rpslope=vars.rep_pen_slope,
rprange=vars.rep_pen_range,
Add TPU support for dynamic WI scan and generation modifiers
2022-01-15 03:39:02 +01:00
soft_embeddings=vars.sp,
soft_tokens=soft_tokens,
)
Use original TPU backend if possible
2022-01-16 05:31:07 +01:00
past = genout
Add TPU support for dynamic WI scan and generation modifiers
2022-01-15 03:39:02 +01:00
for i in range(vars.numseqs):
Use original TPU backend if possible
2022-01-16 05:31:07 +01:00
vars.lua_koboldbridge.generated[i+1] = vars.lua_state.table(*genout[i].tolist())
Fix settings callback, and `genout.shape[-1]` in `tpumtjgenerate()`
2022-01-17 20:52:29 +01:00
vars.lua_koboldbridge.generated_cols = vars.generated_tkns = genout[0].shape[-1]
Integrate TPU backend This commit puts the TPU backend code directly in to the KoboldAI code to make it easier to modify.
2021-11-20 00:06:57 +01:00
except Exception as e:
Catch Lua errors properly
2021-12-13 08:32:09 +01:00
if(issubclass(type(e), lupa.LuaError)):
vars.lua_koboldbridge.obliterate_multiverse()
Display icons for active userscripts and softprompts Also fixes the userscript menu so that the active userscripts preserve the previously selected order as was originally intended.
2021-12-23 05:33:27 +01:00
vars.lua_running = False
Fix typos in error messages
2022-01-14 04:33:55 +01:00
emit('from_server', {'cmd': 'errmsg', 'data': 'Lua script error; please check console.'}, broadcast=True)
Display icons for active userscripts and softprompts Also fixes the userscript menu so that the active userscripts preserve the previously selected order as was originally intended.
2021-12-23 05:33:27 +01:00
sendUSStatItems()
Catch Lua errors properly
2021-12-13 08:32:09 +01:00
print("{0}{1}{2}".format(colors.RED, "***LUA ERROR***: ", colors.END), end="", file=sys.stderr)
Remove escape characters from all error messages
2021-12-13 17:47:34 +01:00
print("{0}{1}{2}".format(colors.RED, str(e).replace("\033", ""), colors.END), file=sys.stderr)
Catch Lua errors properly
2021-12-13 08:32:09 +01:00
print("{0}{1}{2}".format(colors.YELLOW, "Lua engine stopped; please open 'Userscripts' and press Load to reinitialize scripts.", colors.END), file=sys.stderr)
else:
Fix typos in error messages
2022-01-14 04:33:55 +01:00
emit('from_server', {'cmd': 'errmsg', 'data': 'Error occurred during generator call; please check console.'}, broadcast=True)
Show full stack trace for generator errors to help in diagnosing errors
2021-12-15 08:03:08 +01:00
print("{0}{1}{2}".format(colors.RED, traceback.format_exc().replace("\033", ""), colors.END), file=sys.stderr)
Integrate TPU backend This commit puts the TPU backend code directly in to the KoboldAI code to make it easier to modify.
2021-11-20 00:06:57 +01:00
set_aibusy(0)
return
Add TPU support for dynamic WI scan and generation modifiers
2022-01-15 03:39:02 +01:00
Add API for generated tokens and output text
2021-12-13 01:27:20 +01:00
for i in range(vars.numseqs):
Add TPU support for dynamic WI scan and generation modifiers
2022-01-15 03:39:02 +01:00
vars.lua_koboldbridge.outputs[i+1] = tokenizer.decode(past[i])
genout = past
Integrate TPU backend This commit puts the TPU backend code directly in to the KoboldAI code to make it easier to modify.
2021-11-20 00:06:57 +01:00
Add Lua API for reading the text the user submitted to the AI
2021-12-12 07:52:42 +01:00
execute_outmod()
Add API for generated tokens and output text
2021-12-13 01:27:20 +01:00
if(vars.lua_koboldbridge.regeneration_required):
vars.lua_koboldbridge.regeneration_required = False
genout = []
for i in range(vars.numseqs):
genout.append({"generated_text": vars.lua_koboldbridge.outputs[i+1]})
assert type(genout[-1]["generated_text"]) is str
else:
genout = [{"generated_text": tokenizer.decode(txt)} for txt in genout]
Connect bridge.lua to aiserver.py Also enables the use of input modifiers and output modifiers, but not generation modifiers.
2021-12-11 18:45:45 +01:00
Integrate TPU backend This commit puts the TPU backend code directly in to the KoboldAI code to make it easier to modify.
2021-11-20 00:06:57 +01:00
if(len(genout) == 1):
genresult(genout[0]["generated_text"])
else:
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
if(vars.lua_koboldbridge.restart_sequence is not None and vars.lua_koboldbridge.restart_sequence > 0):
genresult(genout[vars.lua_koboldbridge.restart_sequence-1]["generated_text"])
else:
genselect(genout)
Integrate TPU backend This commit puts the TPU backend code directly in to the KoboldAI code to make it easier to modify.
2021-11-20 00:06:57 +01:00
set_aibusy(0)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#==================================================================#
# Replaces returns and newlines with HTML breaks
#==================================================================#
def formatforhtml(txt):
HTML escape the last commit </s> didn't work, needed to be HTML escaped (Thanks for the tip VE!)
2022-01-28 19:21:05 +01:00
return txt.replace("\\r\\n", "<br/>").replace("\\r", "<br/>").replace("\\n", "<br/>").replace("\r\n", "<br/>").replace('\n', '<br/>').replace('\r', '<br/>').replace('&lt;/s&gt;', '<br/>')
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#==================================================================#
# Strips submitted text from the text returned by the AI
#==================================================================#
def getnewcontent(txt):
# If the submitted context was blank, then everything is new
if(vars.lastctx == ""):
return txt
# Tokenize the last context and the generated content
Add safeguards for token budget and text formatting * Error messages are now shown when memory, author's note, etc. exceeds budget by itself * Formatting options no longer break if there are empty chunks in the story (although there shouldn't be any in the first place) * Number of generated tokens is now kept track of from Python
2021-12-27 00:29:54 +01:00
ctxtokens = tokenizer.encode(vars.lastctx, max_length=int(2e9), truncation=True)
txttokens = tokenizer.encode(txt, max_length=int(2e9), truncation=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
dif = (len(txttokens) - len(ctxtokens)) * -1
# Remove the context from the returned text
newtokens = txttokens[dif:]
return tokenizer.decode(newtokens)
#==================================================================#
# Applies chosen formatting options to text submitted to AI
#==================================================================#
def applyinputformatting(txt):
# Add sentence spacing
if(vars.formatoptns["frmtadsnsp"]):
txt = utils.addsentencespacing(txt, vars)
</s> new line mode Needed for Fairseq and XGLM models that do not understand the regular \n .
2022-01-31 18:39:34 +01:00
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
return txt
#==================================================================#
# Applies chosen formatting options to text returned from AI
#==================================================================#
def applyoutputformatting(txt):
Increase newlinemode compatibility Ran into issues with other modes like chatmode and adventure, moved it further down the pipeline and converting </s> back to \n before processing additional formatting. Still has an issue with the html formatting not working, but at least the AI works now.
2022-01-31 19:39:32 +01:00
# Revert S mode on output to maintain compatibility
txt = txt.replace('</s>', "\n")
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Use standard quotes and apostrophes
txt = utils.fixquotes(txt)
# Adventure mode clipping of all characters after '>'
if(vars.adventure):
txt = vars.acregex_ai.sub('', txt)
# Trim incomplete sentences
Chat Mode The Initial commit for Chat Mode, the nickname part of the UI is missing other than that it should be fully functional. To use Chat Mode effectively you first input a small dialogue (Can be around 6 lines 3 of your own inputs and 3 of the character) formatted as Name : it will then automate the actions needed to chat properly. During this mode single line mode is forced on, and Trim Incomplete Sentences is forced off.
2021-12-26 01:51:32 +01:00
if(vars.formatoptns["frmttriminc"] and not vars.chatmode):
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
txt = utils.trimincompletesentence(txt)
# Replace blank lines
Chatmode improvements Blank lines appear often in chatmode so it is best played with blank line removal turned on, this is now forced. Its not compatible with Adventure mode, so they now turn each other off.
2021-12-27 13:32:25 +01:00
if(vars.formatoptns["frmtrmblln"] or vars.chatmode):
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
txt = utils.replaceblanklines(txt)
# Remove special characters
if(vars.formatoptns["frmtrmspch"]):
txt = utils.removespecialchars(txt, vars)
# Single Line Mode
Chat Mode The Initial commit for Chat Mode, the nickname part of the UI is missing other than that it should be fully functional. To use Chat Mode effectively you first input a small dialogue (Can be around 6 lines 3 of your own inputs and 3 of the character) formatted as Name : it will then automate the actions needed to chat properly. During this mode single line mode is forced on, and Trim Incomplete Sentences is forced off.
2021-12-26 01:51:32 +01:00
if(vars.formatoptns["singleline"] or vars.chatmode):
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
txt = utils.singlelineprocessing(txt, vars)
return txt
#==================================================================#
# Sends the current story content to the Game Screen
#==================================================================#
def refresh_story():
Also apply comment formatting to prompt in `refresh_story()`
2021-11-21 06:26:45 +01:00
text_parts = ['<chunk n="0" id="n0" tabindex="-1">', vars.comregex_ui.sub(lambda m: '\n'.join('<comment>' + l + '</comment>' for l in m.group().split('\n')), html.escape(vars.prompt)), '</chunk>']
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
for idx in vars.actions:
item = vars.actions[idx]
idx += 1
item = html.escape(item)
Add syntax for comments <|...|>
2021-11-20 07:27:57 +01:00
item = vars.comregex_ui.sub(lambda m: '\n'.join('<comment>' + l + '</comment>' for l in m.group().split('\n')), item) # Add special formatting to comments
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
item = vars.acregex_ui.sub('<action>\\1</action>', item) # Add special formatting to adventure actions
text_parts.extend(('<chunk n="', str(idx), '" id="n', str(idx), '" tabindex="-1">', item, '</chunk>'))
emit('from_server', {'cmd': 'updatescreen', 'gamestarted': vars.gamestarted, 'data': formatforhtml(''.join(text_parts))}, broadcast=True)
#==================================================================#
# Signals the Game Screen to update one of the chunks
#==================================================================#
def update_story_chunk(idx: Union[int, str]):
if idx == 'last':
if len(vars.actions) <= 1:
# In this case, we are better off just refreshing the whole thing as the
# prompt might not have been shown yet (with a "Generating story..."
# message instead).
refresh_story()
Call `setgamesaved(False)` in `update_story_chunk` and `remove_story_chunk`
2022-01-21 22:39:51 +01:00
setgamesaved(False)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
return
idx = (vars.actions.get_last_key() if len(vars.actions) else 0) + 1
if idx == 0:
text = vars.prompt
else:
# Actions are 0 based, but in chunks 0 is the prompt.
# So the chunk index is one more than the corresponding action index.
Don't error in `update_story_chunk` if chunk index doesn't exist
2022-01-22 03:19:32 +01:00
if(idx - 1 not in vars.actions):
return
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
text = vars.actions[idx - 1]
item = html.escape(text)
Add syntax for comments <|...|>
2021-11-20 07:27:57 +01:00
item = vars.comregex_ui.sub(lambda m: '\n'.join('<comment>' + l + '</comment>' for l in m.group().split('\n')), item) # Add special formatting to comments
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
item = vars.acregex_ui.sub('<action>\\1</action>', item) # Add special formatting to adventure actions
chunk_text = f'<chunk n="{idx}" id="n{idx}" tabindex="-1">{formatforhtml(item)}</chunk>'
emit('from_server', {'cmd': 'updatechunk', 'data': {'index': idx, 'html': chunk_text}}, broadcast=True)
Call `setgamesaved(False)` in `update_story_chunk` and `remove_story_chunk`
2022-01-21 22:39:51 +01:00
setgamesaved(False)
Added autosave feature. When action is submitted it will save if the save setting is on and if the filename is set.
2022-01-20 13:46:34 +01:00
#If we've set the auto save flag, we'll now save the file
if vars.autosave and (".json" in vars.savedir):
save()
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#==================================================================#
# Signals the Game Screen to remove one of the chunks
#==================================================================#
def remove_story_chunk(idx: int):
emit('from_server', {'cmd': 'removechunk', 'data': idx}, broadcast=True)
Call `setgamesaved(False)` in `update_story_chunk` and `remove_story_chunk`
2022-01-21 22:39:51 +01:00
setgamesaved(False)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#==================================================================#
# Sends the current generator settings to the Game Menu
#==================================================================#
def refresh_settings():
# Suppress toggle change events while loading state
emit('from_server', {'cmd': 'allowtoggle', 'data': False}, broadcast=True)
if(vars.model != "InferKit"):
emit('from_server', {'cmd': 'updatetemp', 'data': vars.temp}, broadcast=True)
emit('from_server', {'cmd': 'updatetopp', 'data': vars.top_p}, broadcast=True)
emit('from_server', {'cmd': 'updatetopk', 'data': vars.top_k}, broadcast=True)
emit('from_server', {'cmd': 'updatetfs', 'data': vars.tfs}, broadcast=True)
emit('from_server', {'cmd': 'updatereppen', 'data': vars.rep_pen}, broadcast=True)
Repetition penalty slope and range
2022-01-24 21:30:38 +01:00
emit('from_server', {'cmd': 'updatereppenslope', 'data': vars.rep_pen_slope}, broadcast=True)
emit('from_server', {'cmd': 'updatereppenrange', 'data': vars.rep_pen_range}, broadcast=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
emit('from_server', {'cmd': 'updateoutlen', 'data': vars.genamt}, broadcast=True)
emit('from_server', {'cmd': 'updatetknmax', 'data': vars.max_length}, broadcast=True)
emit('from_server', {'cmd': 'updatenumseq', 'data': vars.numseqs}, broadcast=True)
else:
emit('from_server', {'cmd': 'updatetemp', 'data': vars.temp}, broadcast=True)
emit('from_server', {'cmd': 'updatetopp', 'data': vars.top_p}, broadcast=True)
emit('from_server', {'cmd': 'updateikgen', 'data': vars.ikgen}, broadcast=True)
emit('from_server', {'cmd': 'updateanotedepth', 'data': vars.andepth}, broadcast=True)
emit('from_server', {'cmd': 'updatewidepth', 'data': vars.widepth}, broadcast=True)
emit('from_server', {'cmd': 'updateuseprompt', 'data': vars.useprompt}, broadcast=True)
emit('from_server', {'cmd': 'updateadventure', 'data': vars.adventure}, broadcast=True)
Chat Mode The Initial commit for Chat Mode, the nickname part of the UI is missing other than that it should be fully functional. To use Chat Mode effectively you first input a small dialogue (Can be around 6 lines 3 of your own inputs and 3 of the character) formatted as Name : it will then automate the actions needed to chat properly. During this mode single line mode is forced on, and Trim Incomplete Sentences is forced off.
2021-12-26 01:51:32 +01:00
emit('from_server', {'cmd': 'updatechatmode', 'data': vars.chatmode}, broadcast=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
emit('from_server', {'cmd': 'updatedynamicscan', 'data': vars.dynamicscan}, broadcast=True)
No Prompt Gen Allow people to enter a prompt without generating anything by the AI. Combined with the always add prompt this is a very useful feature that allows people to write world information first, and then do a specific action. This mimics the behavior previously seen in AI Dungeon forks where it prompts for world information and then asks an action and can be particularly useful for people who want the prompt to always be part of the generation.
2021-12-16 12:47:44 +01:00
emit('from_server', {'cmd': 'updatenopromptgen', 'data': vars.nopromptgen}, broadcast=True)
Add Memory box to Random Story dialog and "Random Story Persist"
2021-12-30 05:15:59 +01:00
emit('from_server', {'cmd': 'updaterngpersist', 'data': vars.rngpersist}, broadcast=True)
Use original TPU backend if possible
2022-01-16 05:31:07 +01:00
emit('from_server', {'cmd': 'updatenogenmod', 'data': vars.nogenmod}, broadcast=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
emit('from_server', {'cmd': 'updatefrmttriminc', 'data': vars.formatoptns["frmttriminc"]}, broadcast=True)
emit('from_server', {'cmd': 'updatefrmtrmblln', 'data': vars.formatoptns["frmtrmblln"]}, broadcast=True)
emit('from_server', {'cmd': 'updatefrmtrmspch', 'data': vars.formatoptns["frmtrmspch"]}, broadcast=True)
emit('from_server', {'cmd': 'updatefrmtadsnsp', 'data': vars.formatoptns["frmtadsnsp"]}, broadcast=True)
emit('from_server', {'cmd': 'updatesingleline', 'data': vars.formatoptns["singleline"]}, broadcast=True)
# Allow toggle events again
emit('from_server', {'cmd': 'allowtoggle', 'data': True}, broadcast=True)
#==================================================================#
# Sets the logical and display states for the AI Busy condition
#==================================================================#
def set_aibusy(state):
if(state):
vars.aibusy = True
emit('from_server', {'cmd': 'setgamestate', 'data': 'wait'}, broadcast=True)
else:
vars.aibusy = False
emit('from_server', {'cmd': 'setgamestate', 'data': 'ready'}, broadcast=True)
#==================================================================#
#
#==================================================================#
def editrequest(n):
if(n == 0):
txt = vars.prompt
else:
txt = vars.actions[n-1]
vars.editln = n
emit('from_server', {'cmd': 'setinputtext', 'data': txt}, broadcast=True)
emit('from_server', {'cmd': 'enablesubmit', 'data': ''}, broadcast=True)
#==================================================================#
#
#==================================================================#
def editsubmit(data):
vars.recentedit = True
if(vars.editln == 0):
vars.prompt = data
else:
vars.actions[vars.editln-1] = data
vars.mode = "play"
update_story_chunk(vars.editln)
emit('from_server', {'cmd': 'texteffect', 'data': vars.editln}, broadcast=True)
emit('from_server', {'cmd': 'editmode', 'data': 'false'})
#==================================================================#
#
#==================================================================#
def deleterequest():
vars.recentedit = True
# Don't delete prompt
if(vars.editln == 0):
# Send error message
pass
else:
del vars.actions[vars.editln-1]
vars.mode = "play"
remove_story_chunk(vars.editln)
emit('from_server', {'cmd': 'editmode', 'data': 'false'})
#==================================================================#
#
#==================================================================#
def inlineedit(chunk, data):
vars.recentedit = True
chunk = int(chunk)
if(chunk == 0):
if(len(data.strip()) == 0):
return
vars.prompt = data
else:
Lua API fixes for deleting/editing story chunks
2022-01-01 00:28:03 +01:00
if(chunk-1 in vars.actions):
vars.actions[chunk-1] = data
Don't error in `update_story_chunk` if chunk index doesn't exist
2022-01-22 03:19:32 +01:00
else:
print(f"WARNING: Attempted to edit non-existent chunk {chunk}")
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(False)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
update_story_chunk(chunk)
emit('from_server', {'cmd': 'texteffect', 'data': chunk}, broadcast=True)
emit('from_server', {'cmd': 'editmode', 'data': 'false'}, broadcast=True)
#==================================================================#
#
#==================================================================#
def inlinedelete(chunk):
vars.recentedit = True
chunk = int(chunk)
# Don't delete prompt
if(chunk == 0):
# Send error message
update_story_chunk(chunk)
emit('from_server', {'cmd': 'errmsg', 'data': "Cannot delete the prompt."})
emit('from_server', {'cmd': 'editmode', 'data': 'false'}, broadcast=True)
else:
Lua API fixes for deleting/editing story chunks
2022-01-01 00:28:03 +01:00
if(chunk-1 in vars.actions):
del vars.actions[chunk-1]
Don't error in `update_story_chunk` if chunk index doesn't exist
2022-01-22 03:19:32 +01:00
else:
print(f"WARNING: Attempted to delete non-existent chunk {chunk}")
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(False)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
remove_story_chunk(chunk)
emit('from_server', {'cmd': 'editmode', 'data': 'false'}, broadcast=True)
#==================================================================#
# Toggles the game mode for memory editing and sends UI commands
#==================================================================#
def togglememorymode():
if(vars.mode == "play"):
vars.mode = "memory"
emit('from_server', {'cmd': 'memmode', 'data': 'true'}, broadcast=True)
emit('from_server', {'cmd': 'setinputtext', 'data': vars.memory}, broadcast=True)
emit('from_server', {'cmd': 'setanote', 'data': vars.authornote}, broadcast=True)
Author's Note Template
2021-12-30 05:43:36 +01:00
emit('from_server', {'cmd': 'setanotetemplate', 'data': vars.authornotetemplate}, broadcast=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
elif(vars.mode == "memory"):
vars.mode = "play"
emit('from_server', {'cmd': 'memmode', 'data': 'false'}, broadcast=True)
#==================================================================#
# Toggles the game mode for WI editing and sends UI commands
#==================================================================#
def togglewimode():
if(vars.mode == "play"):
vars.mode = "wi"
emit('from_server', {'cmd': 'wimode', 'data': 'true'}, broadcast=True)
elif(vars.mode == "wi"):
# Commit WI fields first
requestwi()
# Then set UI state back to Play
vars.mode = "play"
emit('from_server', {'cmd': 'wimode', 'data': 'false'}, broadcast=True)
sendwi()
#==================================================================#
#
#==================================================================#
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
def addwiitem(folder_uid=None):
assert folder_uid is None or folder_uid in vars.wifolders_d
ob = {"key": "", "keysecondary": "", "content": "", "comment": "", "folder": folder_uid, "num": len(vars.worldinfo), "init": False, "selective": False, "constant": False}
vars.worldinfo.append(ob)
Upload bridge.lua, default.lua and some Lua libs base64 inspect json.lua Lua-hashings Lua-nums Moses mt19937ar-lua Penlight Serpent
2021-12-11 01:45:57 +01:00
while(True):
uid = int.from_bytes(os.urandom(4), "little", signed=True)
if(uid not in vars.worldinfo_u):
break
vars.worldinfo_u[uid] = vars.worldinfo[-1]
vars.worldinfo[-1]["uid"] = uid
Fix numerous problems in the Lua world info API
2021-12-12 01:11:38 +01:00
if(folder_uid is not None):
vars.wifolders_u[folder_uid].append(vars.worldinfo[-1])
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
emit('from_server', {'cmd': 'addwiitem', 'data': ob}, broadcast=True)
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
#==================================================================#
# Creates a new WI folder with an unused cryptographically secure random UID
#==================================================================#
def addwifolder():
while(True):
uid = int.from_bytes(os.urandom(4), "little", signed=True)
if(uid not in vars.wifolders_d):
break
ob = {"name": "", "collapsed": False}
vars.wifolders_d[uid] = ob
vars.wifolders_l.append(uid)
Upload bridge.lua, default.lua and some Lua libs base64 inspect json.lua Lua-hashings Lua-nums Moses mt19937ar-lua Penlight Serpent
2021-12-11 01:45:57 +01:00
vars.wifolders_u[uid] = []
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
emit('from_server', {'cmd': 'addwifolder', 'uid': uid, 'data': ob}, broadcast=True)
addwiitem(folder_uid=uid)
#==================================================================#
Make all WI commands use UIDs instead of nums
2022-01-01 03:22:51 +01:00
# Move the WI entry with UID src so that it immediately precedes
# the WI entry with UID dst
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
#==================================================================#
def movewiitem(dst, src):
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(False)
Make all WI commands use UIDs instead of nums
2022-01-01 03:22:51 +01:00
if(vars.worldinfo_u[src]["folder"] is not None):
for i, e in enumerate(vars.wifolders_u[vars.worldinfo_u[src]["folder"]]):
if(e is vars.worldinfo_u[src]):
vars.wifolders_u[vars.worldinfo_u[src]["folder"]].pop(i)
Fix numerous problems in the Lua world info API
2021-12-12 01:11:38 +01:00
break
Make all WI commands use UIDs instead of nums
2022-01-01 03:22:51 +01:00
if(vars.worldinfo_u[dst]["folder"] is not None):
vars.wifolders_u[vars.worldinfo_u[dst]["folder"]].append(vars.worldinfo_u[src])
vars.worldinfo_u[src]["folder"] = vars.worldinfo_u[dst]["folder"]
for i, e in enumerate(vars.worldinfo):
if(e is vars.worldinfo_u[src]):
_src = i
elif(e is vars.worldinfo_u[dst]):
_dst = i
vars.worldinfo.insert(_dst - (_dst >= _src), vars.worldinfo.pop(_src))
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
sendwi()
#==================================================================#
# Move the WI folder with UID src so that it immediately precedes
# the WI folder with UID dst
#==================================================================#
def movewifolder(dst, src):
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(False)
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
vars.wifolders_l.remove(src)
if(dst is None):
# If dst is None, that means we should move src to be the last folder
vars.wifolders_l.append(src)
else:
vars.wifolders_l.insert(vars.wifolders_l.index(dst), src)
sendwi()
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#==================================================================#
#
#==================================================================#
def sendwi():
# Cache len of WI
ln = len(vars.worldinfo)
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Clear contents of WI container
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
emit('from_server', {'cmd': 'wistart', 'wifolders_d': vars.wifolders_d, 'wifolders_l': vars.wifolders_l, 'data': ''}, broadcast=True)
# Stable-sort WI entries in order of folder
stablesortwi()
Fix numerous problems in the Lua world info API
2021-12-12 01:11:38 +01:00
vars.worldinfo_i = [wi for wi in vars.worldinfo if wi["init"]]
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# If there are no WI entries, send an empty WI object
if(ln == 0):
addwiitem()
else:
# Send contents of WI array
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
last_folder = ...
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
for wi in vars.worldinfo:
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
if(wi["folder"] != last_folder):
emit('from_server', {'cmd': 'addwifolder', 'uid': wi["folder"], 'data': vars.wifolders_d[wi["folder"]] if wi["folder"] is not None else None}, broadcast=True)
last_folder = wi["folder"]
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
ob = wi
emit('from_server', {'cmd': 'addwiitem', 'data': ob}, broadcast=True)
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
emit('from_server', {'cmd': 'wifinish', 'data': ''}, broadcast=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#==================================================================#
# Request current contents of all WI HTML elements
#==================================================================#
def requestwi():
list = []
for wi in vars.worldinfo:
list.append(wi["num"])
emit('from_server', {'cmd': 'requestwiitem', 'data': list})
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
#==================================================================#
# Stable-sort WI items so that items in the same folder are adjacent,
# and items in different folders are sorted based on the order of the folders
#==================================================================#
def stablesortwi():
mapping = {uid: index for index, uid in enumerate(vars.wifolders_l)}
vars.worldinfo.sort(key=lambda x: mapping[x["folder"]] if x["folder"] is not None else float("inf"))
last_folder = ...
last_wi = None
Upload bridge.lua, default.lua and some Lua libs base64 inspect json.lua Lua-hashings Lua-nums Moses mt19937ar-lua Penlight Serpent
2021-12-11 01:45:57 +01:00
for i, wi in enumerate(vars.worldinfo):
wi["num"] = i
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
wi["init"] = True
if(wi["folder"] != last_folder):
if(last_wi is not None and last_folder is not ...):
last_wi["init"] = False
last_folder = wi["folder"]
last_wi = wi
Fix some minor coding errors
2021-12-05 17:39:59 +01:00
if(last_wi is not None):
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
last_wi["init"] = False
Upload bridge.lua, default.lua and some Lua libs base64 inspect json.lua Lua-hashings Lua-nums Moses mt19937ar-lua Penlight Serpent
2021-12-11 01:45:57 +01:00
for folder in vars.wifolders_u:
Fix numerous problems in the Lua world info API
2021-12-12 01:11:38 +01:00
vars.wifolders_u[folder].sort(key=lambda x: x["num"])
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#==================================================================#
# Extract object from server and send it to WI objects
#==================================================================#
def commitwi(ar):
for ob in ar:
Make all WI commands use UIDs instead of nums
2022-01-01 03:22:51 +01:00
ob["uid"] = int(ob["uid"])
vars.worldinfo_u[ob["uid"]]["key"] = ob["key"]
vars.worldinfo_u[ob["uid"]]["keysecondary"] = ob["keysecondary"]
vars.worldinfo_u[ob["uid"]]["content"] = ob["content"]
vars.worldinfo_u[ob["uid"]]["comment"] = ob.get("comment", "")
vars.worldinfo_u[ob["uid"]]["folder"] = ob.get("folder", None)
vars.worldinfo_u[ob["uid"]]["selective"] = ob["selective"]
vars.worldinfo_u[ob["uid"]]["constant"] = ob.get("constant", False)
stablesortwi()
vars.worldinfo_i = [wi for wi in vars.worldinfo if wi["init"]]
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#==================================================================#
#
#==================================================================#
Make all WI commands use UIDs instead of nums
2022-01-01 03:22:51 +01:00
def deletewi(uid):
if(uid in vars.worldinfo_u):
Deleting world info entries should call `setgamesaved(False)`
2022-01-19 01:36:20 +01:00
setgamesaved(False)
Make all WI commands use UIDs instead of nums
2022-01-01 03:22:51 +01:00
# Store UID of deletion request
vars.deletewi = uid
if(vars.deletewi is not None):
if(vars.worldinfo_u[vars.deletewi]["folder"] is not None):
for i, e in enumerate(vars.wifolders_u[vars.worldinfo_u[vars.deletewi]["folder"]]):
if(e is vars.worldinfo_u[vars.deletewi]):
vars.wifolders_u[vars.worldinfo_u[vars.deletewi]["folder"]].pop(i)
for i, e in enumerate(vars.worldinfo):
if(e is vars.worldinfo_u[vars.deletewi]):
del vars.worldinfo[i]
break
del vars.worldinfo_u[vars.deletewi]
# Send the new WI array structure
sendwi()
# And reset deletewi
vars.deletewi = None
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
#==================================================================#
#
#==================================================================#
def deletewifolder(uid):
uid = int(uid)
Upload bridge.lua, default.lua and some Lua libs base64 inspect json.lua Lua-hashings Lua-nums Moses mt19937ar-lua Penlight Serpent
2021-12-11 01:45:57 +01:00
del vars.wifolders_u[uid]
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
del vars.wifolders_d[uid]
del vars.wifolders_l[vars.wifolders_l.index(uid)]
Deleting world info entries should call `setgamesaved(False)`
2022-01-19 01:36:20 +01:00
setgamesaved(False)
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
# Delete uninitialized entries in the folder we're going to delete
vars.worldinfo = [wi for wi in vars.worldinfo if wi["folder"] != uid or wi["init"]]
Fix numerous problems in the Lua world info API
2021-12-12 01:11:38 +01:00
vars.worldinfo_i = [wi for wi in vars.worldinfo if wi["init"]]
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
# Move WI entries that are inside of the folder we're going to delete
# so that they're outside of all folders
for wi in vars.worldinfo:
if(wi["folder"] == uid):
wi["folder"] = None
sendwi()
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#==================================================================#
# Look for WI keys in text to generator
#==================================================================#
Fix a bug with dynamic WI scan when using a soft prompt The problem was that when a soft prompt is being used, the dynamic scanning criteria searches a different set of tokens for world info keys than the `_generate()` function, which results in generation loops when a world info key appears in the former set of tokens but not the latter.
2022-01-10 21:52:49 +01:00
def checkworldinfo(txt, allowed_entries=None, allowed_folders=None, force_use_txt=False, scan_story=True, actions=None):
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
original_txt = txt
Fix a bug with dynamic WI scan when using a soft prompt The problem was that when a soft prompt is being used, the dynamic scanning criteria searches a different set of tokens for world info keys than the `_generate()` function, which results in generation loops when a world info key appears in the former set of tokens but not the latter.
2022-01-10 21:52:49 +01:00
if(actions is None):
actions = vars.actions
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Dont go any further if WI is empty
if(len(vars.worldinfo) == 0):
return "", set()
# Cache actions length
Fix a bug with dynamic WI scan when using a soft prompt The problem was that when a soft prompt is being used, the dynamic scanning criteria searches a different set of tokens for world info keys than the `_generate()` function, which results in generation loops when a world info key appears in the former set of tokens but not the latter.
2022-01-10 21:52:49 +01:00
ln = len(actions)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Don't bother calculating action history if widepth is 0
Add option to `compute_context()` to not scan story
2022-01-05 01:26:59 +01:00
if(vars.widepth > 0 and scan_story):
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
depth = vars.widepth
# If this is not a continue, add 1 to widepth since submitted
# text is already in action history @ -1
if(not force_use_txt and (txt != "" and vars.prompt != txt)):
txt = ""
depth += 1
if(ln > 0):
chunks = collections.deque()
i = 0
Fix a bug with dynamic WI scan when using a soft prompt The problem was that when a soft prompt is being used, the dynamic scanning criteria searches a different set of tokens for world info keys than the `_generate()` function, which results in generation loops when a world info key appears in the former set of tokens but not the latter.
2022-01-10 21:52:49 +01:00
for key in reversed(actions):
chunk = actions[key]
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
chunks.appendleft(chunk)
i += 1
if(i == depth):
break
if(ln >= depth):
txt = "".join(chunks)
elif(ln > 0):
Remove comments from prompt in WI processor and InferKit mode
2021-11-21 04:23:06 +01:00
txt = vars.comregex_ai.sub('', vars.prompt) + "".join(chunks)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
elif(ln == 0):
Remove comments from prompt in WI processor and InferKit mode
2021-11-21 04:23:06 +01:00
txt = vars.comregex_ai.sub('', vars.prompt)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if(force_use_txt):
txt += original_txt
# Scan text for matches on WI keys
wimem = ""
found_entries = set()
for wi in vars.worldinfo:
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
if(allowed_entries is not None and wi["uid"] not in allowed_entries):
continue
if(allowed_folders is not None and wi["folder"] not in allowed_folders):
continue
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if(wi.get("constant", False)):
wimem = wimem + wi["content"] + "\n"
found_entries.add(id(wi))
continue
Add option to `compute_context()` to not scan story
2022-01-05 01:26:59 +01:00
if(len(wi["key"].strip()) > 0 and (not wi.get("selective", False) or len(wi.get("keysecondary", "").strip()) > 0)):
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Split comma-separated keys
keys = wi["key"].split(",")
keys_secondary = wi.get("keysecondary", "").split(",")
for k in keys:
ky = k
# Remove leading/trailing spaces if the option is enabled
if(vars.wirmvwhtsp):
ky = k.strip()
if ky in txt:
if wi.get("selective", False) and len(keys_secondary):
found = False
for ks in keys_secondary:
ksy = ks
if(vars.wirmvwhtsp):
ksy = ks.strip()
if ksy in txt:
wimem = wimem + wi["content"] + "\n"
found_entries.add(id(wi))
found = True
break
if found:
break
else:
wimem = wimem + wi["content"] + "\n"
found_entries.add(id(wi))
break
return wimem, found_entries
#==================================================================#
# Commit changes to Memory storage
#==================================================================#
def memsubmit(data):
Show confirmation dialog when navigating away from UI window
2022-01-17 18:11:06 +01:00
emit('from_server', {'cmd': 'setinputtext', 'data': data}, broadcast=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Maybe check for length at some point
# For now just send it to storage
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
if(data != vars.memory):
setgamesaved(False)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
vars.memory = data
vars.mode = "play"
emit('from_server', {'cmd': 'memmode', 'data': 'false'}, broadcast=True)
# Ask for contents of Author's Note field
emit('from_server', {'cmd': 'getanote', 'data': ''})
#==================================================================#
# Commit changes to Author's Note
#==================================================================#
Author's Note Template
2021-12-30 05:43:36 +01:00
def anotesubmit(data, template=""):
assert type(data) is str and type(template) is str
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Maybe check for length at some point
# For now just send it to storage
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
if(data != vars.authornote):
setgamesaved(False)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
vars.authornote = data
Author's Note Template
2021-12-30 05:43:36 +01:00
if(vars.authornotetemplate != template):
vars.setauthornotetemplate = template
settingschanged()
vars.authornotetemplate = template
Show confirmation dialog when navigating away from UI window
2022-01-17 18:11:06 +01:00
emit('from_server', {'cmd': 'setanote', 'data': vars.authornote}, broadcast=True)
emit('from_server', {'cmd': 'setanotetemplate', 'data': vars.authornotetemplate}, broadcast=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#==================================================================#
# Assembles game data into a request to InferKit API
#==================================================================#
def ikrequest(txt):
# Log request to console
print("{0}Len:{1}, Txt:{2}{3}".format(colors.YELLOW, len(txt), txt, colors.END))
# Build request JSON data
reqdata = {
'forceNoEnd': True,
'length': vars.ikgen,
'prompt': {
'isContinuation': False,
'text': txt
},
'startFromBeginning': False,
'streamResponse': False,
'temperature': vars.temp,
'topP': vars.top_p
}
# Create request
req = requests.post(
vars.url,
json = reqdata,
headers = {
'Authorization': 'Bearer '+vars.apikey
}
)
# Deal with the response
if(req.status_code == 200):
genout = req.json()["data"]["text"]
Add API for generated tokens and output text
2021-12-13 01:27:20 +01:00
vars.lua_koboldbridge.outputs[1] = genout
Add Lua API for reading the text the user submitted to the AI
2021-12-12 07:52:42 +01:00
execute_outmod()
Add API for generated tokens and output text
2021-12-13 01:27:20 +01:00
if(vars.lua_koboldbridge.regeneration_required):
vars.lua_koboldbridge.regeneration_required = False
genout = vars.lua_koboldbridge.outputs[1]
assert genout is str
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
print("{0}{1}{2}".format(colors.CYAN, genout, colors.END))
vars.actions.append(genout)
update_story_chunk('last')
Fix AI output text flash showing up on wrong chunk
2021-12-29 20:23:22 +01:00
emit('from_server', {'cmd': 'texteffect', 'data': vars.actions.get_last_key() + 1 if len(vars.actions) else 0}, broadcast=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
set_aibusy(0)
else:
# Send error message to web client
er = req.json()
if("error" in er):
code = er["error"]["extensions"]["code"]
elif("errors" in er):
code = er["errors"][0]["extensions"]["code"]
errmsg = "InferKit API Error: {0} - {1}".format(req.status_code, code)
emit('from_server', {'cmd': 'errmsg', 'data': errmsg}, broadcast=True)
set_aibusy(0)
#==================================================================#
# Assembles game data into a request to OpenAI API
#==================================================================#
def oairequest(txt, min, max):
# Log request to console
print("{0}Len:{1}, Txt:{2}{3}".format(colors.YELLOW, len(txt), txt, colors.END))
# Store context in memory to use it for comparison with generated content
vars.lastctx = txt
# Build request JSON data
reqdata = {
'prompt': txt,
'max_tokens': max,
'temperature': vars.temp,
'top_p': vars.top_p,
'n': 1,
'stream': False
}
req = requests.post(
vars.oaiurl,
json = reqdata,
headers = {
'Authorization': 'Bearer '+vars.oaiapikey,
'Content-Type': 'application/json'
}
)
# Deal with the response
if(req.status_code == 200):
genout = req.json()["choices"][0]["text"]
Add API for generated tokens and output text
2021-12-13 01:27:20 +01:00
vars.lua_koboldbridge.outputs[1] = genout
Add Lua API for reading the text the user submitted to the AI
2021-12-12 07:52:42 +01:00
execute_outmod()
Add API for generated tokens and output text
2021-12-13 01:27:20 +01:00
if(vars.lua_koboldbridge.regeneration_required):
vars.lua_koboldbridge.regeneration_required = False
genout = vars.lua_koboldbridge.outputs[1]
assert genout is str
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
print("{0}{1}{2}".format(colors.CYAN, genout, colors.END))
vars.actions.append(genout)
update_story_chunk('last')
Fix AI output text flash showing up on wrong chunk
2021-12-29 20:23:22 +01:00
emit('from_server', {'cmd': 'texteffect', 'data': vars.actions.get_last_key() + 1 if len(vars.actions) else 0}, broadcast=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
set_aibusy(0)
else:
# Send error message to web client
er = req.json()
if("error" in er):
type = er["error"]["type"]
message = er["error"]["message"]
errmsg = "OpenAI API Error: {0} - {1}".format(type, message)
emit('from_server', {'cmd': 'errmsg', 'data': errmsg}, broadcast=True)
set_aibusy(0)
#==================================================================#
# Forces UI to Play mode
#==================================================================#
def exitModes():
if(vars.mode == "edit"):
emit('from_server', {'cmd': 'editmode', 'data': 'false'}, broadcast=True)
elif(vars.mode == "memory"):
emit('from_server', {'cmd': 'memmode', 'data': 'false'}, broadcast=True)
elif(vars.mode == "wi"):
emit('from_server', {'cmd': 'wimode', 'data': 'false'}, broadcast=True)
vars.mode = "play"
#==================================================================#
# Launch in-browser save prompt
#==================================================================#
def saveas(name):
# Check if filename exists already
name = utils.cleanfilename(name)
if(not fileops.saveexists(name) or (vars.saveow and vars.svowname == name)):
# All clear to save
e = saveRequest(fileops.storypath(name))
vars.saveow = False
vars.svowname = ""
if(e is None):
emit('from_server', {'cmd': 'hidesaveas', 'data': ''})
else:
print("{0}{1}{2}".format(colors.RED, str(e), colors.END))
emit('from_server', {'cmd': 'popuperror', 'data': str(e)})
else:
# File exists, prompt for overwrite
vars.saveow = True
vars.svowname = name
emit('from_server', {'cmd': 'askforoverwrite', 'data': ''})
#==================================================================#
# Launch in-browser story-delete prompt
#==================================================================#
def deletesave(name):
name = utils.cleanfilename(name)
e = fileops.deletesave(name)
if(e is None):
if(vars.smandelete):
emit('from_server', {'cmd': 'hidepopupdelete', 'data': ''})
getloadlist()
else:
emit('from_server', {'cmd': 'popuperror', 'data': "The server denied your request to delete this story"})
else:
print("{0}{1}{2}".format(colors.RED, str(e), colors.END))
emit('from_server', {'cmd': 'popuperror', 'data': str(e)})
#==================================================================#
# Launch in-browser story-rename prompt
#==================================================================#
def renamesave(name, newname):
# Check if filename exists already
name = utils.cleanfilename(name)
newname = utils.cleanfilename(newname)
if(not fileops.saveexists(newname) or name == newname or (vars.saveow and vars.svowname == newname)):
e = fileops.renamesave(name, newname)
vars.saveow = False
vars.svowname = ""
if(e is None):
if(vars.smanrename):
emit('from_server', {'cmd': 'hidepopuprename', 'data': ''})
getloadlist()
else:
emit('from_server', {'cmd': 'popuperror', 'data': "The server denied your request to rename this story"})
else:
print("{0}{1}{2}".format(colors.RED, str(e), colors.END))
emit('from_server', {'cmd': 'popuperror', 'data': str(e)})
else:
# File exists, prompt for overwrite
vars.saveow = True
vars.svowname = newname
emit('from_server', {'cmd': 'askforoverwrite', 'data': ''})
#==================================================================#
# Save the currently running story
#==================================================================#
def save():
# Check if a file is currently open
if(".json" in vars.savedir):
saveRequest(vars.savedir)
else:
emit('from_server', {'cmd': 'saveas', 'data': ''})
#==================================================================#
# Save the story via file browser
#==================================================================#
def savetofile():
savpath = fileops.getsavepath(vars.savedir, "Save Story As", [("Json", "*.json")])
saveRequest(savpath)
#==================================================================#
# Save the story to specified path
#==================================================================#
def saveRequest(savpath):
if(savpath):
# Leave Edit/Memory mode before continuing
exitModes()
# Save path for future saves
vars.savedir = savpath
txtpath = os.path.splitext(savpath)[0] + ".txt"
# Build json to write
js = {}
js["gamestarted"] = vars.gamestarted
js["prompt"] = vars.prompt
js["memory"] = vars.memory
js["authorsnote"] = vars.authornote
Author's Note Template
2021-12-30 05:43:36 +01:00
js["anotetemplate"] = vars.authornotetemplate
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
js["actions"] = tuple(vars.actions.values())
js["worldinfo"] = []
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
js["wifolders_d"] = vars.wifolders_d
js["wifolders_l"] = vars.wifolders_l
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Extract only the important bits of WI
Always save all world info entries
2022-01-10 22:36:36 +01:00
for wi in vars.worldinfo_i:
if(True):
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
js["worldinfo"].append({
"key": wi["key"],
"keysecondary": wi["keysecondary"],
"content": wi["content"],
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
"comment": wi["comment"],
"folder": wi["folder"],
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
"selective": wi["selective"],
"constant": wi["constant"]
})
txt = vars.prompt + "".join(vars.actions.values())
# Write it
try:
file = open(savpath, "w")
except Exception as e:
return e
try:
file.write(json.dumps(js, indent=3))
except Exception as e:
file.close()
return e
file.close()
try:
file = open(txtpath, "w")
except Exception as e:
return e
try:
file.write(txt)
except Exception as e:
file.close()
return e
file.close()
filename = path.basename(savpath)
if(filename.endswith('.json')):
filename = filename[:-5]
vars.laststory = filename
emit('from_server', {'cmd': 'setstoryname', 'data': vars.laststory}, broadcast=True)
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
print("{0}Story saved to {1}!{2}".format(colors.GREEN, path.basename(savpath), colors.END))
#==================================================================#
# Show list of saved stories
#==================================================================#
def getloadlist():
emit('from_server', {'cmd': 'buildload', 'data': fileops.getstoryfiles()})
#==================================================================#
# Show list of soft prompts
#==================================================================#
def getsplist():
if(vars.allowsp):
emit('from_server', {'cmd': 'buildsp', 'data': fileops.getspfiles(vars.modeldim)})
Add Userscripts menu into GUI
2021-12-13 07:03:26 +01:00
#==================================================================#
Display icons for active userscripts and softprompts Also fixes the userscript menu so that the active userscripts preserve the previously selected order as was originally intended.
2021-12-23 05:33:27 +01:00
# Get list of userscripts
Add Userscripts menu into GUI
2021-12-13 07:03:26 +01:00
#==================================================================#
def getuslist():
files = {i: v for i, v in enumerate(fileops.getusfiles())}
loaded = []
unloaded = []
userscripts = set(vars.userscripts)
for i in range(len(files)):
Display icons for active userscripts and softprompts Also fixes the userscript menu so that the active userscripts preserve the previously selected order as was originally intended.
2021-12-23 05:33:27 +01:00
if files[i]["filename"] not in userscripts:
Add Userscripts menu into GUI
2021-12-13 07:03:26 +01:00
unloaded.append(files[i])
Display icons for active userscripts and softprompts Also fixes the userscript menu so that the active userscripts preserve the previously selected order as was originally intended.
2021-12-23 05:33:27 +01:00
files = {files[k]["filename"]: files[k] for k in files}
userscripts = set(files.keys())
for filename in vars.userscripts:
if filename in userscripts:
loaded.append(files[filename])
return unloaded, loaded
Add Userscripts menu into GUI
2021-12-13 07:03:26 +01:00
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#==================================================================#
# Load a saved story via file browser
#==================================================================#
def loadfromfile():
loadpath = fileops.getloadpath(vars.savedir, "Select Story File", [("Json", "*.json")])
loadRequest(loadpath)
#==================================================================#
# Load a stored story from a file
#==================================================================#
def loadRequest(loadpath, filename=None):
if(loadpath):
# Leave Edit/Memory mode before continuing
exitModes()
# Read file contents into JSON object
if(isinstance(loadpath, str)):
with open(loadpath, "r") as file:
js = json.load(file)
if(filename is None):
filename = path.basename(loadpath)
else:
js = loadpath
if(filename is None):
filename = "untitled.json"
# Copy file contents to vars
vars.gamestarted = js["gamestarted"]
vars.prompt = js["prompt"]
vars.memory = js["memory"]
vars.worldinfo = []
Fix numerous problems in the Lua world info API
2021-12-12 01:11:38 +01:00
vars.worldinfo = []
Upload bridge.lua, default.lua and some Lua libs base64 inspect json.lua Lua-hashings Lua-nums Moses mt19937ar-lua Penlight Serpent
2021-12-11 01:45:57 +01:00
vars.worldinfo_u = {}
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
vars.wifolders_d = {int(k): v for k, v in js.get("wifolders_d", {}).items()}
vars.wifolders_l = js.get("wifolders_l", [])
Upload bridge.lua, default.lua and some Lua libs base64 inspect json.lua Lua-hashings Lua-nums Moses mt19937ar-lua Penlight Serpent
2021-12-11 01:45:57 +01:00
vars.wifolders_u = {uid: [] for uid in vars.wifolders_d}
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
vars.lastact = ""
Add Lua API for reading the text the user submitted to the AI
2021-12-12 07:52:42 +01:00
vars.submission = ""
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
vars.lastctx = ""
del vars.actions
vars.actions = structures.KoboldStoryRegister()
actions = collections.deque(js["actions"])
if(len(vars.prompt.strip()) == 0):
while(len(actions)):
action = actions.popleft()
if(len(action.strip()) != 0):
vars.prompt = action
break
else:
vars.gamestarted = False
if(vars.gamestarted):
for s in actions:
vars.actions.append(s)
# Try not to break older save files
if("authorsnote" in js):
vars.authornote = js["authorsnote"]
else:
vars.authornote = ""
Author's Note Template
2021-12-30 05:43:36 +01:00
if("anotetemplate" in js):
vars.authornotetemplate = js["anotetemplate"]
else:
Consistent capitalization of "Author's note"
2021-12-30 07:48:25 +01:00
vars.authornotetemplate = "[Author's note: <|>]"
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if("worldinfo" in js):
num = 0
for wi in js["worldinfo"]:
vars.worldinfo.append({
"key": wi["key"],
"keysecondary": wi.get("keysecondary", ""),
"content": wi["content"],
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
"comment": wi.get("comment", ""),
"folder": wi.get("folder", None),
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
"num": num,
"init": True,
"selective": wi.get("selective", False),
Upload bridge.lua, default.lua and some Lua libs base64 inspect json.lua Lua-hashings Lua-nums Moses mt19937ar-lua Penlight Serpent
2021-12-11 01:45:57 +01:00
"constant": wi.get("constant", False),
"uid": None,
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
})
Upload bridge.lua, default.lua and some Lua libs base64 inspect json.lua Lua-hashings Lua-nums Moses mt19937ar-lua Penlight Serpent
2021-12-11 01:45:57 +01:00
while(True):
uid = int.from_bytes(os.urandom(4), "little", signed=True)
if(uid not in vars.worldinfo_u):
break
vars.worldinfo_u[uid] = vars.worldinfo[-1]
vars.worldinfo[-1]["uid"] = uid
Make sure `vars.wifolders_u` is set up properly on loading a save
2022-01-04 20:13:36 +01:00
if(vars.worldinfo[-1]["folder"] is not None):
vars.wifolders_u[vars.worldinfo[-1]["folder"]].append(vars.worldinfo[-1])
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
num += 1
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
for uid in vars.wifolders_l + [None]:
Upload bridge.lua, default.lua and some Lua libs base64 inspect json.lua Lua-hashings Lua-nums Moses mt19937ar-lua Penlight Serpent
2021-12-11 01:45:57 +01:00
vars.worldinfo.append({"key": "", "keysecondary": "", "content": "", "comment": "", "folder": uid, "num": None, "init": False, "selective": False, "constant": False, "uid": None})
Make sure uninitialized WI entries are given UIDs when loading saves
2021-12-19 00:00:06 +01:00
while(True):
uid = int.from_bytes(os.urandom(4), "little", signed=True)
if(uid not in vars.worldinfo_u):
break
vars.worldinfo_u[uid] = vars.worldinfo[-1]
vars.worldinfo[-1]["uid"] = uid
Make sure `vars.wifolders_u` is set up properly on loading a save
2022-01-04 20:13:36 +01:00
if(vars.worldinfo[-1]["folder"] is not None):
vars.wifolders_u[vars.worldinfo[-1]["folder"]].append(vars.worldinfo[-1])
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
stablesortwi()
Fix numerous problems in the Lua world info API
2021-12-12 01:11:38 +01:00
vars.worldinfo_i = [wi for wi in vars.worldinfo if wi["init"]]
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Save path for save button
vars.savedir = loadpath
# Clear loadselect var
vars.loadselect = ""
# Refresh game screen
_filename = filename
if(filename.endswith('.json')):
_filename = filename[:-5]
vars.laststory = _filename
emit('from_server', {'cmd': 'setstoryname', 'data': vars.laststory}, broadcast=True)
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
sendwi()
emit('from_server', {'cmd': 'setmemory', 'data': vars.memory}, broadcast=True)
emit('from_server', {'cmd': 'setanote', 'data': vars.authornote}, broadcast=True)
Author's Note Template
2021-12-30 05:43:36 +01:00
emit('from_server', {'cmd': 'setanotetemplate', 'data': vars.authornotetemplate}, broadcast=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
refresh_story()
emit('from_server', {'cmd': 'setgamestate', 'data': 'ready'}, broadcast=True)
emit('from_server', {'cmd': 'hidegenseqs', 'data': ''}, broadcast=True)
print("{0}Story loaded from {1}!{2}".format(colors.GREEN, filename, colors.END))
#==================================================================#
# Load a soft prompt from a file
#==================================================================#
def spRequest(filename):
Save softprompt filename in settings
2021-12-23 19:02:11 +01:00
vars.spfilename = ""
Fix the behaviour of the `softprompt` setting
2021-12-23 22:14:09 +01:00
settingschanged()
Save softprompt filename in settings
2021-12-23 19:02:11 +01:00
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if(len(filename) == 0):
vars.sp = None
Add soft prompt support to TPU backend
2021-11-22 00:08:04 +01:00
vars.sp_length = 0
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
return
global np
if 'np' not in globals():
import numpy as np
z, version, shape, fortran_order, dtype = fileops.checksp(filename, vars.modeldim)
assert isinstance(z, zipfile.ZipFile)
Display icons for active userscripts and softprompts Also fixes the userscript menu so that the active userscripts preserve the previously selected order as was originally intended.
2021-12-23 05:33:27 +01:00
with z.open('meta.json') as f:
vars.spmeta = json.load(f)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
z.close()
with np.load(fileops.sppath(filename), allow_pickle=False) as f:
tensor = f['tensor.npy']
# If the tensor is in bfloat16 format, convert it to float32
if(tensor.dtype == 'V2'):
tensor.dtype = np.uint16
tensor = np.uint32(tensor) << 16
tensor.dtype = np.float32
if(tensor.dtype != np.float16):
tensor = np.float32(tensor)
assert not np.isinf(tensor).any() and not np.isnan(tensor).any()
Show author and SP length in soft prompt menu
2022-01-18 22:30:09 +01:00
vars.sp_length = tensor.shape[-2]
vars.spmeta["n_tokens"] = vars.sp_length
Add soft prompt support to TPU backend
2021-11-22 00:08:04 +01:00
if(vars.model in ("TPUMeshTransformerGPTJ",)):
rows = tensor.shape[0]
TPU backend no longer needs to recompile after changing softprompt
2021-12-05 08:49:15 +01:00
padding_amount = tpu_mtj_backend.params["seq"] - (tpu_mtj_backend.params["seq"] % -tpu_mtj_backend.params["cores_per_replica"]) - rows
Add soft prompt support to TPU backend
2021-11-22 00:08:04 +01:00
tensor = np.pad(tensor, ((0, padding_amount), (0, 0)))
tensor = tensor.reshape(
tpu_mtj_backend.params["cores_per_replica"],
-1,
tpu_mtj_backend.params["d_model"],
)
xmaps for moving things onto TPU
2022-01-13 03:45:30 +01:00
vars.sp = tpu_mtj_backend.shard_xmap(np.float32(tensor))
Add soft prompt support to TPU backend
2021-11-22 00:08:04 +01:00
else:
vars.sp = torch.from_numpy(tensor)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
Save softprompt filename in settings
2021-12-23 19:02:11 +01:00
vars.spfilename = filename
Fix the behaviour of the `softprompt` setting
2021-12-23 22:14:09 +01:00
settingschanged()
Save softprompt filename in settings
2021-12-23 19:02:11 +01:00
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#==================================================================#
# Import an AIDungon game exported with Mimi's tool
#==================================================================#
def importRequest():
importpath = fileops.getloadpath(vars.savedir, "Select AID CAT File", [("Json", "*.json")])
if(importpath):
# Leave Edit/Memory mode before continuing
exitModes()
# Read file contents into JSON object
file = open(importpath, "rb")
vars.importjs = json.load(file)
# If a bundle file is being imported, select just the Adventures object
if type(vars.importjs) is dict and "stories" in vars.importjs:
vars.importjs = vars.importjs["stories"]
# Clear Popup Contents
emit('from_server', {'cmd': 'clearpopup', 'data': ''}, broadcast=True)
# Initialize vars
num = 0
vars.importnum = -1
# Get list of stories
for story in vars.importjs:
ob = {}
ob["num"] = num
if(story["title"] != "" and story["title"] != None):
ob["title"] = story["title"]
else:
ob["title"] = "(No Title)"
if(story["description"] != "" and story["description"] != None):
ob["descr"] = story["description"]
else:
ob["descr"] = "(No Description)"
if("actions" in story):
ob["acts"] = len(story["actions"])
elif("actionWindow" in story):
ob["acts"] = len(story["actionWindow"])
emit('from_server', {'cmd': 'addimportline', 'data': ob})
num += 1
# Show Popup
emit('from_server', {'cmd': 'popupshow', 'data': True})
#==================================================================#
# Import an AIDungon game selected in popup
#==================================================================#
def importgame():
if(vars.importnum >= 0):
# Cache reference to selected game
ref = vars.importjs[vars.importnum]
# Copy game contents to vars
vars.gamestarted = True
# Support for different versions of export script
if("actions" in ref):
if(len(ref["actions"]) > 0):
vars.prompt = ref["actions"][0]["text"]
else:
vars.prompt = ""
elif("actionWindow" in ref):
if(len(ref["actionWindow"]) > 0):
vars.prompt = ref["actionWindow"][0]["text"]
else:
vars.prompt = ""
else:
vars.prompt = ""
vars.memory = ref["memory"]
vars.authornote = ref["authorsNote"] if type(ref["authorsNote"]) is str else ""
Consistent capitalization of "Author's note"
2021-12-30 07:48:25 +01:00
vars.authornotetemplate = "[Author's note: <|>]"
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
vars.actions = structures.KoboldStoryRegister()
vars.worldinfo = []
Fix numerous problems in the Lua world info API
2021-12-12 01:11:38 +01:00
vars.worldinfo_i = []
Upload bridge.lua, default.lua and some Lua libs base64 inspect json.lua Lua-hashings Lua-nums Moses mt19937ar-lua Penlight Serpent
2021-12-11 01:45:57 +01:00
vars.worldinfo_u = {}
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
vars.wifolders_d = {}
vars.wifolders_l = []
Upload bridge.lua, default.lua and some Lua libs base64 inspect json.lua Lua-hashings Lua-nums Moses mt19937ar-lua Penlight Serpent
2021-12-11 01:45:57 +01:00
vars.wifolders_u = {uid: [] for uid in vars.wifolders_d}
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
vars.lastact = ""
Add Lua API for reading the text the user submitted to the AI
2021-12-12 07:52:42 +01:00
vars.submission = ""
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
vars.lastctx = ""
# Get all actions except for prompt
if("actions" in ref):
if(len(ref["actions"]) > 1):
for act in ref["actions"][1:]:
vars.actions.append(act["text"])
elif("actionWindow" in ref):
if(len(ref["actionWindow"]) > 1):
for act in ref["actionWindow"][1:]:
vars.actions.append(act["text"])
# Get just the important parts of world info
if(ref["worldInfo"] != None):
if(len(ref["worldInfo"]) > 1):
num = 0
for wi in ref["worldInfo"]:
vars.worldinfo.append({
"key": wi["keys"],
"keysecondary": wi.get("keysecondary", ""),
"content": wi["entry"],
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
"comment": wi.get("comment", ""),
"folder": wi.get("folder", None),
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
"num": num,
"init": True,
"selective": wi.get("selective", False),
Upload bridge.lua, default.lua and some Lua libs base64 inspect json.lua Lua-hashings Lua-nums Moses mt19937ar-lua Penlight Serpent
2021-12-11 01:45:57 +01:00
"constant": wi.get("constant", False),
"uid": None,
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
})
Upload bridge.lua, default.lua and some Lua libs base64 inspect json.lua Lua-hashings Lua-nums Moses mt19937ar-lua Penlight Serpent
2021-12-11 01:45:57 +01:00
while(True):
uid = int.from_bytes(os.urandom(4), "little", signed=True)
if(uid not in vars.worldinfo_u):
break
vars.worldinfo_u[uid] = vars.worldinfo[-1]
vars.worldinfo[-1]["uid"] = uid
Make sure `vars.wifolders_u` is set up properly on loading a save
2022-01-04 20:13:36 +01:00
if(vars.worldinfo[-1]["folder"]) is not None:
vars.wifolders_u[vars.worldinfo[-1]["folder"]].append(vars.worldinfo[-1])
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
num += 1
Fix numerous problems in the Lua world info API
2021-12-12 01:11:38 +01:00
for uid in vars.wifolders_l + [None]:
vars.worldinfo.append({"key": "", "keysecondary": "", "content": "", "comment": "", "folder": uid, "num": None, "init": False, "selective": False, "constant": False, "uid": None})
Make sure uninitialized WI entries are given UIDs when loading saves
2021-12-19 00:00:06 +01:00
while(True):
uid = int.from_bytes(os.urandom(4), "little", signed=True)
if(uid not in vars.worldinfo_u):
break
vars.worldinfo_u[uid] = vars.worldinfo[-1]
vars.worldinfo[-1]["uid"] = uid
Make sure `vars.wifolders_u` is set up properly on loading a save
2022-01-04 20:13:36 +01:00
if(vars.worldinfo[-1]["folder"] is not None):
vars.wifolders_u[vars.worldinfo[-1]["folder"]].append(vars.worldinfo[-1])
Fix numerous problems in the Lua world info API
2021-12-12 01:11:38 +01:00
stablesortwi()
vars.worldinfo_i = [wi for wi in vars.worldinfo if wi["init"]]
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Clear import data
vars.importjs = {}
# Reset current save
vars.savedir = getcwd()+"\stories"
# Refresh game screen
vars.laststory = None
emit('from_server', {'cmd': 'setstoryname', 'data': vars.laststory}, broadcast=True)
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(False)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
sendwi()
emit('from_server', {'cmd': 'setmemory', 'data': vars.memory}, broadcast=True)
emit('from_server', {'cmd': 'setanote', 'data': vars.authornote}, broadcast=True)
Author's Note Template
2021-12-30 05:43:36 +01:00
emit('from_server', {'cmd': 'setanotetemplate', 'data': vars.authornotetemplate}, broadcast=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
refresh_story()
emit('from_server', {'cmd': 'setgamestate', 'data': 'ready'}, broadcast=True)
emit('from_server', {'cmd': 'hidegenseqs', 'data': ''}, broadcast=True)
#==================================================================#
# Import an aidg.club prompt and start a new game with it.
#==================================================================#
def importAidgRequest(id):
exitModes()
urlformat = "https://prompts.aidg.club/api/"
req = requests.get(urlformat+id)
if(req.status_code == 200):
js = req.json()
# Import game state
vars.gamestarted = True
vars.prompt = js["promptContent"]
vars.memory = js["memory"]
vars.authornote = js["authorsNote"]
Consistent capitalization of "Author's note"
2021-12-30 07:48:25 +01:00
vars.authornotetemplate = "[Author's note: <|>]"
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
vars.actions = structures.KoboldStoryRegister()
vars.worldinfo = []
Fix numerous problems in the Lua world info API
2021-12-12 01:11:38 +01:00
vars.worldinfo_i = []
Upload bridge.lua, default.lua and some Lua libs base64 inspect json.lua Lua-hashings Lua-nums Moses mt19937ar-lua Penlight Serpent
2021-12-11 01:45:57 +01:00
vars.worldinfo_u = {}
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
vars.wifolders_d = {}
vars.wifolders_l = []
Upload bridge.lua, default.lua and some Lua libs base64 inspect json.lua Lua-hashings Lua-nums Moses mt19937ar-lua Penlight Serpent
2021-12-11 01:45:57 +01:00
vars.wifolders_u = {uid: [] for uid in vars.wifolders_d}
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
vars.lastact = ""
Add Lua API for reading the text the user submitted to the AI
2021-12-12 07:52:42 +01:00
vars.submission = ""
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
vars.lastctx = ""
num = 0
for wi in js["worldInfos"]:
vars.worldinfo.append({
"key": wi["keys"],
"keysecondary": wi.get("keysecondary", ""),
"content": wi["entry"],
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
"comment": wi.get("comment", ""),
"folder": wi.get("folder", None),
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
"num": num,
"init": True,
"selective": wi.get("selective", False),
Upload bridge.lua, default.lua and some Lua libs base64 inspect json.lua Lua-hashings Lua-nums Moses mt19937ar-lua Penlight Serpent
2021-12-11 01:45:57 +01:00
"constant": wi.get("constant", False),
"uid": None,
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
})
Upload bridge.lua, default.lua and some Lua libs base64 inspect json.lua Lua-hashings Lua-nums Moses mt19937ar-lua Penlight Serpent
2021-12-11 01:45:57 +01:00
while(True):
uid = int.from_bytes(os.urandom(4), "little", signed=True)
if(uid not in vars.worldinfo_u):
break
vars.worldinfo_u[uid] = vars.worldinfo[-1]
vars.worldinfo[-1]["uid"] = uid
Make sure `vars.wifolders_u` is set up properly on loading a save
2022-01-04 20:13:36 +01:00
if(vars.worldinfo[-1]["folder"]) is not None:
vars.wifolders_u[vars.worldinfo[-1]["folder"]].append(vars.worldinfo[-1])
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
num += 1
Fix numerous problems in the Lua world info API
2021-12-12 01:11:38 +01:00
for uid in vars.wifolders_l + [None]:
vars.worldinfo.append({"key": "", "keysecondary": "", "content": "", "comment": "", "folder": uid, "num": None, "init": False, "selective": False, "constant": False, "uid": None})
Make sure uninitialized WI entries are given UIDs when loading saves
2021-12-19 00:00:06 +01:00
while(True):
uid = int.from_bytes(os.urandom(4), "little", signed=True)
if(uid not in vars.worldinfo_u):
break
vars.worldinfo_u[uid] = vars.worldinfo[-1]
vars.worldinfo[-1]["uid"] = uid
Make sure `vars.wifolders_u` is set up properly on loading a save
2022-01-04 20:13:36 +01:00
if(vars.worldinfo[-1]["folder"] is not None):
vars.wifolders_u[vars.worldinfo[-1]["folder"]].append(vars.worldinfo[-1])
Fix numerous problems in the Lua world info API
2021-12-12 01:11:38 +01:00
stablesortwi()
vars.worldinfo_i = [wi for wi in vars.worldinfo if wi["init"]]
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Reset current save
vars.savedir = getcwd()+"\stories"
# Refresh game screen
vars.laststory = None
emit('from_server', {'cmd': 'setstoryname', 'data': vars.laststory}, broadcast=True)
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(False)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
sendwi()
emit('from_server', {'cmd': 'setmemory', 'data': vars.memory}, broadcast=True)
emit('from_server', {'cmd': 'setanote', 'data': vars.authornote}, broadcast=True)
Author's Note Template
2021-12-30 05:43:36 +01:00
emit('from_server', {'cmd': 'setanotetemplate', 'data': vars.authornotetemplate}, broadcast=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
refresh_story()
emit('from_server', {'cmd': 'setgamestate', 'data': 'ready'}, broadcast=True)
#==================================================================#
# Import World Info JSON file
#==================================================================#
def wiimportrequest():
importpath = fileops.getloadpath(vars.savedir, "Select World Info File", [("Json", "*.json")])
if(importpath):
file = open(importpath, "rb")
js = json.load(file)
if(len(js) > 0):
# If the most recent WI entry is blank, remove it.
if(not vars.worldinfo[-1]["init"]):
del vars.worldinfo[-1]
# Now grab the new stuff
num = len(vars.worldinfo)
for wi in js:
vars.worldinfo.append({
"key": wi["keys"],
"keysecondary": wi.get("keysecondary", ""),
"content": wi["entry"],
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
"comment": wi.get("comment", ""),
"folder": wi.get("folder", None),
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
"num": num,
"init": True,
"selective": wi.get("selective", False),
Upload bridge.lua, default.lua and some Lua libs base64 inspect json.lua Lua-hashings Lua-nums Moses mt19937ar-lua Penlight Serpent
2021-12-11 01:45:57 +01:00
"constant": wi.get("constant", False),
"uid": None,
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
})
Upload bridge.lua, default.lua and some Lua libs base64 inspect json.lua Lua-hashings Lua-nums Moses mt19937ar-lua Penlight Serpent
2021-12-11 01:45:57 +01:00
while(True):
uid = int.from_bytes(os.urandom(4), "little", signed=True)
if(uid not in vars.worldinfo_u):
break
vars.worldinfo_u[uid] = vars.worldinfo[-1]
vars.worldinfo[-1]["uid"] = uid
Make sure `vars.wifolders_u` is set up properly on loading a save
2022-01-04 20:13:36 +01:00
if(vars.worldinfo[-1]["folder"]) is not None:
vars.wifolders_u[vars.worldinfo[-1]["folder"]].append(vars.worldinfo[-1])
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
num += 1
Make sure uninitialized WI entries are given UIDs when loading saves
2021-12-19 00:00:06 +01:00
for uid in [None]:
vars.worldinfo.append({"key": "", "keysecondary": "", "content": "", "comment": "", "folder": uid, "num": None, "init": False, "selective": False, "constant": False, "uid": None})
while(True):
uid = int.from_bytes(os.urandom(4), "little", signed=True)
if(uid not in vars.worldinfo_u):
break
vars.worldinfo_u[uid] = vars.worldinfo[-1]
vars.worldinfo[-1]["uid"] = uid
Make sure `vars.wifolders_u` is set up properly on loading a save
2022-01-04 20:13:36 +01:00
if(vars.worldinfo[-1]["folder"] is not None):
vars.wifolders_u[vars.worldinfo[-1]["folder"]].append(vars.worldinfo[-1])
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
print("{0}".format(vars.worldinfo[0]))
# Refresh game screen
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(False)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
sendwi()
#==================================================================#
# Starts a new story
#==================================================================#
def newGameRequest():
# Leave Edit/Memory mode before continuing
exitModes()
# Clear vars values
vars.gamestarted = False
vars.prompt = ""
vars.memory = ""
vars.actions = structures.KoboldStoryRegister()
vars.authornote = ""
Author's Note Template
2021-12-30 05:43:36 +01:00
vars.authornotetemplate = vars.setauthornotetemplate
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
vars.worldinfo = []
Fix numerous problems in the Lua world info API
2021-12-12 01:11:38 +01:00
vars.worldinfo_i = []
Upload bridge.lua, default.lua and some Lua libs base64 inspect json.lua Lua-hashings Lua-nums Moses mt19937ar-lua Penlight Serpent
2021-12-11 01:45:57 +01:00
vars.worldinfo_u = {}
WI folders and WI drag-and-drop
2021-12-05 05:59:28 +01:00
vars.wifolders_d = {}
vars.wifolders_l = []
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
vars.lastact = ""
Add Lua API for reading the text the user submitted to the AI
2021-12-12 07:52:42 +01:00
vars.submission = ""
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
vars.lastctx = ""
# Reset current save
vars.savedir = getcwd()+"\stories"
# Refresh game screen
vars.laststory = None
emit('from_server', {'cmd': 'setstoryname', 'data': vars.laststory}, broadcast=True)
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
sendwi()
emit('from_server', {'cmd': 'setmemory', 'data': vars.memory}, broadcast=True)
emit('from_server', {'cmd': 'setanote', 'data': vars.authornote}, broadcast=True)
Author's Note Template
2021-12-30 05:43:36 +01:00
emit('from_server', {'cmd': 'setanotetemplate', 'data': vars.authornotetemplate}, broadcast=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
setStartState()
Add Memory box to Random Story dialog and "Random Story Persist"
2021-12-30 05:15:59 +01:00
def randomGameRequest(topic, memory=""):
if(vars.noai):
newGameRequest()
Sync memory properly after random game request
2022-01-20 21:14:55 +01:00
vars.memory = memory
emit('from_server', {'cmd': 'setmemory', 'data': vars.memory}, broadcast=True)
Add Memory box to Random Story dialog and "Random Story Persist"
2021-12-30 05:15:59 +01:00
return
Allow retry button to regenerate random story
2021-12-27 01:52:56 +01:00
vars.recentrng = topic
Retrying random story generation now also remembers memory
2022-01-04 20:40:10 +01:00
vars.recentrngm = memory
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
newGameRequest()
Detect and show properly when story is unsaved
2022-01-18 23:20:45 +01:00
setgamesaved(False)
Make sure we don't keep the trimmed memory in `randomGameRequest()`
2021-12-30 07:45:27 +01:00
_memory = memory
Add Memory box to Random Story dialog and "Random Story Persist"
2021-12-30 05:15:59 +01:00
if(len(memory) > 0):
Make sure we don't keep the trimmed memory in `randomGameRequest()`
2021-12-30 07:45:27 +01:00
_memory = memory.rstrip() + "\n\n"
vars.memory = _memory + "You generate the following " + topic + " story concept :"
Lua API fixes * `print()` and `warn()` now work correctly with `nil` arguments * Typo: `gpt-neo-1.3M` has been corrected to `gpt-neo-1.3B` * Regeneration is no longer triggered when writing to `keysecondary` of a non-selective key * Handle `genamt` changes in generation modifier properly * Writing to `kobold.settings.numseqs` from a generation modifier no longer affects * Formatting options in `kobold.settings` have been fixed * Added aliases for setting names * Fix behaviour of editing story chunks from a generation modifier * Warnings are now yellow instead of red * kobold.logits is now the raw logits prior to being filtered, like the documentation says, rather than after being filtered * Some erroneous comments and error messages have been corrected * These parts of the API have now been implemented properly: * `compute_context()` methods * `kobold.authorsnote` * `kobold.restart_generation()`
2021-12-20 02:18:28 +01:00
vars.lua_koboldbridge.feedback = None
Fix random story generator when No Prompt Gen is enabled
2021-12-27 01:40:20 +01:00
actionsubmit("", force_submit=True, force_prompt_gen=True)
Add Memory box to Random Story dialog and "Random Story Persist"
2021-12-30 05:15:59 +01:00
vars.memory = memory
Sync memory properly after random game request
2022-01-20 21:14:55 +01:00
emit('from_server', {'cmd': 'setmemory', 'data': vars.memory}, broadcast=True)
Add Memory box to Random Story dialog and "Random Story Persist"
2021-12-30 05:15:59 +01:00
Universal Model Settings No longer depends on a local config file enabling the configuration to work in --colab mode.
2022-01-20 10:11:11 +01:00
# Load desired settings from both the model and the users config file
Add Memory box to Random Story dialog and "Random Story Persist"
2021-12-30 05:15:59 +01:00
loadsettings()
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
Prevent tokenizer from taking extra time the first time it's used
2022-01-18 04:55:25 +01:00
# Prevent tokenizer from taking extra time the first time it's used
def __preempt_tokenizer():
if("tokenizer" not in globals()):
return
tokenizer.decode([25678, 559])
tokenizer.encode("eunoia")
threading.Thread(target=__preempt_tokenizer).start()
Use original TPU backend if possible
2022-01-16 05:31:07 +01:00
# Precompile TPU backend if required
if(vars.model in ("TPUMeshTransformerGPTJ",)):
soft_tokens = tpumtjgetsofttokens()
if(vars.dynamicscan or (not vars.nogenmod and vars.has_genmod)):
threading.Thread(
target=tpu_mtj_backend.infer_dynamic,
args=(np.tile(np.uint32((23403, 727, 20185)), (vars.numseqs, 1)),),
kwargs={
"soft_embeddings": vars.sp,
"soft_tokens": soft_tokens,
"gen_len": 1,
"use_callback": False,
"numseqs": vars.numseqs,
"excluded_world_info": list(set() for _ in range(vars.numseqs)),
},
).start()
else:
threading.Thread(
target=tpu_mtj_backend.infer_static,
args=(np.uint32((23403, 727, 20185)),),
kwargs={
"soft_embeddings": vars.sp,
"soft_tokens": soft_tokens,
"gen_len": 1,
"numseqs": vars.numseqs,
},
).start()
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
#==================================================================#
# Final startup commands to launch Flask app
#==================================================================#
Compile TPU backend in background
2022-01-07 19:47:21 +01:00
print("", end="", flush=True)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
if __name__ == "__main__":
Compile TPU backend in background
2022-01-07 19:47:21 +01:00
print("{0}\nStarting webserver...{1}".format(colors.GREEN, colors.END), flush=True)
Restore missing `loadsettings()` call
2021-12-14 00:39:33 +01:00
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
# Start Flask/SocketIO (Blocking, so this must be last method!)
#socketio.run(app, host='0.0.0.0', port=5000)
if(vars.remote):
Ngrok Support Not recommended for home users due to DDoS risks, but might make Colab tunnels more reliable.
2021-11-29 18:11:14 +01:00
if(args.ngrok):
from flask_ngrok import _run_ngrok
cloudflare = _run_ngrok()
else:
from flask_cloudflared import _run_cloudflared
cloudflare = _run_cloudflared(5000)
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
with open('cloudflare.log', 'w') as cloudflarelog:
Update aiserver.py
2021-12-14 16:26:23 +01:00
cloudflarelog.write("KoboldAI has finished loading and is available at the following link : " + cloudflare)
Accept command line arguments in `KOBOLDAI_ARGS` environment var So that you can use gunicorn or whatever with command-line arguments by passing the arguments in an environment variable.
2022-01-04 23:11:14 +01:00
print(format(colors.GREEN) + "KoboldAI has finished loading and is available at the following link : " + cloudflare + format(colors.END))
Catch Lua errors properly
2021-12-13 08:32:09 +01:00
vars.serverstarted = True
Breakmodel automation The only changes are a small addition to the breakmodel section where GPU0 is automatically chosen if the CLI options are used without specifying breakmodel. Lineendings have been changed to Linux formatting for compatibility reasons.
2021-11-14 03:13:52 +01:00
socketio.run(app, host='0.0.0.0', port=5000)
else:
import webbrowser
webbrowser.open_new('http://localhost:5000')
Accept command line arguments in `KOBOLDAI_ARGS` environment var So that you can use gunicorn or whatever with command-line arguments by passing the arguments in an environment variable.
2022-01-04 23:11:14 +01:00
print("{0}Server started!\nYou may now connect with a browser at http://127.0.0.1:5000/{1}".format(colors.GREEN, colors.END))
Catch Lua errors properly
2021-12-13 08:32:09 +01:00
vars.serverstarted = True
Use eventlet instead of gevent-websocket
2021-12-13 23:19:04 +01:00
socketio.run(app, port=5000)
Accept command line arguments in `KOBOLDAI_ARGS` environment var So that you can use gunicorn or whatever with command-line arguments by passing the arguments in an environment variable.
2022-01-04 23:11:14 +01:00
else:
Compile TPU backend in background
2022-01-07 19:47:21 +01:00
print("{0}\nServer started in WSGI mode!{1}".format(colors.GREEN, colors.END), flush=True)