rixty/KoboldAI-Client
1
1
Fork 0
mirror of https://github.com/KoboldAI/KoboldAI-Client.git synced 2025-06-05 21:59:24 +02:00
Code Issues Projects Releases Wiki Activity

EOS Hiding Workaround (For models that want EOS behavior)

Browse Source
This commit is contained in:
Henk
2023-01-31 01:46:55 +01:00
parent fa496e6a50
commit b276c5ff15
1 changed files with 6 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
aiserver.py
View File

@@ -6491,6 +6491,12 @@ def applyoutputformatting(txt, no_sentence_trimming=False, no_single_line=False)
if len(txt) == 0: if len(txt) == 0:
return txt return txt
# Workaround for endoftext appearing in models that need it, you can supposedly do this directly with the tokenizer but it keeps showing up
# So for now since we only have two known end of text tokens and only one model that wishes to have its generation stopped this is easier
# If you see this and you wish to do a universal implementation for this, feel free just make sure to test it on all platforms - Henk
txt = txt.replace("<|endoftext|>", "")
txt = txt.replace("</s>", "")
# Use standard quotes and apostrophes # Use standard quotes and apostrophes
txt = utils.fixquotes(txt) txt = utils.fixquotes(txt)