rixty/SillyTavern
1
0
Fork 0
mirror of https://github.com/SillyTavern/SillyTavern.git synced 2025-03-02 10:57:45 +01:00
Code Issues Packages Projects Releases Wiki Activity

vLLM fixes

Browse Source 
* Enable seed field for vLLM

* Enable beam search for vLLM

* Set the default length penalty to 1
(There is validation error from vLLM when beam search is disabled and the value is not equal to 1)
This commit is contained in:
sasha0552 2024-05-19 04:28:01 +00:00 committed by GitHub
parent d9b55df883
commit db5e2d95c2
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 3 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
public/index.html
View File

@ -1324,7 +1324,7 @@
</div>
</div>
</div>
<div data-newbie-hidden data-tg-type="ooba" name="beamSearchBlock" class="wide100p">
<div data-newbie-hidden data-tg-type="ooba, vllm" name="beamSearchBlock" class="wide100p">
<h4 class="wide100p textAlignCenter">
<label>
<span data-i18n="Beam search">Beam Search</span>
@ -1407,7 +1407,7 @@
</label>
</div>
</div>
<div data-tg-type="mancer, ooba, koboldcpp, aphrodite, llamaccp, ollama" data-newbie-hidden class="flex-container flexFlowColumn alignitemscenter flexBasis48p flexGrow flexShrink gap0">
<div data-tg-type="mancer, ooba, koboldcpp, vllm, aphrodite, llamaccp, ollama" data-newbie-hidden class="flex-container flexFlowColumn alignitemscenter flexBasis48p flexGrow flexShrink gap0">
<small data-i18n="Seed" class="textAlignCenter">Seed</small>
<input type="number" id="seed_textgenerationwebui" class="text_pole textAlignCenter" min="-1" value="-1" maxlength="100" />
</div>

2
public/scripts/textgen-settings.js
View File

@ -659,7 +659,7 @@ jQuery(function () {
'no_repeat_ngram_size_textgenerationwebui': 0,
'min_length_textgenerationwebui': 0,
'num_beams_textgenerationwebui': 1,
'length_penalty_textgenerationwebui': 0,
'length_penalty_textgenerationwebui': 1,
'penalty_alpha_textgenerationwebui': 0,
'typical_p_textgenerationwebui': 1, // Added entry
'guidance_scale_textgenerationwebui': 1,