There have been a lot of reports from newer users who experience AI breakdown because not all models properly handle 2048 max tokens. 1024 is the only value that all models support and was the original value KoboldAI used. This commit reverts the decision to increase this to 2048, any existing configurations are not effected. Users who wish to increase the max tokens can do so themselves. Most models handle up to 1900 well (The GPT2 models are excluded), for many you can go all the way. (It is currently not yet known why some finetunes cause a decrease in maxtoken support,
In addition this commit contains a request for more consistent slider behavior, allowing the sliders to be changed at 0.01 intervals instead of some sliders being capped to 0.05.
Not only does iOS also have that issue that Chromium-based browsers
have, but it also has a different issue where it selects all text in the
last chunk of your story, so I added some code to deselect the text in
that case.
In Chromium based browsers you can now click the blank part of the
editor to submit changes. This is to maintain consistency with the
editor behaviour in Firefox which already did this when you clicked on
the blank part of the editor.
When you retry, it actually sends an empty submission, so if you have
add sentence spacing on, retrying could add an extra action with a
single space.
(cherry picked from commit 151407a001)
Specifically, we merge blank actions into the next action and we move
whitespace at the end of non-blank actions to the beginning of the next
action.
(cherry picked from commit 4b16600e49)
Transformers issued an important change for the OPT models breaking their compatibility with all older versions. In order for people to be able to use all models on the menu they need 4.20.1 so this is now forced in the dependencies making the update easier.
Transformers 4.20 now requires accelerate to be installed for some of the features we use in KoboldAI. This is now a required dependency for updated users.
Makes transformers 4.20 mandatory in the dependency lists, not because the old versions are no longer supported but because it contains fixes that benefit our users and this makes it easier for them to update to it. If you stick to an older version the OPT and XGLM workarounds we have in place will remain functional, but you miss on the enhancements newer transformers versions bring.
henk717
vfbd
Gnome Ann