54 Commits

Author	SHA1	Message	Date
Cohee	c91ab3b5e0	Add Kobold tokenization to best match logic. Fix not being able to stop group chat regeneration	2023-08-24 21:23:35 +03:00
Cohee	ab52af4fb5	Add support for Koboldcpp tokenization endpoint	2023-08-24 20:19:57 +03:00
Cohee	e77da62b85	Add padding to cache key. Fix Safari display issues. Fix 400 on empty translate. Reset bias cache on changing model.	2023-08-23 10:32:48 +03:00
Cohee	bc5fc67906	Put tokenizer functions to a separate file. Cache local models token counts	2023-08-23 02:38:43 +03:00