5 Commits

Author	SHA1	Message	Date
Cohee	fd9c8b86e9	Token counter layout adjust	2023-11-05 22:55:10 +02:00
Cohee	f0c0949aa0	Add token ids viewer to tokenizer plugin	2023-11-05 22:45:37 +02:00
Cohee	85de505553	Move counter to plugin. Use chat context to get messages	2023-10-21 14:23:56 +03:00
Cohee	bc5fc67906	Put tokenizer functions to a separate file. Cache local models token counts	2023-08-23 02:38:43 +03:00
Cohee	edd41989fd	Initial commit	2023-07-20 20:32:15 +03:00