Cohee
|
57e845d0d7
|
Resolve best match tokenizer for itemization. Adjust styles of token counter
|
2023-11-06 20:25:59 +02:00 |
RossAscends
|
0effbebae7
|
tokenizer colorization underpinnings (inactive)
|
2023-11-06 18:04:47 +09:00 |
|
Cohee
|
e8ba328a14
|
Add text chunks display to token counter
|
2023-11-06 02:42:51 +02:00 |
|
Cohee
|
fd9c8b86e9
|
Token counter layout adjust
|
2023-11-05 22:55:10 +02:00 |
|
Cohee
|
f0c0949aa0
|
Add token ids viewer to tokenizer plugin
|
2023-11-05 22:45:37 +02:00 |
|
Cohee
|
85de505553
|
Move counter to plugin. Use chat context to get messages
|
2023-10-21 14:23:56 +03:00 |
|
Cohee
|
bc5fc67906
|
Put tokenizer functions to a separate file. Cache local models token counts
|
2023-08-23 02:38:43 +03:00 |
Cohee
|
edd41989fd
|
Initial commit
|
2023-07-20 20:32:15 +03:00 |