rixty/KoboldAI-Client
1
1
Fork 0
mirror of https://github.com/KoboldAI/KoboldAI-Client.git synced 2025-06-05 21:59:24 +02:00
Code Issues Projects Releases Wiki Activity
4,612 Commits 1 Branch 5 Tags
fe53cb2f7eb852b0de45c26165b857707b1d1bd8
Commit Graph

4612 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Llama
fe53cb2f7e Merge pull request #70 from pi6am/feat/exllama-unban-eos 
Hook up use_default_badwordids in exllama
 2023-08-29 23:17:37 -07:00
Llama
d6ed75f993 Hook up use_default_badwordids in exllama 
Use the value of the use_default_badwordids setting to configure
bad_words_ids. Also add square brackets to bad_words_ids if the
use_default_badwordids setting is True. Fix an issue with
attempting to use the tokenizer too early, and fix an exception
populating Lua bridge data when zero tokens are generated, which
can now happen if use_default_badwordids is False and the first
token generated is EOS.
 2023-08-29 23:08:51 -07:00
Llama
36f53cc915 Merge pull request #69 from pi6am/merge/united-exllama 
Merge branch henk717/united into exllama
 2023-08-29 19:54:14 -07:00
Llama
b5b0e3faea Merge branch 'henk717/united' into merge/united-exllama 2023-08-29 19:48:44 -07:00
Henk
49fa63052f Allow EOS unbanning 2023-08-29 20:51:09 +02:00
Henk
d77acf17eb Transformers 4.32.1 2023-08-28 22:55:33 +02:00
Henk
ec4cecf2f8 Restore Horde UI for local users 2023-08-28 21:46:30 +02:00
Henk
c29a5019b3 Disable Horde UI due to lockups 2023-08-28 20:02:29 +02:00
Llama
6e64763b63 Merge pull request #68 from pi6am/merge/united-exllama 
Merge henk717/united into exllama
 2023-08-28 09:57:35 -07:00
Llama
2c48e05f7c Add exllama dependency back to requirements. 2023-08-28 09:52:31 -07:00
Llama
6151cbd053 Merge branch 'united' into merge/united-exllama 2023-08-28 09:32:19 -07:00
henk717
030ca26aa3 Merge pull request #445 from db0/scribe 
fix: Avoid conflicting args
 2023-08-28 18:04:15 +02:00
db0
b20b910568 remove debug 2023-08-28 17:49:34 +02:00
db0
31161409f6 fix argsv placement 2023-08-28 17:47:52 +02:00
db0
25b78abcd2 Merge branch 'united' into scribe 2023-08-28 17:45:49 +02:00
Llama
5229987ab7 Merge pull request #66 from pi6am/feat/exllama-config 
Modify exllama to load unrenamed gptq quantized models
 2023-08-28 00:09:50 -07:00
Llama
554af7b175 Modify exllama to load unrenamed gptq quantized models 
Read config.json and enable exllama loading if the model has a
`quantization_config` with `quant_methdod` of `gptq`. Note that this
implementation is limited and only supports model.safetensors.
That said, this supports loading popular gptq quantized models
without renaming or symlinking the model file.
 2023-08-27 23:56:02 -07:00
Llama
812df5ea56 Merge pull request #65 from pi6am/feat/exllama-badwords 
Add the eos token to exllama bad words.
 2023-08-27 17:03:25 -07:00
Llama
08ff7c138c Add the eos token to exllama bad words. 
The bos token was already hardcoded as a bad word id.
Store badwords in a list and iterate over them during generation.
Add the Llama eos token to the list of bad words.
Also support "single line mode", which adds newline (13) to badwords.
 2023-08-27 16:34:52 -07:00
Henk
3e0b8279f2 Rename GPTQ loading 2023-08-27 20:51:14 +02:00
Llama
0d150e412e Merge pull request #64 from pi6am/fix/multinomial-workaround 
Resample to work around a bug in torch.multinomial
 2023-08-26 22:42:21 -07:00
Llama
b7e38b4757 Resample to work around a bug in torch.multinomial 
There is a bug in PyTorch 2.0.1 that allows torch.multinomial to
sometimes choose elements that have zero probability. Since
this is uncommon we can continue to use torch.multinomial as
long as we verify that the results are valid. If they aren't,
try again until the probability of each selected token is positive.
 2023-08-26 22:26:26 -07:00
Henk
290f2ce05e CPU only warning 2023-08-26 00:03:28 +02:00
db0
4b2d591354 avoid conflictinng sys args 2023-08-25 15:05:36 +02:00
Henk
f40236c04a Modern llama tokenizer 2023-08-25 14:27:44 +02:00
Henk
2887467eec Safetensors 0.3.3 2023-08-24 14:30:44 +02:00
Henk
39c1b39b4a Fix markers 2023-08-23 21:42:06 +02:00
Henk
5d9f180489 Fix typo 2023-08-23 21:36:26 +02:00
Henk
85810cd3fd AutoGPTQ for Colab 2023-08-23 21:30:58 +02:00
Henk
c20ea949d7 Fix duplicate safetensors 2023-08-23 21:02:11 +02:00
Henk
91155ed2f3 HF dependencies 2023-08-23 20:34:40 +02:00
Llama
b1895de518 Merge pull request #63 from pi6am/feat/exllama-stoppers 
Add stopper hooks suppport to exllama
 2023-08-22 23:14:00 -07:00
Llama
b96d5d8646 Add stopper hooks suppport to exllama 2023-08-22 23:06:16 -07:00
Henk
f66173f2a0 Git gonna git 2023-08-22 20:43:44 +02:00
Henk
2b6dcbe55e New Horde Worker 2023-08-22 20:40:04 +02:00
Henk
3f438fda53 Scribe name instead of worker name 2023-08-22 18:56:23 +02:00
Henk
e5aca6fdad Cleaned horde 2023-08-22 18:43:29 +02:00
Henk
69c794506b HF 4.32 2023-08-22 17:48:00 +02:00
Henk
4b482a0619 Pending trick 2023-08-22 14:58:44 +02:00
Henk
b41bf99b55 Cleanup 2023-08-22 14:00:05 +02:00
Henk
179c4ad07f Restore UI 2023-08-22 13:58:02 +02:00
Henk
f570787077 Allow worker to stop 2023-08-22 13:38:28 +02:00
0cc4m
22fd49937a Merge pull request #62 from pi6am/fix/exllama-eos-space 
Strip the eos token from exllama generations.
 2023-08-22 08:06:02 +02:00
Henk
401cc1609a Kaiemb branch 2023-08-21 19:05:35 +02:00
db0
148a7c21b8 using stop() 2023-08-21 19:02:15 +02:00
Henk
d9815d4b1f New worker fixes 2023-08-21 17:52:21 +02:00
Henk
8abb5746f8 Add bridge back 2023-08-21 16:50:17 +02:00
Henk
7b8fba31f7 Git is stubborn 2023-08-21 16:46:02 +02:00
Henk
be8f527911 Horde URL fixes 2023-08-21 16:44:58 +02:00
Henk
a7251fa599 Bridge settings 2023-08-21 16:44:09 +02:00