rixty/KoboldAI-Client
1
1
Fork 0
mirror of https://github.com/KoboldAI/KoboldAI-Client.git synced 2025-06-05 21:59:24 +02:00
Code Issues Projects Releases Wiki Activity
3,883 Commits 1 Branch 5 Tags
b256a8fbc7f64d1f917c83e2294d67ba321844b5
Commit Graph

57 Commits

Author SHA1 Message Date
somebody
b256a8fbc7 Debug 2023-04-27 19:33:03 -05:00
onesome
467f2f25eb More loading fixes 2023-04-26 16:58:33 -05:00
onesome
d4f7b60dc9 Fix for multiple paths 2023-04-26 16:49:12 -05:00
onesome
6776a71532 Add more info to custom model error 2023-04-26 16:36:52 -05:00
onesome
bbf4963d6e Fix custmodpth stuff for hf loading 2023-04-26 16:18:45 -05:00
onesome
c146ae9d84 Delete legacy gpt2 custom loader 2023-04-26 16:07:18 -05:00
onesome
9579298df7 Better fallback 2023-04-25 22:28:07 -05:00
onesome
6e3aebc1ea Zap debug 2023-04-25 21:13:17 -05:00
onesome
d496e861f4 Undo pretty code because I haven't cracked the jax enigma yet 2023-04-25 21:11:49 -05:00
onesome
1db9d9ba61 Lazyload: Whoops 2023-04-25 18:46:54 -05:00
onesome
e28e268a2d Use safetensors only when available 2023-04-25 18:32:37 -05:00
onesome
0268305cfe Change fallback notifications to warnings 2023-04-25 18:26:49 -05:00
onesome
b8bef641ff Merge branch 'united' of https://github.com/henk717/KoboldAI into model-structure-and-maybe-rwkv 2023-04-25 16:54:53 -05:00
somebody
f9fb5eba89 Remove debug 2023-04-15 18:56:49 -05:00
somebody
5dd67d027a Workaround for socketio context errors for loading 2023-04-15 18:54:21 -05:00
somebody
08b4e317ff Fix double slashing 2023-04-15 13:30:05 -05:00
somebody
d3a73aaeba Fix api 2023-04-15 13:17:20 -05:00
somebody
4dcf570407 Fix legacy model loading 2023-04-15 12:57:35 -05:00
one-some
1b500c7179 Merge pull request #5 from LostRuins/concedo_api 
Added stop sequences functionality for API calls
 2023-04-15 10:51:31 -05:00
somebody
2b950f08d3 Remove legacy no accelerate fallback code 
Was causing issues with disk cache the old code had a
`and not utils.HAS_ACCELERATE` preceding it (a variable which no longer
exists), and since disk cache is accelerate only, there was no disk
handling code in here. Anyway its bad so blast it
 2023-04-15 10:47:31 -05:00
Concedo
9705b7b79c increase API version (+1 squashed commits) 
Squashed commits:

[c168c08] Added stop sequences functionality for API calls
 2023-04-15 18:09:53 +08:00
somebody
ea8df4c0d3 Merge branch 'united' of https://github.com/henk717/KoboldAI into model-structure-and-maybe-rwkv 2023-04-14 20:38:56 -05:00
somebody
38c53191d3 possible fix for cache dl thing 2023-04-14 20:25:03 -05:00
somebody
334c09606b Fix for tokenizer stuff on pythia 2023-04-09 18:23:58 -05:00
somebody
8412f83ce5 Breakmodel: Fix typo 2023-04-03 18:41:18 -05:00
somebody
77f0797b1a Model fix 2023-04-02 15:47:52 -05:00
somebody
9d70646e4d Lazyload: Safetensors 2023-04-02 15:40:34 -05:00
somebody
91bb433b5f GenericTokenizer: Fall back to defined tokenizer 
Shouldn't be relied on for model-agnostic code, but for loading
processes where you know the tokenizer class used it should be okie
dokie
 2023-03-19 19:03:20 -05:00
somebody
ffe85ce8a1 Modeling: Fix logits processors (probs, biasing, lua) 2023-03-17 16:56:47 -05:00
somebody
692dbfeb37 Merge branch 'united' of https://github.com/henk717/KoboldAI into model-structure-and-maybe-rwkv 2023-03-17 16:20:13 -05:00
somebody
8d0bc404a5 Model: More Jax import fixes and formatting 2023-03-17 15:36:44 -05:00
somebody
03af06638c Modeling: Maybe fix samplers 2023-03-13 20:42:35 -05:00
somebody
b93c339145 Model: Lazyload backends 2023-03-13 20:29:29 -05:00
somebody
938c97b75a RWKV: Fix yet another typo 2023-03-13 19:39:19 -05:00
somebody
14b2543c7c RWKV: Fix typo 2023-03-13 19:36:58 -05:00
somebody
b10b201701 Model: Add basic RWKV implementation 2023-03-13 19:34:38 -05:00
somebody
bf8b60ac2d Model: Add GenericTokenizer 
Because Hugging Face doesnt have a consistant API across their own
libraries
 2023-03-13 17:36:58 -05:00
somebody
0320678b27 Model: WIP horde and API tests 2023-03-13 14:11:06 -05:00
somebody
cd8ccf0a5e Modeling: Add seed parameter to raw_generate 
Yahooo, decoupling from koboldai_vars. This makes the generation test
pass in `test_generation.py`, and makes full determinism outside of
core_generate work.
 2023-03-12 21:49:10 -05:00
somebody
d2696dad66 Remove ancient debug 2023-03-10 20:00:39 -06:00
somebody
8937e7f6df Model: Add basic tests 
We now do some basic tests for:
- hf torch loading (normal, lazy, lowmem)
- hf torch generation (shape batches, shape tokencount, faulty
  determinism)

Currently full determinism is failing; yahoo, the tests work!

All of the tests initally failed (note the test environment functions
different from the aiserver environment due to aiserver doing a lot of
initalizing stuff, working on phasing that out) but now only one fails.
Very useful for finding bugs!
 2023-03-10 20:00:39 -06:00
somebody
8c8bdfaf6a Model: Fix assorted bugs 
and ignore warnings in pytest
 2023-03-10 20:00:39 -06:00
somebody
3646aa9e83 Model: Respect model lazyload over kaivars 
kaivars dictates model config unless its from outside aiserver or
whatever.
 2023-03-10 20:00:39 -06:00
somebody
52095054a3 Model: Reformat and clean up 2023-03-10 20:00:39 -06:00
somebody
fb0b2f0467 Model: Ditch awful current_model hack 
thanks to whjms for spotting that this could be zapped
 2023-03-10 20:00:38 -06:00
somebody
cb6010d666 Model: Respect sampler bounds in torch 
A rather embarassing way to spend an hour debugging after I told myself
"I'd better remember to add this important thing to the torch side".

Samplers were being applied when in their "off values" causing
boring mathmatical operations to take place (ie anything x 0 is always
0)
 2023-03-07 21:14:37 -06:00
somebody
6b45367cc7 Model: Rename "Colab" to "Basic API" 2023-03-07 17:07:01 -06:00
somebody
dc3dd38c4e Merge branch 'united' of https://github.com/henk717/KoboldAI into model-structure-and-maybe-rwkv 2023-03-07 17:02:28 -06:00
somebody
3b05359e4b Model: Refuse to serve certain models over the API 2023-03-07 16:54:08 -06:00
somebody
3a128e76b4 Attempts at dynamic wi fixes 2023-03-07 16:33:23 -06:00