rixty/KoboldAI-Client
1
1
Fork 0
mirror of https://github.com/KoboldAI/KoboldAI-Client.git synced 2025-06-05 21:59:24 +02:00
Code Issues Projects Releases Wiki Activity
4,219 Commits 1 Branch 5 Tags
c43d60772b95e1479b81e494aee4969eaac4b72d
Commit Graph

4219 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Henk
c43d60772b BnB dependency check 2023-07-15 18:56:13 +02:00
Henk
160effb9ea Add 4-bit BnB toggle 2023-07-15 18:20:10 +02:00
Henk
c833ab23e4 Add scipy 2023-07-15 14:57:42 +02:00
Henk
2c50d5d092 Don't ruin breakmodel 2023-07-15 14:14:06 +02:00
Henk
003105ae8b Bump BnB 2023-07-15 04:26:58 +02:00
Henk
1f045110a4 Basic 4-bit backend 2023-07-15 02:49:31 +02:00
henk717
6dd73287c9 Merge pull request #391 from one-some/basic-hf-backend 
Basic HF backend
 2023-07-15 02:20:38 +02:00
onesome
7e2e75070b Fix prioritization in load from dir 
In the future the module/backend list should probably contain objects
that would make this whole deal a lot less hackier.
 2023-07-14 18:47:52 -05:00
onesome
afa8766ea6 Add is_valid 2023-07-14 18:01:18 -05:00
somebody
8549c7c896 Basic backend module prioritization 
not secure; we're loading these modules so they can obviously execute
code that manipulates the prioritization
 2023-07-12 19:03:49 -05:00
somebody
f67cb7fa05 Make basic hf independant from hf 2023-07-12 18:36:30 -05:00
somebody
d17ce8461d Use device_map="auto" 2023-07-12 17:27:48 -05:00
somebody
60473d4c23 Fix and add some documentation to basic hf backend 2023-07-12 17:16:05 -05:00
Henk
1fed1b0524 New dependencies 2023-07-12 22:00:17 +02:00
onesome
8077d6c3f9 Self-contained sampler patch (Don't merge) 
Completely untested 3:00 AM code; beware! I will test and add more
documentation tomorrow.
 2023-07-12 03:22:43 -05:00
henk717
922fcdd6ac Merge pull request #392 from ebolam/Model_Plugins 
Fix for UI2 model loading not showing progress
 2023-07-11 19:02:17 +02:00
ebolam
56e1c5df3a Fix for UI2 model loading not showing progress 2023-07-10 20:59:16 -04:00
henk717
95913aebb8 Merge pull request #390 from one-some/accelerate-offloading 
Fix bleeding edge model loads and add lazyload fallback
 2023-07-09 01:02:12 +02:00
somebody
20b4b4bcef Add basic hf backend 2023-07-08 17:12:16 -05:00
somebody
f9c38acea8 Merge branch 'accelerate-offloading' into basic-hf-backend 2023-07-08 17:04:23 -05:00
somebody
3928d86339 Fall back to unpatched HF 2023-07-08 14:36:45 -05:00
somebody
c2ee30af32 Add --panic to raise when loading fails 2023-07-08 14:04:46 -05:00
somebody
fd6f66a98d Patch _rebuild_from_type_v2 to not try converting LazyTensors to Tensors 2023-07-08 13:57:05 -05:00
henk717
60965b7b0c Merge pull request #389 from one-some/accelerate-offloading 
Stub seek_offset for cache sorting in load loop
 2023-07-07 22:40:08 +02:00
somebody
802929f5f2 Patch safetensors again 2023-07-07 14:54:40 -05:00
somebody
35f3687667 Merge branch 'united' of https://github.com/henk717/KoboldAI into accelerate-offloading 2023-07-07 14:54:12 -05:00
somebody
cfe1f5b514 Stub seek_offset for cache sorting in load loop 
The way Safetensors individual weight loading is implemented doesn't
take full advantage of the cache ordering system thing, so this can just
be left at zero for now.
 2023-07-07 14:49:46 -05:00
Henk
76d21bb142 Universal ziproot for lazy_loader 2023-07-07 13:37:32 +02:00
henk717
f0d161b9c6 Merge pull request #387 from LostRuins/concedo_united 
Updated Kobold Lite to v46
 2023-07-06 15:51:04 +02:00
Concedo
e3d8443edf Updated Kobold Lite to v46 2023-07-06 21:41:51 +08:00
Henk
548db92df5 No longer pin United 2023-07-06 14:55:54 +02:00
henk717
58c2dc08ae Merge pull request #386 from one-some/accelerate-offloading 
Use VE's patched load_from_state_dict on TPU for loading empty weights
 2023-07-06 14:54:54 +02:00
somebody
6b83944e9b Use VE's patched load_from_state_dict on TPU for loading empty weights 2023-07-05 18:36:57 -05:00
Henk
c72c0d0052 Pin United branch properly 2023-07-05 03:17:00 +02:00
Henk
22cf752411 Revert United colab pin since it doesn't work 2023-07-05 02:38:04 +02:00
Henk
1769691eda Possible fix 2 2023-07-05 02:35:16 +02:00
Henk
a192c74058 Possible colab URL fix 2023-07-05 02:28:48 +02:00
Henk
def39f4118 Typo fix 2023-07-05 00:02:29 +02:00
Henk
041d4135c8 Route TPU to stable commit while we fix the loader 2023-07-05 00:01:44 +02:00
Henk
16240878bc Restore --peft support 2023-07-04 20:42:29 +02:00
henk717
94c62a4f90 Merge pull request #385 from one-some/accelerate-offloading 
Accelerate offloading
 2023-07-04 03:14:51 +02:00
somebody
ebe478458f Just use string sort on key 
Doesn't seem to affect cache hit number
 2023-07-03 19:59:53 -05:00
somebody
bce1a907e5 Update aux device to depend on primary device 2023-07-03 19:36:31 -05:00
somebody
6f7e6422ef Actually get correct primary device 2023-07-03 19:04:48 -05:00
somebody
59c731f805 Fix static primary_device 
and some small cleanup
 2023-07-03 18:37:48 -05:00
somebody
32917fd651 Remove unused file open 2023-07-03 17:51:54 -05:00
somebody
7f869a54d8 Just use accelerate on tpu 2023-07-03 17:18:48 -05:00
somebody
1bb2d2621c Make TPU in line with new lazyload behavior 2023-07-03 17:12:07 -05:00
somebody
31a3046a18 Load empty modules without accelerate 2023-07-03 17:07:18 -05:00
somebody
686c3d1592 Don't patch lazyload on TPU 2023-07-03 16:52:18 -05:00