rixty/KoboldAI-Client
1
1
Fork 0
mirror of https://github.com/KoboldAI/KoboldAI-Client.git synced 2025-06-05 21:59:24 +02:00
Code Issues Projects Releases Wiki Activity
4,205 Commits 1 Branch 5 Tags
f67cb7fa055fb9ddc9a6f99385ae380b2ac0c0a4
Commit Graph

4205 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
somebody
f67cb7fa05 Make basic hf independant from hf 2023-07-12 18:36:30 -05:00
somebody
d17ce8461d Use device_map="auto" 2023-07-12 17:27:48 -05:00
somebody
60473d4c23 Fix and add some documentation to basic hf backend 2023-07-12 17:16:05 -05:00
onesome
8077d6c3f9 Self-contained sampler patch (Don't merge) 
Completely untested 3:00 AM code; beware! I will test and add more
documentation tomorrow.
 2023-07-12 03:22:43 -05:00
somebody
20b4b4bcef Add basic hf backend 2023-07-08 17:12:16 -05:00
somebody
f9c38acea8 Merge branch 'accelerate-offloading' into basic-hf-backend 2023-07-08 17:04:23 -05:00
somebody
3928d86339 Fall back to unpatched HF 2023-07-08 14:36:45 -05:00
somebody
c2ee30af32 Add --panic to raise when loading fails 2023-07-08 14:04:46 -05:00
somebody
fd6f66a98d Patch _rebuild_from_type_v2 to not try converting LazyTensors to Tensors 2023-07-08 13:57:05 -05:00
henk717
60965b7b0c Merge pull request #389 from one-some/accelerate-offloading 
Stub seek_offset for cache sorting in load loop
 2023-07-07 22:40:08 +02:00
somebody
802929f5f2 Patch safetensors again 2023-07-07 14:54:40 -05:00
somebody
35f3687667 Merge branch 'united' of https://github.com/henk717/KoboldAI into accelerate-offloading 2023-07-07 14:54:12 -05:00
somebody
cfe1f5b514 Stub seek_offset for cache sorting in load loop 
The way Safetensors individual weight loading is implemented doesn't
take full advantage of the cache ordering system thing, so this can just
be left at zero for now.
 2023-07-07 14:49:46 -05:00
Henk
76d21bb142 Universal ziproot for lazy_loader 2023-07-07 13:37:32 +02:00
henk717
f0d161b9c6 Merge pull request #387 from LostRuins/concedo_united 
Updated Kobold Lite to v46
 2023-07-06 15:51:04 +02:00
Concedo
e3d8443edf Updated Kobold Lite to v46 2023-07-06 21:41:51 +08:00
Henk
548db92df5 No longer pin United 2023-07-06 14:55:54 +02:00
henk717
58c2dc08ae Merge pull request #386 from one-some/accelerate-offloading 
Use VE's patched load_from_state_dict on TPU for loading empty weights
 2023-07-06 14:54:54 +02:00
somebody
6b83944e9b Use VE's patched load_from_state_dict on TPU for loading empty weights 2023-07-05 18:36:57 -05:00
Henk
c72c0d0052 Pin United branch properly 2023-07-05 03:17:00 +02:00
Henk
22cf752411 Revert United colab pin since it doesn't work 2023-07-05 02:38:04 +02:00
Henk
1769691eda Possible fix 2 2023-07-05 02:35:16 +02:00
Henk
a192c74058 Possible colab URL fix 2023-07-05 02:28:48 +02:00
Henk
def39f4118 Typo fix 2023-07-05 00:02:29 +02:00
Henk
041d4135c8 Route TPU to stable commit while we fix the loader 2023-07-05 00:01:44 +02:00
Henk
16240878bc Restore --peft support 2023-07-04 20:42:29 +02:00
henk717
94c62a4f90 Merge pull request #385 from one-some/accelerate-offloading 
Accelerate offloading
 2023-07-04 03:14:51 +02:00
somebody
ebe478458f Just use string sort on key 
Doesn't seem to affect cache hit number
 2023-07-03 19:59:53 -05:00
somebody
bce1a907e5 Update aux device to depend on primary device 2023-07-03 19:36:31 -05:00
somebody
6f7e6422ef Actually get correct primary device 2023-07-03 19:04:48 -05:00
somebody
59c731f805 Fix static primary_device 
and some small cleanup
 2023-07-03 18:37:48 -05:00
somebody
32917fd651 Remove unused file open 2023-07-03 17:51:54 -05:00
somebody
7f869a54d8 Just use accelerate on tpu 2023-07-03 17:18:48 -05:00
somebody
1bb2d2621c Make TPU in line with new lazyload behavior 2023-07-03 17:12:07 -05:00
somebody
31a3046a18 Load empty modules without accelerate 2023-07-03 17:07:18 -05:00
somebody
686c3d1592 Don't patch lazyload on TPU 2023-07-03 16:52:18 -05:00
one-some
39049e8a46 Merge pull request #20 from henk7171/accelerate-offloading 
Accelerate offloading fixes
 2023-07-03 16:06:20 -04:00
Henk
b603690e4c Link message fix 2023-07-03 14:28:33 +02:00
Henk
a0e21ad4e6 Restore finished loading messages 2023-07-03 14:16:21 +02:00
Henk
062e3ed27c Torch bump 2023-07-02 23:19:20 +02:00
Henk
4a45a0e551 Fix bar spam 2023-07-02 23:02:02 +02:00
Henk
51c4622694 Hide all Nvidia GPU's in CPU mode 2023-07-02 22:27:26 +02:00
Henk
81e72329af CPU fixes 2023-07-02 21:50:23 +02:00
henk717
c66f995c97 Merge pull request #382 from jojorne/jojorne-fix-move-delete-wi-item-for-ui1 
Fix move and delete WI item for UI1
 2023-06-28 00:30:29 +02:00
jojorne
edcd065980 Fix move and delete wi item for UI1 2023-06-27 17:48:30 -03:00
henk717
df70f92c9e Merge pull request #378 from jojorne/jojorne-patch-fix-wi-for-ui1 
Fix WI for UI1
 2023-06-27 14:04:20 +02:00
henk717
b5bdb1d380 Merge pull request #377 from ebolam/Model_Plugins 
Fix for model backends that use toggles always returning true
 2023-06-26 01:54:25 +02:00
jojorne
f8962d0636 Fix WI for UI1 2023-06-22 18:04:43 -03:00
one-some
e62e3560bf Merge pull request #19 from henk7171/accelerate-offloading 
Remove wrong usegpu behavior
 2023-06-22 15:05:03 -05:00
Henk
1da4580e8b Remove wrong usegpu behavior 2023-06-22 07:07:02 +02:00