henk717
fee79928c8
Merge pull request #404 from one-some/united
...
Delete basic 4bit
2023-07-18 01:19:14 +02:00
somebody
1637760fa1
Delete basic 4bit
...
And add code to handle dangling __pycache__s
2023-07-17 18:16:03 -05:00
henk717
5c3a8e295a
Merge pull request #402 from one-some/united
...
Patches: Make lazyload work with quantization
2023-07-17 23:53:14 +02:00
somebody
23b95343bd
Patches: Make lazyload work on quantized
...
i wanna watch youtube while my model is loading without locking up my
system >:(
2023-07-17 16:47:31 -05:00
henk717
e8d84bb787
Merge pull request #400 from ebolam/Model_Plugins
...
missed the elif
2023-07-17 15:16:34 +02:00
ebolam
eafb699bbf
missed the elif
2023-07-17 09:12:45 -04:00
henk717
a3b0c6dd60
Merge pull request #399 from ebolam/Model_Plugins
...
Update to the upload_file function
2023-07-17 15:11:40 +02:00
ebolam
bfb26ab55d
Ban uploading to the modeling directory
2023-07-17 09:05:22 -04:00
ebolam
52e061d0f9
Fix for potential jailbreak
2023-07-17 08:55:23 -04:00
henk717
f7561044c6
Merge pull request #398 from Alephrin/patch-1
...
Speeds up bnb 4bit with a custom BitsAndBytesConfig
2023-07-17 13:22:44 +02:00
Alephrin
145a43a000
Removed extra load_in_4bit.
2023-07-17 04:53:47 -06:00
Alephrin
e9913d657a
Speeds up bnb 4bit with a custom BitsAndBytesConfig
...
With this BitsAndBytesConfig I get about double the speed compared to running without it. (Tested on llama 13B with a 3090)
2023-07-17 04:43:43 -06:00
Henk
6d7e9e6771
Post4 BnB for Linux
2023-07-16 02:13:42 +02:00
Henk
8bef2e5fef
Fixes 16-bit if BnB is not installed
2023-07-16 02:02:58 +02:00
henk717
fac006125e
Merge pull request #397 from ebolam/Model_Plugins
...
Fixes for model backend UI
2023-07-15 23:58:24 +02:00
ebolam
869bcadd03
Fix for toggles showing as check boxes in model loading
...
Fix for resubmit_model_info loosing selected model backend
2023-07-15 15:48:31 -04:00
ebolam
aaec472f73
Fix for toggles not showing up right in model backend settings
2023-07-15 15:20:18 -04:00
ebolam
bc337bf090
Merge branch 'henk717:united' into Model_Plugins
2023-07-15 15:02:00 -04:00
Henk
0622810bc4
Better way of doing the if statement
2023-07-15 20:00:29 +02:00
Henk
521d6ff227
Clean-up accidental uploads
2023-07-15 19:55:37 +02:00
Henk
23a104a4fe
Only show 4-bit toggle on valid model
2023-07-15 19:42:26 +02:00
Henk
71b6e8d6d4
Fix accidental parameters overwrite
2023-07-15 19:35:40 +02:00
Henk
c43d60772b
BnB dependency check
2023-07-15 18:56:13 +02:00
Henk
160effb9ea
Add 4-bit BnB toggle
2023-07-15 18:20:10 +02:00
ebolam
13db6821fe
Fix for default inference model is_valid and requested_parameters having vram as a required parameter.
2023-07-15 11:11:29 -04:00
Henk
c833ab23e4
Add scipy
2023-07-15 14:57:42 +02:00
Henk
2c50d5d092
Don't ruin breakmodel
2023-07-15 14:14:06 +02:00
Henk
003105ae8b
Bump BnB
2023-07-15 04:26:58 +02:00
Henk
1f045110a4
Basic 4-bit backend
2023-07-15 02:49:31 +02:00
henk717
6dd73287c9
Merge pull request #391 from one-some/basic-hf-backend
...
Basic HF backend
2023-07-15 02:20:38 +02:00
onesome
7e2e75070b
Fix prioritization in load from dir
...
In the future the module/backend list should probably contain objects
that would make this whole deal a lot less hackier.
2023-07-14 18:47:52 -05:00
onesome
afa8766ea6
Add is_valid
2023-07-14 18:01:18 -05:00
somebody
8549c7c896
Basic backend module prioritization
...
not secure; we're loading these modules so they can obviously execute
code that manipulates the prioritization
2023-07-12 19:03:49 -05:00
somebody
f67cb7fa05
Make basic hf independant from hf
2023-07-12 18:36:30 -05:00
somebody
d17ce8461d
Use device_map="auto"
2023-07-12 17:27:48 -05:00
somebody
60473d4c23
Fix and add some documentation to basic hf backend
2023-07-12 17:16:05 -05:00
Henk
1fed1b0524
New dependencies
2023-07-12 22:00:17 +02:00
onesome
8077d6c3f9
Self-contained sampler patch (Don't merge)
...
Completely untested 3:00 AM code; beware! I will test and add more
documentation tomorrow.
2023-07-12 03:22:43 -05:00
henk717
922fcdd6ac
Merge pull request #392 from ebolam/Model_Plugins
...
Fix for UI2 model loading not showing progress
2023-07-11 19:02:17 +02:00
ebolam
56e1c5df3a
Fix for UI2 model loading not showing progress
2023-07-10 20:59:16 -04:00
henk717
95913aebb8
Merge pull request #390 from one-some/accelerate-offloading
...
Fix bleeding edge model loads and add lazyload fallback
2023-07-09 01:02:12 +02:00
somebody
20b4b4bcef
Add basic hf backend
2023-07-08 17:12:16 -05:00
somebody
f9c38acea8
Merge branch 'accelerate-offloading' into basic-hf-backend
2023-07-08 17:04:23 -05:00
somebody
3928d86339
Fall back to unpatched HF
2023-07-08 14:36:45 -05:00
somebody
c2ee30af32
Add --panic to raise when loading fails
2023-07-08 14:04:46 -05:00
somebody
fd6f66a98d
Patch _rebuild_from_type_v2 to not try converting LazyTensors to Tensors
2023-07-08 13:57:05 -05:00
henk717
60965b7b0c
Merge pull request #389 from one-some/accelerate-offloading
...
Stub seek_offset for cache sorting in load loop
2023-07-07 22:40:08 +02:00
somebody
802929f5f2
Patch safetensors again
2023-07-07 14:54:40 -05:00
somebody
35f3687667
Merge branch 'united' of https://github.com/henk717/KoboldAI into accelerate-offloading
2023-07-07 14:54:12 -05:00
somebody
cfe1f5b514
Stub seek_offset for cache sorting in load loop
...
The way Safetensors individual weight loading is implemented doesn't
take full advantage of the cache ordering system thing, so this can just
be left at zero for now.
2023-07-07 14:49:46 -05:00
