rixty/KoboldAI-Client
1
1
Fork 0
mirror of https://github.com/KoboldAI/KoboldAI-Client.git synced 2025-06-05 21:59:24 +02:00
Code Issues Projects Releases Wiki Activity
4,252 Commits 1 Branch 5 Tags
b9b3cd3aba4d076cf9c703b5cb29a2ecc4b6e431
Commit Graph

4252 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
somebody
b9b3cd3aba API: Fix /story 2023-07-19 12:02:53 -05:00
somebody
813e210127 Bump tiny API version 
As we're adding a new (though optional) parameter to load endpoint
 2023-07-19 11:52:49 -05:00
somebody
fef42a6273 API: Fix loading 2023-07-19 11:52:39 -05:00
henk717
5f2600d338 Merge pull request #406 from ebolam/Model_Plugins 
Clarified message on what's required for model backend parameters
 2023-07-18 02:42:23 +02:00
ebolam
66192efdb7 Clarified message on what's required for model backend parameters in the command line 2023-07-17 20:30:41 -04:00
Henk
5bbcdc47da 4-bit on Colab 2023-07-18 01:48:01 +02:00
henk717
da9226fba5 Merge pull request #401 from ebolam/Model_Plugins 
Save the 4-bit flag to the model settings.
 2023-07-18 01:19:43 +02:00
henk717
fee79928c8 Merge pull request #404 from one-some/united 
Delete basic 4bit
 2023-07-18 01:19:14 +02:00
somebody
1637760fa1 Delete basic 4bit 
And add code to handle dangling __pycache__s
 2023-07-17 18:16:03 -05:00
henk717
5c3a8e295a Merge pull request #402 from one-some/united 
Patches: Make lazyload work with quantization
 2023-07-17 23:53:14 +02:00
somebody
23b95343bd Patches: Make lazyload work on quantized 
i wanna watch youtube while my model is loading without locking up my
system >:(
 2023-07-17 16:47:31 -05:00
ebolam
4acf9235db Merge branch 'Model_Plugins' of https://github.com/ebolam/KoboldAI into Model_Plugins 2023-07-17 09:52:10 -04:00
ebolam
b9ee6e336a Save the 4-bit flag to the model settings. 2023-07-17 09:50:03 -04:00
ebolam
66377fc09e Save the 4-bit flag to the model settings. 2023-07-17 09:48:01 -04:00
henk717
e8d84bb787 Merge pull request #400 from ebolam/Model_Plugins 
missed the elif
 2023-07-17 15:16:34 +02:00
ebolam
eafb699bbf missed the elif 2023-07-17 09:12:45 -04:00
henk717
a3b0c6dd60 Merge pull request #399 from ebolam/Model_Plugins 
Update to the upload_file function
 2023-07-17 15:11:40 +02:00
ebolam
bfb26ab55d Ban uploading to the modeling directory 2023-07-17 09:05:22 -04:00
ebolam
52e061d0f9 Fix for potential jailbreak 2023-07-17 08:55:23 -04:00
henk717
f7561044c6 Merge pull request #398 from Alephrin/patch-1 
Speeds up bnb 4bit with a custom BitsAndBytesConfig
 2023-07-17 13:22:44 +02:00
Alephrin
145a43a000 Removed extra load_in_4bit. 2023-07-17 04:53:47 -06:00
Alephrin
e9913d657a Speeds up bnb 4bit with a custom BitsAndBytesConfig 
With this BitsAndBytesConfig I get about double the speed compared to running without it. (Tested on llama 13B with a 3090)
 2023-07-17 04:43:43 -06:00
Henk
6d7e9e6771 Post4 BnB for Linux 2023-07-16 02:13:42 +02:00
Henk
8bef2e5fef Fixes 16-bit if BnB is not installed 2023-07-16 02:02:58 +02:00
henk717
fac006125e Merge pull request #397 from ebolam/Model_Plugins 
Fixes for model backend UI
 2023-07-15 23:58:24 +02:00
ebolam
869bcadd03 Fix for toggles showing as check boxes in model loading 
Fix for resubmit_model_info loosing selected model backend
 2023-07-15 15:48:31 -04:00
ebolam
aaec472f73 Fix for toggles not showing up right in model backend settings 2023-07-15 15:20:18 -04:00
ebolam
bc337bf090 Merge branch 'henk717:united' into Model_Plugins 2023-07-15 15:02:00 -04:00
Henk
0622810bc4 Better way of doing the if statement 2023-07-15 20:00:29 +02:00
Henk
521d6ff227 Clean-up accidental uploads 2023-07-15 19:55:37 +02:00
Henk
23a104a4fe Only show 4-bit toggle on valid model 2023-07-15 19:42:26 +02:00
Henk
71b6e8d6d4 Fix accidental parameters overwrite 2023-07-15 19:35:40 +02:00
Henk
c43d60772b BnB dependency check 2023-07-15 18:56:13 +02:00
Henk
160effb9ea Add 4-bit BnB toggle 2023-07-15 18:20:10 +02:00
ebolam
13db6821fe Fix for default inference model is_valid and requested_parameters having vram as a required parameter. 2023-07-15 11:11:29 -04:00
Henk
c833ab23e4 Add scipy 2023-07-15 14:57:42 +02:00
Henk
2c50d5d092 Don't ruin breakmodel 2023-07-15 14:14:06 +02:00
Henk
003105ae8b Bump BnB 2023-07-15 04:26:58 +02:00
Henk
1f045110a4 Basic 4-bit backend 2023-07-15 02:49:31 +02:00
henk717
6dd73287c9 Merge pull request #391 from one-some/basic-hf-backend 
Basic HF backend
 2023-07-15 02:20:38 +02:00
onesome
7e2e75070b Fix prioritization in load from dir 
In the future the module/backend list should probably contain objects
that would make this whole deal a lot less hackier.
 2023-07-14 18:47:52 -05:00
onesome
afa8766ea6 Add is_valid 2023-07-14 18:01:18 -05:00
somebody
8549c7c896 Basic backend module prioritization 
not secure; we're loading these modules so they can obviously execute
code that manipulates the prioritization
 2023-07-12 19:03:49 -05:00
somebody
f67cb7fa05 Make basic hf independant from hf 2023-07-12 18:36:30 -05:00
somebody
d17ce8461d Use device_map="auto" 2023-07-12 17:27:48 -05:00
somebody
60473d4c23 Fix and add some documentation to basic hf backend 2023-07-12 17:16:05 -05:00
Henk
1fed1b0524 New dependencies 2023-07-12 22:00:17 +02:00
onesome
8077d6c3f9 Self-contained sampler patch (Don't merge) 
Completely untested 3:00 AM code; beware! I will test and add more
documentation tomorrow.
 2023-07-12 03:22:43 -05:00
henk717
922fcdd6ac Merge pull request #392 from ebolam/Model_Plugins 
Fix for UI2 model loading not showing progress
 2023-07-11 19:02:17 +02:00
ebolam
56e1c5df3a Fix for UI2 model loading not showing progress 2023-07-10 20:59:16 -04:00