05a4695ad2
Merge pull request #198 from db0/soft_prompts_list
...
Adds /config/soft_prompts_list API endpoint
2022-09-01 16:35:08 +02:00
b0aa615ef5
Merge pull request #196 from db0/mult_gen_api_cluster
...
fix for multiple gens breaking API/CLUSTER
2022-09-01 16:34:52 +02:00
c1bf91f86c
Adds /config/soft_prompts_list API endpoint
2022-08-31 23:45:26 +02:00
339225e400
fix for multiple gens breaking API/CLUSTER
2022-08-31 22:58:58 +02:00
8626debeff
Fix for cluster key saving
2022-08-31 15:46:08 -04:00
b07a649e3e
Fix for API key not being saved
2022-08-31 13:17:30 -04:00
417cfe20bf
Fix for saving key in cluster mode
2022-08-31 11:50:27 -04:00
24ac6f3db8
First working CLUSTER ui. Might need change when multiple models selected.
2022-08-31 10:46:16 -04:00
569f4cbce4
Merge branch 'united' of https://github.com/ebolam/KoboldAI into united
2022-08-31 09:34:24 -04:00
1031b70731
Starts of adding cluster to UI
2022-08-31 09:34:14 -04:00
496ef1472d
updated
2022-08-30 21:35:17 +02:00
42e04afc83
init
2022-08-30 21:29:55 +02:00
181c93424c
Fix for KoboldAI API as a model option
2022-08-30 15:10:11 -04:00
8292f17ab0
Don't allow changing model during generation
2022-08-29 13:23:19 -04:00
807ddf6f26
Add PUT /model endpoint
2022-08-28 15:53:15 -04:00
cbab98cc23
Merge branch 'united' into mkultra
2022-08-24 15:06:02 -04:00
6faa27ef87
Merge pull request #187 from VE-FORBRYDERNE/offline
...
Fix the model selection GUI when there is no internet connection
2022-08-24 20:13:02 +02:00
51135e192b
Merge branch 'united' into mkultra
2022-08-23 21:29:29 -04:00
cbfe456409
Repetition penalty is now added to sampler list when loading from settings files
2022-08-23 15:30:07 -04:00
6ffaf43548
Repetition penalty is now sampler #6 in the sampler order
2022-08-23 15:10:21 -04:00
74922966bd
Merge branch 'avril' into rep-pen-order
2022-08-23 14:47:29 -04:00
1e9f0e68a0
Merge branch 'united' into mkultra
2022-08-22 21:25:42 -04:00
09750acfa0
prompt_tuner.py now shows layer configuration
2022-08-22 20:02:21 -04:00
3d5c83fc23
prompt_tuner.py now uses lazy loader and accelerate
2022-08-22 19:29:20 -04:00
d7ebd2ae20
Dont broadcast token usage
2022-08-22 17:25:33 -05:00
55f45c4912
Fix the model selection GUI when there is no internet connection
2022-08-22 14:45:02 -04:00
137695106d
Fix for gooseai
2022-08-17 18:03:48 -04:00
a19300b3ca
Merge branch 'henk717:united' into united
2022-08-17 09:07:01 -04:00
0032462837
Fix for vars.model getting set on AI selection in the UI rather than when actually loaded
2022-08-13 20:12:35 -04:00
6acccbf7a4
Save null seed to settings
2022-08-14 02:09:53 +02:00
a7fb2c8414
Merge branch 'united' into api
2022-08-12 13:57:50 -04:00
09a709f0dc
Merge branch 'token-usage-textsize' into united
2022-08-12 01:46:33 +02:00
6ac970b1c0
Update author's template effect token usage live
2022-08-11 18:38:29 -05:00
a28faa0cb2
Fix author's note token usage
2022-08-11 18:21:49 -05:00
e879d1c5f3
Hide the warning about `torch.distributed.reduce_op` being deprecated
2022-08-11 18:42:56 -04:00
555ca5fd05
Add token usage indicator
2022-08-11 17:31:12 -05:00
8c7ed92fef
--no_ui for disabling the main GUI and Socket.IO server
2022-08-11 18:21:35 -04:00
ca2c60d423
Fix for --nobreakmodel
2022-08-11 18:12:50 -04:00
bddcd7ab7f
Deeper disable of --nobreakmodel attempt
2022-08-11 17:47:19 -04:00
8fbca2db5a
actionsubmit should not ignore vars.aibusy if vars.standalone is True
2022-08-11 15:27:07 -04:00
45495d8792
Fix for --cpu on command line and MAYBE --nobreakmodel
2022-08-11 15:23:35 -04:00
8b299525fd
sendtoapi now automatically detects tokenizer
2022-08-11 14:57:13 -04:00
d328c2c1de
Add GET /model endpoint
2022-08-11 14:55:38 -04:00
cd7ff2b141
Change behaviour of disable_input/output_formatting
2022-08-11 14:30:14 -04:00
bd703cd36a
Warn about disable_input_formatting and disable_output_formatting
2022-08-11 14:12:07 -04:00
df111b944d
Input formatting is now actually applied when generating with API
2022-08-11 14:11:17 -04:00
c0c9d62cd7
Latest API version is now automatically calculated
2022-08-11 14:00:43 -04:00
8482df0d8d
Remove HTTP error 422 from specifications where it is never thrown
2022-08-11 13:56:00 -04:00
78cc5da87f
`calcsubmitbudget` no longer adds `_koboldai_header` to API requests
2022-08-11 13:37:09 -04:00
43e318bdc2
Safer method of determining request URL in sendtoapi
2022-08-11 13:29:47 -04:00
64664dc61e
Fix for the AI menu to respect the --cpu command line flag
2022-08-11 10:40:32 -04:00
1527db894e
Fix specification of GET /story/nums/{num}
2022-08-10 21:09:29 -04:00
6853625570
Allow KoboldAI to use its own API to generate text
2022-08-10 21:00:17 -04:00
4eff7bf3ba
/api now redirects to /api/latest
2022-08-10 18:22:46 -04:00
d2c06182f2
Remove annotation from api_version
2022-08-10 18:05:04 -04:00
2af57adff3
API v1.1.0
2022-08-10 14:48:01 -04:00
becda8b842
Error 405 now sets Allow header
2022-08-09 22:32:24 -04:00
5352c14c59
Fix typo in /config/soft_prompt documentation
2022-08-08 19:20:48 -04:00
c04e3c5666
Fix /docs/ redirects
2022-08-08 18:21:46 -04:00
55c4acad8f
Disable probability viewer and output streaming when using API
2022-08-08 18:16:08 -04:00
82ae749396
Merge branch 'united' into api
2022-08-08 18:14:50 -04:00
aa01d1419d
Add /story/end/delete and /story endpoints
2022-08-08 18:08:55 -04:00
1f629ee254
Add more endpoints
2022-08-08 17:51:40 -04:00
a93087aecd
Fix `api_format_docstring`
2022-08-08 14:21:50 -04:00
ddda981436
Improve /generate description
2022-08-08 14:19:43 -04:00
dc0fa9bff1
Add redirects to /api/v1/docs/
2022-08-08 14:16:38 -04:00
ce064168e3
Additional validation for soft_prompt in API
2022-08-08 13:52:07 -04:00
de1e8f266a
ValidationErrorSchema now has minItems 1 for its arrays
2022-08-08 13:22:18 -04:00
596f619999
Unknown values in API input are now ignored instead of causing error
2022-08-08 13:17:53 -04:00
3b56859c12
vars.disable_input_formatting and vars.disable_output_formatting fix
2022-08-08 13:04:46 -04:00
34c9535667
Upload basic API with /generate POST endpoint
2022-08-08 02:27:48 -04:00
77e2a7972c
Fix incorrect max tokens
2022-08-06 17:28:55 +02:00
fe00581b83
Merge branch 'main' into united
2022-08-06 17:10:09 +02:00
c71fd0cc3f
OPT Nerys V2 6B
2022-08-06 17:04:22 +02:00
610257b36e
Output Streaming on by Default
2022-08-06 16:47:04 +02:00
8bcf4187ac
Merge pull request #178 from one-some/token-prob
...
Add token probability visualizer
2022-08-05 14:27:46 +02:00
f6d046fe1b
Add token probability visualizer
2022-08-04 13:49:37 -05:00
71e119f0b7
Fix for secondary model loads leaking settings into secondary model's settings file.
2022-08-02 19:45:36 -04:00
050e195420
Merge pull request #173 from one-some/token-streaming
...
Add token streaming option
2022-07-30 18:32:51 +02:00
a63f7cfa5a
Merge pull request #174 from ebolam/united
...
Fix for blank model info box when downloading model
2022-07-29 22:15:58 +02:00
f97c10b794
Fix for blank model info box when downloading model
2022-07-28 19:40:27 -04:00
a4d81292f8
Add token streaming option
2022-07-27 22:13:08 -05:00
fe64e480ee
Merge pull request #171 from ebolam/united
...
Add Download Model Status
2022-07-26 00:52:12 +02:00
7721b72184
Merge branch 'KoboldAI:main' into united
2022-07-26 00:42:35 +02:00
4d8a633351
Aetherroom instead of aidg.club
2022-07-26 00:41:51 +02:00
12acb50ee0
Fix for getting "model download status" when downloading config to figure out layer counts
2022-07-25 18:29:14 -04:00
9dc9966433
Added functionality to add any/all args via json
2022-07-23 22:02:03 -06:00
907cf74b13
Added status bar for downloading models
2022-07-22 13:58:20 -04:00
2b53598307
Fixes for file editor ( #170 )
...
Various fixes for the file editor by Ebolam
2022-07-20 00:50:03 +02:00
f58064e72c
Revert "Fix for aidg.club website being taken read-only"
...
This reverts commit 23a031d852
2022-07-19 16:54:32 -04:00
23a031d852
Fix for aidg.club website being taken read-only
2022-07-19 13:40:55 -04:00
68d143b80c
Merge branch 'united' of https://github.com/ebolam/KoboldAI into united
2022-07-15 12:30:18 -04:00
d91ed3141d
Fix for non ascii files in edit mode
2022-07-15 12:30:02 -04:00
e8c39992a1
Merge pull request #166 from ebolam/united
...
Add file browser to soft prompts and user scripts
2022-07-04 19:52:05 +02:00
328c0a38d7
Removed breadcrumbs on file browser before the jail directory
2022-07-03 16:02:55 -04:00
fd44f0ded3
Merge branch 'KoboldAI:main' into united
2022-07-03 15:12:12 +02:00
d041ec0921
Safer defaults and more flexibility
...
There have been a lot of reports from newer users who experience AI breakdown because not all models properly handle 2048 max tokens. 1024 is the only value that all models support and was the original value KoboldAI used. This commit reverts the decision to increase this to 2048, any existing configurations are not effected. Users who wish to increase the max tokens can do so themselves. Most models handle up to 1900 well (The GPT2 models are excluded), for many you can go all the way. (It is currently not yet known why some finetunes cause a decrease in maxtoken support,
In addition this commit contains a request for more consistent slider behavior, allowing the sliders to be changed at 0.01 intervals instead of some sliders being capped to 0.05.
2022-07-03 15:07:54 +02:00
a99518d0a8
Merge branch 'KoboldAI:main' into united
2022-07-02 12:59:53 +02:00
e2f7fed99f
Don't turn gamestarted off
2022-07-02 12:59:14 +02:00
aeed9bd8f7
Fix base fairseq dense models when using accelerate with a GPU
2022-07-01 20:16:39 -04:00
3f8a7ab4bb
Allowing edit in userscripts
2022-06-30 19:41:11 -04:00
813540fe9b
Added folder browser for softprompts and userscripts
2022-06-30 19:13:05 -04:00
97e0df45d7
File Dialog complete
2022-06-30 15:57:27 -04:00
58418c4aa5
Basic file browser with edit and delete functionality
...
Can be shown by going to /popup_test in a second tab.
2022-06-30 09:44:04 -04:00
048bd0ff3b
Add support for setting the RNG seed and full determinism
2022-06-28 13:21:05 -04:00
edd6dd7cd7
Fix for saved breakmodel settings on custom models
...
Fix for unit tests with new disk breakmodel
2022-06-27 10:12:54 -04:00
46678931b2
Better sentence spacing
2022-06-26 20:27:21 +02:00
ebba79fed6
Remove trailing whitespace from submissions
...
(cherry picked from commit b99d1449c9
2022-06-26 14:06:34 -04:00
2a4d37ce60
Clean up whitespace at the end of actions when loading story
...
Specifically, we merge blank actions into the next action and we move
whitespace at the end of non-blank actions to the beginning of the next
action.
(cherry picked from commit 4b16600e49
2022-06-26 14:04:36 -04:00
b99d1449c9
Remove trailing whitespace from submissions
2022-06-26 13:15:55 -04:00
fa97d28cb3
Nerys V2 for United
2022-06-25 14:06:51 +02:00
9e7eb80db4
Nerys V2 part 2
2022-06-25 14:03:19 +02:00
ecc6ee9474
Nerys V2
2022-06-25 13:47:49 +02:00
10e85db89d
Merge pull request #162 from VE-FORBRYDERNE/whitespace-cleanup
...
Story whitespace cleanup
2022-06-25 13:36:03 +02:00
d3fce44095
Merge branch 'main' into united
2022-06-24 18:31:45 +02:00
8be0964427
AIDG Import Fix
2022-06-24 18:29:06 +02:00
4b16600e49
Clean up whitespace at the end of actions when loading story
...
Specifically, we merge blank actions into the next action and we move
whitespace at the end of non-blank actions to the beginning of the next
action.
2022-06-24 12:03:35 -04:00
3da885d408
GPT-NeoX HF model badwords fix
2022-06-23 15:02:43 -04:00
8098f4ec8f
Merge branch 'KoboldAI:main' into united
2022-06-23 17:20:48 +02:00
0eb9f8a879
Account for lnheader in budget calculation
2022-06-22 19:16:24 -04:00
53034ee533
Delete all torch tensors before loading model
2022-06-22 12:07:36 -04:00
922394c68f
Don't blacklist </s> token in "s" newline mode
2022-06-22 11:23:03 -04:00
8c594c6869
Correct the padding token for GPT-NeoX
2022-06-21 19:37:43 -04:00
a7f667c34c
Use NeoX badwords when loading from HF GPT-NeoX model
2022-06-21 19:33:25 -04:00
8593bf339b
Another typo fix
2022-06-21 15:36:25 -04:00
7e0ded6b47
Typo fix
2022-06-21 15:12:55 -04:00
91643be10a
Change soft prompt implementation to a more universal one
2022-06-21 15:03:43 -04:00
0ea4fa9c87
Automatically calculate badwords and pad_token_id
2022-06-21 14:35:52 -04:00
6b172306f6
move_model_to_devices no longer crashes if you don't have accelerate
2022-06-21 13:15:46 -04:00
ff69e9fbfe
Put layers_module_names, module_names and named_buffers in utils.py
2022-06-20 17:17:42 -04:00
1620ac4148
Lazy loader needs to cache named buffers of layers in the disk cache
2022-06-20 17:08:52 -04:00
ab5ab79003
Set primary device to CPU if in CPU-only mode
2022-06-20 16:25:01 -04:00
bd7d7b41a1
Don't enable accelerate if no layers are in disk cache or GPUs
2022-06-20 16:21:44 -04:00
90fd8b1845
Disk cache support in CPU-only mode
2022-06-20 16:06:09 -04:00
af07d7a15f
Disk cache support for computers with at least one GPU
2022-06-20 14:49:54 -04:00
47a58a36b8
Add disk cache slider
2022-06-19 22:53:30 -04:00
4dd59e0a9d
Correct the type hint for lazy_load_callback
2022-06-19 17:17:41 -04:00
21de36c4b0
Lazy loader now moves all non-layer weights to primary device
2022-06-19 16:44:23 -04:00
26c319519e
Lazy loader now attempts to pin layers if accelerate is enabled
2022-06-19 16:35:23 -04:00
042cf3e560
Automatically support soft prompts for all transformers models
2022-06-19 13:11:58 -04:00
cc56718a7e
Fix lazy loader putting too many layers on CPU
2022-06-19 00:29:35 -04:00
1380eb0bb0
Disable lazy loader when using GPT-2
2022-06-18 23:54:11 -04:00
f9732eb143
Always enable breakmodel if accelerate is available
2022-06-18 23:46:09 -04:00
8b4efc5d0a
Use `accelerate.dispatch_model()` instead of breakmodel if possible
2022-06-18 23:41:36 -04:00
f7ffdd7b6b
Add more model querying utilities
2022-06-18 18:16:56 -04:00
e143963161
Merge branch 'united' into accelerate
2022-06-18 13:47:38 -04:00
b209cf9868
NS mode as default
...
Experimental change that makes NS the default, more and more models seem to be requiring this as megatron based models are getting traction, neither does this seem to break the original models (with the exception of a user not being able to use </s> in generated outputs, the extremely rare case someone would be effected by this they can manually switch the mode by editing their settings file).
If this breaks nothing ns will remain the default, however the n mode should remain a choice for those who need it. In case it does get reversed I have also added the bloom model type to the ns list since its models require this.
2022-06-18 19:46:16 +02:00
0eedc541c8
Merge branch 'main' into united-merge
2022-06-18 13:39:23 -04:00
5e71f7fe97
Use slow tokenizer if fast tokenizer is not available
2022-06-17 21:08:37 -04:00
f71bae254a
Fix OPT tokenization problems
2022-06-17 13:29:42 -04:00
henk717
Divided by Zer0
ebolam
vfbd
somebody
scott-ca
Gnome Ann