6b172306f6
move_model_to_devices no longer crashes if you don't have accelerate
2022-06-21 13:15:46 -04:00
ff69e9fbfe
Put layers_module_names, module_names and named_buffers in utils.py
2022-06-20 17:17:42 -04:00
1620ac4148
Lazy loader needs to cache named buffers of layers in the disk cache
2022-06-20 17:08:52 -04:00
ab5ab79003
Set primary device to CPU if in CPU-only mode
2022-06-20 16:25:01 -04:00
bd7d7b41a1
Don't enable accelerate if no layers are in disk cache or GPUs
2022-06-20 16:21:44 -04:00
90fd8b1845
Disk cache support in CPU-only mode
2022-06-20 16:06:09 -04:00
af07d7a15f
Disk cache support for computers with at least one GPU
2022-06-20 14:49:54 -04:00
47a58a36b8
Add disk cache slider
2022-06-19 22:53:30 -04:00
4dd59e0a9d
Correct the type hint for lazy_load_callback
2022-06-19 17:17:41 -04:00
21de36c4b0
Lazy loader now moves all non-layer weights to primary device
2022-06-19 16:44:23 -04:00
26c319519e
Lazy loader now attempts to pin layers if accelerate is enabled
2022-06-19 16:35:23 -04:00
042cf3e560
Automatically support soft prompts for all transformers models
2022-06-19 13:11:58 -04:00
cc56718a7e
Fix lazy loader putting too many layers on CPU
2022-06-19 00:29:35 -04:00
1380eb0bb0
Disable lazy loader when using GPT-2
2022-06-18 23:54:11 -04:00
f9732eb143
Always enable breakmodel if accelerate is available
2022-06-18 23:46:09 -04:00
8b4efc5d0a
Use `accelerate.dispatch_model()` instead of breakmodel if possible
2022-06-18 23:41:36 -04:00
f7ffdd7b6b
Add more model querying utilities
2022-06-18 18:16:56 -04:00
e143963161
Merge branch 'united' into accelerate
2022-06-18 13:47:38 -04:00
b209cf9868
NS mode as default
...
Experimental change that makes NS the default, more and more models seem to be requiring this as megatron based models are getting traction, neither does this seem to break the original models (with the exception of a user not being able to use </s> in generated outputs, the extremely rare case someone would be effected by this they can manually switch the mode by editing their settings file).
If this breaks nothing ns will remain the default, however the n mode should remain a choice for those who need it. In case it does get reversed I have also added the bloom model type to the ns list since its models require this.
2022-06-18 19:46:16 +02:00
0eedc541c8
Merge branch 'main' into united-merge
2022-06-18 13:39:23 -04:00
5e71f7fe97
Use slow tokenizer if fast tokenizer is not available
2022-06-17 21:08:37 -04:00
f71bae254a
Fix OPT tokenization problems
2022-06-17 13:29:42 -04:00
2964175d8b
Fix for flaskwebgui
2022-06-17 08:17:22 -04:00
f112fc3493
Initial flaskwebgui support
2022-06-17 13:49:03 +02:00
8bdf17f598
Lazy loader can now use accelerate's `init_empty_weights()`
2022-06-16 18:56:16 -04:00
5253cdcb36
Lazy loader no longer requires map file except when loading to TPU
2022-06-16 18:45:11 -04:00
96d3d397ab
Don't use fallback loading if we run out of memory during loading
2022-06-15 14:35:32 -04:00
fb2b6f1026
Model Path Hardening
2022-06-15 13:29:10 +02:00
24d34647e0
Block navigation on all remote modes
2022-06-15 12:32:19 +02:00
f39e24d87f
Localtunnel fix, small polish
2022-06-15 12:22:00 +02:00
de07b1749f
Merge pull request #150 from ebolam/Web-UI
...
Delete model fixes and model info ui cleanup
2022-06-15 01:50:39 +02:00
095cd2a19d
Prevent on server side deletion of folders other than in models in the executing directory
...
Removed delete icon for model folders outside the models directory
2022-06-14 19:39:11 -04:00
f444ad851f
Potential catch for if somehow a user sends a delete model with a .. in it.
2022-06-14 19:30:01 -04:00
9add3b0761
Merge pull request #149 from ebolam/Web-UI
...
--remote jailed to model directory and delete of models from UI
2022-06-15 01:14:06 +02:00
462206fa86
added --remote not allowing navigation outside of the model folder for custom models.
...
added a delete custom models option (will not delete models outside of the models directory, nor will it delete non-model directories)
2022-06-14 19:11:30 -04:00
01b3c9932a
1.18.1 version bump
2022-06-15 00:58:49 +02:00
661a2d2727
1.18.1 version bump
...
Since 1.18 Kobold had a few smaller features added, specifically the ability to re-order sampling options and a new sampler. Since it is a smaller addition a minor version bump was chosen since there are no breaking changes.
2022-06-15 00:55:12 +02:00
107966fef8
Merge branch 'united' into overhaul-merge
2022-06-14 18:47:38 -04:00
a61e06f876
Merge commit '4c7d6f42d99d557130511f5d185249b34f9db5a1' into overhaul-merge
2022-06-14 18:43:25 -04:00
979640bd2f
Merge commit '2d3db7b4ba388f566aaec88a0e76678fe4fade8d' into overhaul-merge
2022-06-14 18:42:14 -04:00
130d530e7c
Merge commit 'a273a5ebc49935bfafdcf1aaf4b98c9bf4bc33b1' into overhaul-merge
2022-06-14 18:38:25 -04:00
18218a99bc
Merge commit '8a38b258f497281af06fcb0c2559f382b419b938' into overhaul-merge
2022-06-14 18:36:37 -04:00
6231106f95
Add Samplers menu
2022-06-13 20:18:09 -04:00
4c7d6f42d9
Add `sampler_order` to settings file
2022-06-13 19:14:38 -04:00
2d3db7b4ba
Implement support for sampler order in the backend code
2022-06-13 19:12:23 -04:00
11ed55f34a
Added custom text box for loading models from specific path, or loading other models from hugging face.
2022-06-13 13:48:45 -04:00
66c0dda485
Hide (Broken) Chatbot Models
...
Removing this option because they are currently unavailable. People who still have them can load them trough the load from file option. Once they have been retrained and reuploaded I will add the menu back.
2022-06-11 22:54:51 +02:00
5c81374a48
Top A for GooseAi
2022-06-11 22:04:37 +02:00
fdb2a7fa4c
Top-A sampling
2022-06-10 22:28:20 -04:00
cfd1147d5a
Bug fix for loading model after loading a model duplicating the settings menu until the website is refreshed
...
Fixed escaping warnings
Added back/redo unit test
2022-06-10 14:47:52 -04:00
ed428f2e73
Merge branch 'Web-UI' of https://github.com/ebolam/KoboldAI into Web-UI
2022-06-10 09:12:18 -04:00
4a920724d9
fix for folder paths on linux
2022-06-10 09:12:04 -04:00
6200908582
Merge pull request #10 from henk717/overhaul
...
Overhaul
2022-06-10 08:40:15 -04:00
13f17d3eca
Changed unit tests so that they run with a simple pytest command
2022-06-10 08:39:15 -04:00
ce582f188f
Merge branch 'united' into overhaul-merge
2022-06-09 23:48:28 -04:00
fe619d4677
Update list of versions with broken OPT again
...
They released another version of transformers that still doesn't have
the OPT patch so I decided it would be safer to just mark all 4.19
transformers versions as needing the OPT patch.
2022-06-09 17:42:46 -04:00
663dee784d
Unit Tests using pytest and Minor modifications to allow unit testing
2022-06-09 13:16:32 -04:00
606c276f9d
Potential fix for tokenizer using a fallback
2022-06-09 09:01:40 -04:00
db9a94ca2a
Added GPU name to the UI when using break models.
...
Added total layers to the UI
Added favicon
2022-06-09 08:42:35 -04:00
c565978fff
Fix for multi-gpu not showing appropriately
...
Slight visual improvement for custom model load breadcrumbs
2022-06-08 19:39:04 -04:00
4548dcf1b0
Fix for --model with custom paths
2022-06-08 18:53:56 -04:00
001439be45
Merge pull request #9 from henk717/overhaul
...
Overhaul
2022-06-08 18:44:21 -04:00
622a3fc8db
Fix for model loading by moving monkey patching functions into a run-once function
...
Added folder navigation to custom model loading (Needs prittying)
2022-06-08 18:42:44 -04:00
1a46d97ad5
Send correct settings after load
2022-06-08 13:26:30 +02:00
461cd04932
Fix Essential Code + selectfolder fix
...
As part of the restructuring essential code was removed that handled the --path parameter correctly. This has now been restored. Selectfolder was also updated to use its NeoCustom counterpart instead of specifying a model so that the underlying code that corrects model names is being hit again.
2022-06-08 11:30:00 +02:00
190869f0d3
Fix for selectfolder model to force old style folder select on startup.
2022-06-07 20:24:31 -04:00
88f5ed7b3c
--model selectfolder
2022-06-07 21:32:58 +02:00
66ba165b4c
--noaimenu as seperate parameter
2022-06-07 20:44:14 +02:00
6fd2496d94
Fix for green opening text showing OAI and/or OAI/GooseAI model name rather than the appropriate name.
2022-06-07 13:47:10 -04:00
1df88e1696
TPU fix Attempt
2022-06-07 09:05:51 -04:00
bf4af94abb
Hopefully a fix for InferKit
2022-06-07 08:22:10 -04:00
afb894f5a0
TPU Fix
2022-06-06 21:47:15 -04:00
1b35b55d86
Fix TPU
2022-06-06 21:39:17 -04:00
ae1aed0916
TPU Fix
2022-06-06 21:37:35 -04:00
df76bc4b41
Fix for Colab
2022-06-06 21:29:14 -04:00
edbf36a632
Web UI functional for GooseAI (and presumably OpenAI).
...
Fix for Breakmodel layer info saving
2022-06-06 19:21:10 -04:00
d9480ec439
Fix for lazy loading
2022-06-06 14:27:47 -04:00
60b70bdf8a
Fix
2022-06-06 14:02:17 -04:00
dd07b10b73
Fix for model loading on web ui and removing AI menu when using remote/colab methods
2022-06-06 13:57:19 -04:00
c984f4412d
Fix for web based model loading
2022-06-06 12:49:40 -04:00
1e139594a9
Merge commit 'refs/pull/7/head' of https://github.com/ebolam/KoboldAI into HEAD
2022-06-06 09:49:46 -04:00
e5dcf91a08
Defaults Support
...
This adds support for loading settings from the defaults folder, settings are loaded in the following order and overwritten if needed by the higher number.
1. The model config file.
2. The defaults folder.
3. The users defined settings file.
With this support we can begin to ship better defaults for models we do not manage. Our community tuners have been most helpful at adding good defaults to their configuration files, but for other models such as the base models this gives us the flexibility to define better settings for each model without messing with a users desired settings if they already exist.
2022-06-01 10:34:16 +02:00
707316de31
Kaggle TPU support
2022-05-31 12:20:16 -04:00
1a1f2f6428
30B ram requirements
2022-05-31 13:17:06 +02:00
69da5b7bc2
Update list of transformers versions that have broken OPT
2022-05-28 23:44:19 -04:00
4b65ce9c76
1.18 version bump
2022-05-28 19:39:05 +02:00
b30370bf4b
2048 maxtoken default
...
Almost everyone prefers 2048 max tokens because of the superior coherency. It should only be lower due to ram limits, but the menu already shows the optimal ram for 2048. Negatively effected users can turn it down themselves, for everyone else especially on rented machines or colab 2048 is a better default.
2022-05-27 01:23:48 +02:00
c692987e40
Fix an error that occurs when loading GPT-2 models
...
I forgot that this new_rebuild_tensor function's first argument's type
is different when loading GPT-2 models.
2022-05-20 14:54:49 -04:00
6ae7b48b69
Adding Nerys model 13B
2022-05-18 13:50:57 +02:00
f0df3de610
Adding Nerys model 2.7B
2022-05-16 09:50:45 +02:00
d5ab3ef5b1
Fix `no attribute get_checkpoint_shard_files`
2022-05-14 11:49:04 -04:00
1476e76cfc
Copy fp16 model files instead of resaving them
2022-05-14 00:45:43 -04:00
0c5ca5261e
Loading a sharded model will now display only one progress bar
2022-05-13 23:32:16 -04:00
f9f1a5f3a9
Make sure tqdm progress bars display properly in Colab
2022-05-13 17:37:45 -04:00
91d3672446
Proper progress bar for aria2 downloads
2022-05-13 17:00:10 -04:00
7ea0c49c1a
Merge pull request #128 from VE-FORBRYDERNE/opt
...
OPT breakmodel and TPU support
2022-05-13 18:07:02 +02:00
1200173386
Custom badwords for OPT
...
Generated using:
```
import transformers
tokenizer = transformers.AutoTokenizer.from_pretrained("facebook/opt-350m", fast=False)
badwordsids_opt = [[v] for k, v in tokenizer.vocab.items() if any(c in k for c in "<>[]")]
```
2022-05-13 10:45:28 -04:00
d5fa782483
NS Mode (comment fix)
2022-05-13 10:53:19 +02:00
8376f12e21
Add NS mode
...
OPT supports newlines, but it also needs some of the behavior we use in S mode. NS mode is a more limited version of S mode that still handles the </s> token, but instead of replacing it with a new line we replace it empty and newlines are not converted.
In future if your Fairseq style model has newline support use NS mode, while if it needs artifically inserted newlines use S mode. This also means that people finetuning fairseq models to include newlines might benefit from testing their models on ns mode.
2022-05-13 10:44:12 +02:00
55079f672a
Fix typo in soft prompt patching code
2022-05-13 01:51:55 -04:00
Gnome Ann
henk717
ebolam
Julius ter Pelkwijk