Gnome Ann
47a58a36b8
Add disk cache slider
2022-06-19 22:53:30 -04:00
henk717
efed44ac8d
Merge pull request #155 from VE-FORBRYDERNE/accelerate
...
Initial support for Accelerate
2022-06-20 01:08:54 +02:00
Gnome Ann
4dd59e0a9d
Correct the type hint for lazy_load_callback
2022-06-19 17:17:41 -04:00
Gnome Ann
21de36c4b0
Lazy loader now moves all non-layer weights to primary device
2022-06-19 16:44:23 -04:00
Gnome Ann
26c319519e
Lazy loader now attempts to pin layers if accelerate is enabled
2022-06-19 16:35:23 -04:00
Gnome Ann
042cf3e560
Automatically support soft prompts for all transformers models
2022-06-19 13:11:58 -04:00
Gnome Ann
cc56718a7e
Fix lazy loader putting too many layers on CPU
2022-06-19 00:29:35 -04:00
Gnome Ann
1380eb0bb0
Disable lazy loader when using GPT-2
2022-06-18 23:54:11 -04:00
Gnome Ann
f9732eb143
Always enable breakmodel if accelerate is available
2022-06-18 23:46:09 -04:00
Gnome Ann
8b4efc5d0a
Use accelerate.dispatch_model() instead of breakmodel if possible
2022-06-18 23:41:36 -04:00
Gnome Ann
f7ffdd7b6b
Add more model querying utilities
2022-06-18 18:16:56 -04:00
Gnome Ann
e143963161
Merge branch 'united' into accelerate
2022-06-18 13:47:38 -04:00
henk717
b209cf9868
NS mode as default
...
Experimental change that makes NS the default, more and more models seem to be requiring this as megatron based models are getting traction, neither does this seem to break the original models (with the exception of a user not being able to use </s> in generated outputs, the extremely rare case someone would be effected by this they can manually switch the mode by editing their settings file).
If this breaks nothing ns will remain the default, however the n mode should remain a choice for those who need it. In case it does get reversed I have also added the bloom model type to the ns list since its models require this.
2022-06-18 19:46:16 +02:00
henk717
23aae24f8e
Merge pull request #154 from VE-FORBRYDERNE/united-merge
...
Merge main into united
2022-06-18 19:42:26 +02:00
Gnome Ann
0eedc541c8
Merge branch 'main' into united-merge
2022-06-18 13:39:23 -04:00
henk717
a10446f258
Merge pull request #123 from VE-FORBRYDERNE/tokenizer
...
Fix OPT tokenization problems
2022-06-18 11:38:14 +02:00
Gnome Ann
5e71f7fe97
Use slow tokenizer if fast tokenizer is not available
2022-06-17 21:08:37 -04:00
Gnome Ann
f71bae254a
Fix OPT tokenization problems
2022-06-17 13:29:42 -04:00
henk717
22091bc7e2
Merge pull request #153 from ebolam/united
...
Fix for flaskwebgui
2022-06-17 14:19:22 +02:00
ebolam
2964175d8b
Fix for flaskwebgui
2022-06-17 08:17:22 -04:00
Henk
f112fc3493
Initial flaskwebgui support
2022-06-17 13:49:03 +02:00
Gnome Ann
8bdf17f598
Lazy loader can now use accelerate's init_empty_weights()
2022-06-16 18:56:16 -04:00
Gnome Ann
5253cdcb36
Lazy loader no longer requires map file except when loading to TPU
2022-06-16 18:45:11 -04:00
henk717
b0a01962ab
Merge branch 'KoboldAI:main' into united
2022-06-16 20:42:24 +02:00
Henk
49a3cf132e
Require accelerate
...
Transformers 4.20 now requires accelerate to be installed for some of the features we use in KoboldAI. This is now a required dependency for updated users.
2022-06-16 20:38:58 +02:00
henk717
50d2172aaf
Merge branch 'KoboldAI:main' into united
2022-06-16 19:55:39 +02:00
Henk
3504581015
Transformers dependency bump
...
Makes transformers 4.20 mandatory in the dependency lists, not because the old versions are no longer supported but because it contains fixes that benefit our users and this makes it easier for them to update to it. If you stick to an older version the OPT and XGLM workarounds we have in place will remain functional, but you miss on the enhancements newer transformers versions bring.
2022-06-16 19:52:04 +02:00
henk717
83b1fac7a4
Merge pull request #152 from VE-FORBRYDERNE/oom-passthrough
...
Don't use fallback loading if we run out of memory during model loading
2022-06-15 21:30:24 +02:00
Gnome Ann
96d3d397ab
Don't use fallback loading if we run out of memory during loading
2022-06-15 14:35:32 -04:00
henk717
3974e0a90c
Remove broken chatbot models
2022-06-15 19:14:36 +02:00
Henk
fb2b6f1026
Model Path Hardening
2022-06-15 13:29:10 +02:00
Henk
24d34647e0
Block navigation on all remote modes
2022-06-15 12:32:19 +02:00
Henk
f39e24d87f
Localtunnel fix, small polish
2022-06-15 12:22:00 +02:00
Henk
f49cf919bf
Merge branch 'overhaul' into united
2022-06-15 02:09:30 +02:00
henk717
de07b1749f
Merge pull request #150 from ebolam/Web-UI
...
Delete model fixes and model info ui cleanup
2022-06-15 01:50:39 +02:00
ebolam
095cd2a19d
Prevent on server side deletion of folders other than in models in the executing directory
...
Removed delete icon for model folders outside the models directory
2022-06-14 19:39:11 -04:00
ebolam
f444ad851f
Potential catch for if somehow a user sends a delete model with a .. in it.
2022-06-14 19:30:01 -04:00
ebolam
899f191b51
Fix for model information not being centered and having the wrong background
2022-06-14 19:26:02 -04:00
henk717
9add3b0761
Merge pull request #149 from ebolam/Web-UI
...
--remote jailed to model directory and delete of models from UI
2022-06-15 01:14:06 +02:00
ebolam
462206fa86
added --remote not allowing navigation outside of the model folder for custom models.
...
added a delete custom models option (will not delete models outside of the models directory, nor will it delete non-model directories)
2022-06-14 19:11:30 -04:00
Henk
01b3c9932a
1.18.1 version bump
2022-06-15 00:58:49 +02:00
henk717
f3eb7cba5c
Merge pull request #148 from VE-FORBRYDERNE/overhaul-merge
...
Merge united into overhaul
2022-06-15 00:56:14 +02:00
Henk
661a2d2727
1.18.1 version bump
...
Since 1.18 Kobold had a few smaller features added, specifically the ability to re-order sampling options and a new sampler. Since it is a smaller addition a minor version bump was chosen since there are no breaking changes.
2022-06-15 00:55:12 +02:00
Gnome Ann
32a8f03f13
Merge branch 'united' into overhaul-merge
2022-06-14 18:53:04 -04:00
henk717
1ae410a7b0
Sampler Settings
...
This adds VE's changes to let users define the order of the samplers.
2022-06-15 00:49:09 +02:00
Gnome Ann
107966fef8
Merge branch 'united' into overhaul-merge
2022-06-14 18:47:38 -04:00
Gnome Ann
a61e06f876
Merge commit '4c7d6f42d99d557130511f5d185249b34f9db5a1' into overhaul-merge
2022-06-14 18:43:25 -04:00
Gnome Ann
979640bd2f
Merge commit '2d3db7b4ba388f566aaec88a0e76678fe4fade8d' into overhaul-merge
2022-06-14 18:42:14 -04:00
Gnome Ann
130d530e7c
Merge commit 'a273a5ebc49935bfafdcf1aaf4b98c9bf4bc33b1' into overhaul-merge
2022-06-14 18:38:25 -04:00
Gnome Ann
18218a99bc
Merge commit '8a38b258f497281af06fcb0c2559f382b419b938' into overhaul-merge
2022-06-14 18:36:37 -04:00
