rixty/KoboldAI-Client
1
1
Fork 0
mirror of https://github.com/KoboldAI/KoboldAI-Client.git synced 2024-12-13 17:21:48 +01:00
Code Issues Projects Releases Wiki Activity
490 Commits 1 Branch 5 Tags 120 MiB
e9ed8602b2
Commit Graph

490 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
henk717
c36bc376c0
Revert typo fix 
Wasn't a typo (Testing on git again since its easier with the colabs)
 2021-12-05 18:36:54 +01:00
henk717
337941e356
Typo fix 
Accidentally had a negative which should have been a positive, this should fix repo downloads.
 2021-12-05 18:34:20 +01:00
henk717
a442a2a67e
Merge pull request #41 from VE-FORBRYDERNE/jax21 
TPU backend improvements
 2021-12-05 18:10:52 +01:00
Gnome Ann
6d8517e224 Fix some minor coding errors 2021-12-05 11:39:59 -05:00
Gnome Ann
1393eac882 Add indentation to WI folder contents 2021-12-05 03:29:13 -05:00
Gnome Ann
d46ef8550b Fix WI sortable handle events not being bound correctly 
This fixes a problem where WI entries/folders are sometimes able to be
dragged into places they shouldn't be. Steps to reproduce:

1. Start a blank story
2. Refresh the browser
3. Open the W Info screen
4. Add a world info folder
5. Add a world info entry into that folder
6. Drag that world info entry
 2021-12-05 03:04:45 -05:00
Gnome Ann
85aa180a90 Put safeguards on dragging and dropping into invalid positions 2021-12-05 02:50:42 -05:00
Gnome Ann
150ce033c9 TPU backend no longer needs to recompile after changing softprompt 2021-12-05 02:49:15 -05:00
Gnome Ann
3e0b1a9e63 Fix scrolling problems with WI entries with long names/comments 2021-12-05 01:39:42 -05:00
Gnome Ann
08992dec7e Use a green horizontal line as the drag-and-drop placeholder 2021-12-05 00:34:44 -05:00
Gnome Ann
b99ac92a52 WI folders and WI drag-and-drop 2021-12-04 23:59:28 -05:00
Gnome Ann
d2d338d314 Improve TPU backend compilation times with numseqs > 1 
A Python `for` loop was replaced with a `jax.lax.scan` call so that JAX
only compiles the `transformer.generate_initial` function one time
instead of `numseqs` times. This is because JAX unrolls Python built-in
loops like `for`. The compilation times should now be about the same as
they were before the upgrade to JAX 0.2.21.
 2021-11-30 19:22:40 -05:00
Gnome Ann
c1e7c1643f Fix unbound axis error in tpu_mtj_backend.py when numseqs > 1 2021-11-30 14:06:46 -05:00
Gnome Ann
3c349e6aaf Modify TPU backend code to support JAX 0.2.21 
The original one supported versions of JAX up to 0.2.12, and possibly also some
earlier versions. This new code supports exclusively JAX 0.2.21 and does not
work with any earlier or later versions of JAX. However, this new code benefits
from not needing to recompile when changing "Amount To Generate" and also from
supporting stopping generation early, which makes an implementation of Dynamic
World Info Scan finally possible.
 2021-11-30 10:13:02 -05:00
henk717
9e3318c696 Update colabkobold.sh 
Bugfix
 2021-11-29 18:42:40 +01:00
henk717
4244b588cb Ngrok requirements 
Adds ngrok to the requirements.txt files
 2021-11-29 18:13:30 +01:00
henk717
fd19e2bfd6 Allow Ngrok 
Adds --ngrok to the Colab script
 2021-11-29 18:12:45 +01:00
henk717
44d8068bab Ngrok Support 
Not recommended for home users due to DDoS risks, but might make Colab tunnels more reliable.
 2021-11-29 18:11:14 +01:00
henk717
eef675ce21
Further Streamlining Dependencies 
Hopefully this will make Kaggle work
 2021-11-29 16:43:45 +01:00
henk717
ff99e4c0e9
Merge branch 'KoboldAI:main' into united 2021-11-29 12:06:03 +01:00
henk717
d4a7ff5ccb Better Repetition Penalty Slider 
Allow users more control since 6B is sensitive
 2021-11-29 08:28:51 +01:00
henk717
f7993d5ef1
Merge pull request #84 from adcar/patch-1 
Fixed a small typo
 2021-11-28 17:58:40 +01:00
Alexander D. Cardosi
8d922d83a9
Fixed a small typo 2021-11-28 11:30:23 -05:00
henk717
6b9d744679
Torch needs to be newer 
Uncapping the version to see the effect (Testing on the git since Colabs load from here)
 2021-11-28 11:35:24 +01:00
henk717
b9c7e33410
Switch to official transformers 
Official transformers is now superior, switching over to allow Colab's to use it.
 2021-11-28 05:13:05 +01:00
henk717
939719214d
Merge pull request #40 from VE-FORBRYDERNE/patch 
Allow bad words filter to ban <|endoftext|> token
 2021-11-27 19:23:01 +01:00
Gnome Ann
9f51c42dd4 Allow bad words filter to ban <|endoftext|> token 
The official transformers bad words filter doesn't allow this by
default. Finetune's version does allow this by default, however.
 2021-11-27 11:42:06 -05:00
henk717
2bc93ba37a
Whitelist 6B in breakmodel 
Now that we properly support it, allow the menu option to use breakmodel
 2021-11-27 10:09:54 +01:00
henk717
b56ee07ffa
Fix for CPU mode 
Recent optimizations caused the CPU version to load in an incompatible format, now we convert it back to the correct format after loading it efficiently first.
 2021-11-27 05:34:29 +01:00
henk717
56c2e619f9 ColabKobold 
A brand new launcher to power the colab's, you can use https://henk.tech/ckds as a short URL which points towards this github
 2021-11-27 03:44:08 +01:00
henk717
3b976c9af7 Updated defaults 
Transformers official by default, no more Git versions
 2021-11-27 03:14:47 +01:00
henk717
6008d4f3a5
Merge pull request #39 from VE-FORBRYDERNE/breakmodel 
Official transformers 6B breakmodel support and more RAM-efficient model loading
 2021-11-27 01:11:48 +01:00
Gnome Ann
e5e2fb088a Remember to actually import GPTJModel 2021-11-26 12:38:52 -05:00
Gnome Ann
871ed65570 Remove an unnecessary **maybe_low_cpu_mem_usage() 2021-11-26 11:42:04 -05:00
Gnome Ann
a93a76eb01 Load model directly in fp16 if using GPU or breakmodel 2021-11-26 10:55:52 -05:00
Gnome Ann
95aff61781 Don't pin CPU layers after running out of pinned memory 2021-11-26 10:31:15 -05:00
Gnome Ann
32e1d4a7a8 Enable low_cpu_mem_usage 2021-11-25 18:09:25 -05:00
Gnome Ann
25c9be5d02 Breakmodel support for GPTJModel 2021-11-25 18:09:16 -05:00
Gnome Ann
f8bcc3411b In breakmodel mode, move layers to GPU as soon as model loads 
Rather than during the first generation.
 2021-11-25 11:44:41 -05:00
henk717
978dc486a5
Merge pull request #38 from VE-FORBRYDERNE/warp 
Move TFS warper code into aiserver.py
 2021-11-24 23:45:28 +01:00
Gnome Ann
cbb6efb656 Move TFS warper code into aiserver.py 2021-11-24 13:36:54 -05:00
henk717
96e1d98b7e
Merge branch 'KoboldAI:main' into united 2021-11-24 08:24:08 +01:00
henk717
36b9161667
Portability Bugfix 
Fix an issue where the launcher does not work if the drive is not C: on some systems.
 2021-11-24 08:23:08 +01:00
henk717
a2c82bbcc8 num_layers fixes 
As requested by VE_FORBRYDERNE (Possibly implemented it on to many places, needs testing but since the other one is already broken I am committing it first so I can more easily test)
 2021-11-24 03:44:11 +01:00
henk717
d7a2424d2d No Half on CPU 
Should fix CPU executions
 2021-11-23 17:14:01 +01:00
henk717
11c64c3fe7
Merge pull request #37 from VE-FORBRYDERNE/patch 
Use model.config.n_layer if model.config.num_layers doesn't exist
 2021-11-23 17:02:51 +01:00
Gnome Ann
be0881a8d0 Use model.config.n_layer if model.config.num_layers doesn't exist 2021-11-23 10:09:24 -05:00
henk717
c0df03fc55
Merge pull request #36 from VE-FORBRYDERNE/sp 
Fix a typo in tpu_mtj_backend.py
 2021-11-23 14:23:10 +01:00
Gnome Ann
691febacd6 Fix a typo in tpu_mtj_backend.py 2021-11-22 12:53:19 -05:00
henk717
d877190258
Merge pull request #35 from VE-FORBRYDERNE/sp 
Softprompt support for the TPU backend
 2021-11-22 00:33:31 +01:00