c1e7c1643f
Fix unbound axis error in tpu_mtj_backend.py when numseqs > 1
2021-11-30 14:06:46 -05:00
3c349e6aaf
Modify TPU backend code to support JAX 0.2.21
...
The original one supported versions of JAX up to 0.2.12, and possibly also some
earlier versions. This new code supports exclusively JAX 0.2.21 and does not
work with any earlier or later versions of JAX. However, this new code benefits
from not needing to recompile when changing "Amount To Generate" and also from
supporting stopping generation early, which makes an implementation of Dynamic
World Info Scan finally possible.
2021-11-30 10:13:02 -05:00
henk717
9e3318c696
Update colabkobold.sh
...
Bugfix
2021-11-29 18:42:40 +01:00
henk717
4244b588cb
Ngrok requirements
...
Adds ngrok to the requirements.txt files
2021-11-29 18:13:30 +01:00
henk717
fd19e2bfd6
Allow Ngrok
...
Adds --ngrok to the Colab script
2021-11-29 18:12:45 +01:00
henk717
44d8068bab
Ngrok Support
...
Not recommended for home users due to DDoS risks, but might make Colab tunnels more reliable.
2021-11-29 18:11:14 +01:00
henk717
eef675ce21
Further Streamlining Dependencies
...
Hopefully this will make Kaggle work
2021-11-29 16:43:45 +01:00
henk717
ff99e4c0e9
Merge branch 'KoboldAI:main' into united
2021-11-29 12:06:03 +01:00
henk717
d4a7ff5ccb
Better Repetition Penalty Slider
...
Allow users more control since 6B is sensitive
2021-11-29 08:28:51 +01:00
henk717
f7993d5ef1
Merge pull request #84 from adcar/patch-1
...
Fixed a small typo
2021-11-28 17:58:40 +01:00
Alexander D. Cardosi
8d922d83a9
Fixed a small typo
2021-11-28 11:30:23 -05:00
henk717
6b9d744679
Torch needs to be newer
...
Uncapping the version to see the effect (Testing on the git since Colabs load from here)
2021-11-28 11:35:24 +01:00
henk717
b9c7e33410
Switch to official transformers
...
Official transformers is now superior, switching over to allow Colab's to use it.
2021-11-28 05:13:05 +01:00
henk717
939719214d
Merge pull request #40 from VE-FORBRYDERNE/patch
...
Allow bad words filter to ban <|endoftext|> token
2021-11-27 19:23:01 +01:00
9f51c42dd4
Allow bad words filter to ban <|endoftext|> token
...
The official transformers bad words filter doesn't allow this by
default. Finetune's version does allow this by default, however.
2021-11-27 11:42:06 -05:00
henk717
2bc93ba37a
Whitelist 6B in breakmodel
...
Now that we properly support it, allow the menu option to use breakmodel
2021-11-27 10:09:54 +01:00
henk717
b56ee07ffa
Fix for CPU mode
...
Recent optimizations caused the CPU version to load in an incompatible format, now we convert it back to the correct format after loading it efficiently first.
2021-11-27 05:34:29 +01:00
henk717
56c2e619f9
ColabKobold
...
A brand new launcher to power the colab's, you can use https://henk.tech/ckds as a short URL which points towards this github
2021-11-27 03:44:08 +01:00
henk717
3b976c9af7
Updated defaults
...
Transformers official by default, no more Git versions
2021-11-27 03:14:47 +01:00
henk717
6008d4f3a5
Merge pull request #39 from VE-FORBRYDERNE/breakmodel
...
Official transformers 6B breakmodel support and more RAM-efficient model loading
2021-11-27 01:11:48 +01:00
e5e2fb088a
Remember to actually import GPTJModel
2021-11-26 12:38:52 -05:00
871ed65570
Remove an unnecessary **maybe_low_cpu_mem_usage()
2021-11-26 11:42:04 -05:00
a93a76eb01
Load model directly in fp16 if using GPU or breakmodel
2021-11-26 10:55:52 -05:00
95aff61781
Don't pin CPU layers after running out of pinned memory
2021-11-26 10:31:15 -05:00
32e1d4a7a8
Enable low_cpu_mem_usage
2021-11-25 18:09:25 -05:00
25c9be5d02
Breakmodel support for GPTJModel
2021-11-25 18:09:16 -05:00
f8bcc3411b
In breakmodel mode, move layers to GPU as soon as model loads
...
Rather than during the first generation.
2021-11-25 11:44:41 -05:00
henk717
978dc486a5
Merge pull request #38 from VE-FORBRYDERNE/warp
...
Move TFS warper code into aiserver.py
2021-11-24 23:45:28 +01:00
cbb6efb656
Move TFS warper code into aiserver.py
2021-11-24 13:36:54 -05:00
henk717
96e1d98b7e
Merge branch 'KoboldAI:main' into united
2021-11-24 08:24:08 +01:00
henk717
36b9161667
Portability Bugfix
...
Fix an issue where the launcher does not work if the drive is not C: on some systems.
2021-11-24 08:23:08 +01:00
henk717
a2c82bbcc8
num_layers fixes
...
As requested by VE_FORBRYDERNE (Possibly implemented it on to many places, needs testing but since the other one is already broken I am committing it first so I can more easily test)
2021-11-24 03:44:11 +01:00
henk717
d7a2424d2d
No Half on CPU
...
Should fix CPU executions
2021-11-23 17:14:01 +01:00
henk717
11c64c3fe7
Merge pull request #37 from VE-FORBRYDERNE/patch
...
Use model.config.n_layer if model.config.num_layers doesn't exist
2021-11-23 17:02:51 +01:00
be0881a8d0
Use model.config.n_layer if model.config.num_layers doesn't exist
2021-11-23 10:09:24 -05:00
henk717
c0df03fc55
Merge pull request #36 from VE-FORBRYDERNE/sp
...
Fix a typo in tpu_mtj_backend.py
2021-11-23 14:23:10 +01:00
691febacd6
Fix a typo in tpu_mtj_backend.py
2021-11-22 12:53:19 -05:00
henk717
d877190258
Merge pull request #35 from VE-FORBRYDERNE/sp
...
Softprompt support for the TPU backend
2021-11-22 00:33:31 +01:00
9b8bcb5516
Always convert soft prompt to float32 if using TPU backend
...
TPUs do not support float16. Attempting to use a float16 soft prompt
throws an error.
2021-11-21 18:22:10 -05:00
e068aa9f26
Add soft prompt support to TPU backend
2021-11-21 18:08:04 -05:00
henk717
a60e7d3310
Merge pull request #34 from VE-FORBRYDERNE/comments
...
Add support for comments
2021-11-21 16:24:37 +01:00
df2768b745
Simplify the comment regex
2021-11-21 01:09:19 -05:00
7ab0d96b8a
Change the comment regex again to use fixed-length lookbehind
2021-11-21 01:06:31 -05:00
a1c378deea
Fix CSS issues when editing a chunk that has a comment
2021-11-21 00:48:43 -05:00
624cfbd5a4
Use a smarter regex for comments
...
If the beginning of the comment is at the beginning of a line AND the
end of a comment is at the end of a line, an additional newline will now
be ignored so that the AI doesn't see a blank line where the comment
was.
For example, consider the following message:
```
Hello
<|This is
a comment|>
World
```
The AI will now see this:
```
Hello
World
```
instead of this:
```
Hello
World
```
2021-11-21 00:42:57 -05:00
a51f88aeb3
Also apply comment formatting to prompt in refresh_story()
2021-11-21 00:26:45 -05:00
1968be82bb
Remove comments from prompt in WI processor and InferKit mode
2021-11-20 22:23:06 -05:00
8ce8e621ce
Fix typo (one of the comregex_ui should be comregex_ai)
2021-11-20 22:19:12 -05:00
henk717
d7edd9d04b
Merge pull request #33 from VE-FORBRYDERNE/loader
...
Fix a typo in requirements_mtj.txt
2021-11-21 04:09:12 +01:00
c2ed31de28
Add syntax for comments <|...|>
2021-11-20 01:27:57 -05:00
