0cc4m
43b0afc7a8
Add safe MPT support
2023-05-05 20:07:10 +02:00
0cc4m
4180620999
Remove unnecessary changes, move gptq detection function to 4bit.py
2023-05-04 19:52:56 +02:00
0cc4m
d48fedcbfb
Fix llama 4-bit loading error
2023-05-04 18:31:37 +02:00
0cc4m
ef358fdf5a
Merge remote-tracking branch 'origin/united' into model-structure-update
2023-05-04 07:31:13 +02:00
Henk
a87d5d6f23
Remove HF's llama workaround
2023-05-03 20:18:40 +02:00
Llama
35d344b951
Remove torch dependency and more generic dim0 workaround
...
Remove torch dependency from hf.py
Make workaround for dimension zero values of token_ids
more generic to handle every token, not just newlines.
2023-05-03 09:48:16 -07:00
0cc4m
58f0a336cb
Merge upstream changes, fix conflict
2023-05-03 18:33:11 +02:00
Llama
3768848548
Fix tokenization and whitespace issues with llama-derived models
...
Work around the 'soft' prefix space behavior of sentencepiece.
Override encode to restore the deleted HF support for decode_with_prefix_space.
Override decode to skip the soft space and return true decoded tokens.
Allow submitting chat messages with embedded newlines.
Split sentences between punctuation and whitespace, rather than after whitespace.
Also include trailing quotes and brackets after sentence stoppers.
This avoids splitting ." and .) into two tokens, for instance.
Insert whitespace at the beginning of the author's note, since sentences are
split with leading whitespace.
Remove spurious newlines at the end of chat responses.
2023-05-03 01:27:11 -07:00
somebody
a0f4ab5c6a
Move bad token grabber until after newlinemode has been deduced
2023-05-02 20:23:36 -05:00
somebody
efe268df60
Move overrides to better places
2023-05-02 20:18:33 -05:00
0cc4m
9c3d578d6c
Work on model download support
2023-05-02 21:32:20 +02:00
somebody
f6b5548131
Support safetensors in get_sharded_checkpoint_num_tensors
2023-05-01 19:15:27 -05:00
somebody
97e84928ba
Download all shards correctly on aria2 and raise on bad load key
2023-05-01 18:53:36 -05:00
somebody
933dbd634a
HFInferenceModel: Make badwordsids not unique to torch
2023-05-01 17:13:33 -05:00
somebody
ce3d465972
Remove some debug
2023-05-01 17:03:34 -05:00
0cc4m
f83a0aa122
Merge latest changes, fix conflict
2023-05-01 08:01:54 +02:00
0cc4m
aa67135d42
Implement new model format
...
Remove 4bit toggle
2023-04-30 21:59:22 +02:00
0cc4m
20a5587d66
Always use offloader script, because it speeds up multi gpu
2023-04-30 18:17:43 +02:00
onesome
467f2f25eb
More loading fixes
2023-04-26 16:58:33 -05:00
onesome
d4f7b60dc9
Fix for multiple paths
2023-04-26 16:49:12 -05:00
onesome
6776a71532
Add more info to custom model error
2023-04-26 16:36:52 -05:00
onesome
bbf4963d6e
Fix custmodpth stuff for hf loading
2023-04-26 16:18:45 -05:00
onesome
c146ae9d84
Delete legacy gpt2 custom loader
2023-04-26 16:07:18 -05:00
onesome
9579298df7
Better fallback
2023-04-25 22:28:07 -05:00
onesome
6e3aebc1ea
Zap debug
2023-04-25 21:13:17 -05:00
onesome
0268305cfe
Change fallback notifications to warnings
2023-04-25 18:26:49 -05:00
onesome
b8bef641ff
Merge branch 'united' of https://github.com/henk717/KoboldAI into model-structure-and-maybe-rwkv
2023-04-25 16:54:53 -05:00
0cc4m
934571857b
Fix offloading
2023-04-18 22:52:54 +02:00
0cc4m
1ef515f4c2
Fix lazy-loading on 4-bit
2023-04-17 07:21:18 +02:00
0cc4m
4d34f9b7de
Move 4-bit loading code to separate inference_model file
2023-04-16 14:20:13 +02:00
somebody
f9fb5eba89
Remove debug
2023-04-15 18:56:49 -05:00
somebody
5dd67d027a
Workaround for socketio context errors for loading
2023-04-15 18:54:21 -05:00
somebody
08b4e317ff
Fix double slashing
2023-04-15 13:30:05 -05:00
somebody
d3a73aaeba
Fix api
2023-04-15 13:17:20 -05:00
somebody
4dcf570407
Fix legacy model loading
2023-04-15 12:57:35 -05:00
one-some
1b500c7179
Merge pull request #5 from LostRuins/concedo_api
...
Added stop sequences functionality for API calls
2023-04-15 10:51:31 -05:00
somebody
2b950f08d3
Remove legacy no accelerate fallback code
...
Was causing issues with disk cache the old code had a
`and not utils.HAS_ACCELERATE` preceding it (a variable which no longer
exists), and since disk cache is accelerate only, there was no disk
handling code in here. Anyway its bad so blast it
2023-04-15 10:47:31 -05:00
Concedo
9705b7b79c
increase API version (+1 squashed commits)
...
Squashed commits:
[c168c08
2023-04-15 18:09:53 +08:00
somebody
ea8df4c0d3
Merge branch 'united' of https://github.com/henk717/KoboldAI into model-structure-and-maybe-rwkv
2023-04-14 20:38:56 -05:00
somebody
38c53191d3
possible fix for cache dl thing
2023-04-14 20:25:03 -05:00
somebody
8412f83ce5
Breakmodel: Fix typo
2023-04-03 18:41:18 -05:00
somebody
77f0797b1a
Model fix
2023-04-02 15:47:52 -05:00
somebody
9d70646e4d
Lazyload: Safetensors
2023-04-02 15:40:34 -05:00
somebody
91bb433b5f
GenericTokenizer: Fall back to defined tokenizer
...
Shouldn't be relied on for model-agnostic code, but for loading
processes where you know the tokenizer class used it should be okie
dokie
2023-03-19 19:03:20 -05:00
somebody
ffe85ce8a1
Modeling: Fix logits processors (probs, biasing, lua)
2023-03-17 16:56:47 -05:00
somebody
692dbfeb37
Merge branch 'united' of https://github.com/henk717/KoboldAI into model-structure-and-maybe-rwkv
2023-03-17 16:20:13 -05:00
somebody
8d0bc404a5
Model: More Jax import fixes and formatting
2023-03-17 15:36:44 -05:00
somebody
03af06638c
Modeling: Maybe fix samplers
2023-03-13 20:42:35 -05:00
somebody
b93c339145
Model: Lazyload backends
2023-03-13 20:29:29 -05:00
somebody
938c97b75a
RWKV: Fix yet another typo
2023-03-13 19:39:19 -05:00
