50 Commits

Author	SHA1	Message	Date
Henk	9e51a50bcd	Llama fixes for Mistral	2023-09-28 01:08:53 +02:00
Henk	0d0a671bb9	Better use_cache implementation	2023-09-07 04:29:28 +02:00
Henk	dfb63b2340	HF 4.33.1	2023-09-06 20:21:23 +02:00
Henk	f40236c04a	Modern llama tokenizer	2023-08-25 14:27:44 +02:00
Henk	d93631c889	GPTQ improvements	2023-08-19 14:45:45 +02:00
somebody	906d1f2522	Merge branch 'united' of https://github.com/henk717/KoboldAI into fixing-time	2023-08-07 16:22:04 -05:00
0cc4m	a9aa04fd1b	Merge remote-tracking branch 'upstream/united' into 4bit-plugin	2023-07-23 07:18:58 +02:00
0cc4m	09bb1021dd	Fallback to transformers if hf_bleeding_edge not available	2023-07-23 07:16:52 +02:00
Henk	432cdc9a08	Fix models with good pad tokens	2023-07-21 16:39:58 +02:00
Henk	ec745d8b80	Dont accidentally block pad tokens	2023-07-21 16:25:32 +02:00
somebody	fef42a6273	API: Fix loading	2023-07-19 11:52:39 -05:00
0cc4m	e78361fc8f	Pull upstream changes, fix conflicts	2023-07-15 23:01:52 +02:00
somebody	20b4b4bcef	Add basic hf backend	2023-07-08 17:12:16 -05:00
0cc4m	e49d35afc9	Add 4bit plugin	2023-05-28 22:54:36 +02:00
0cc4m	d71a63fa49	Merge ebolam's model-plugins branch	2023-05-28 09:26:13 +02:00
somebody	1546b9efaa	Hello its breaking breakmodel time	2023-05-27 16:31:53 -05:00
ebolam	47276c3424	Bug Fix	2023-05-27 08:49:21 -04:00
ebolam	cce5c1932c	Fix for custom model names	2023-05-26 21:40:39 -04:00
ebolam	9723154bed	Fix for --path	2023-05-26 20:10:11 -04:00
ebolam	51cea7eb9d	Added ability to add labels that are based on validation data in model loading settings	2023-05-26 12:43:17 -04:00
ebolam	1a1b79a16d	Change default for HF on non-breakmodel models to use GPU instead of CPU	2023-05-24 12:44:26 -04:00
ebolam	068173b24a	Potential BadWords fix	2023-05-24 09:08:34 -04:00
ebolam	9bd445c2a8	gpt2 fixed	2023-05-23 20:33:55 -04:00
ebolam	839d56ebf2	Potential fix for gpt-neo and gpt-j	2023-05-23 19:25:01 -04:00
ebolam	4c25d6fbbb	Fix for loading model multiple times loosing the gpu/cpu splits	2023-05-22 20:34:01 -04:00
ebolam	9e53bcf676	Fix for breakmodel loading to CPU when set to GPU	2023-05-22 20:24:57 -04:00
ebolam	dc20e6dde9	Fix for unloading models	2023-05-22 15:04:33 -04:00
ebolam	925cad2e2f	Better compatibility with hf model backend	2023-05-22 14:50:13 -04:00
ebolam	513b8575e7	Fix for missing import ... Fix for model name being a path which caused save issues	2023-05-20 11:01:49 -04:00
ebolam	9df1f03b12	Fix for custom huggingface model menu entry	2023-05-19 14:28:36 -04:00
ebolam	99cffd4755	Colab GPU edition fixes	2023-05-19 09:11:08 -04:00
ebolam	06f59a7b7b	Moved model backends to separate folders ... added some model backend settings save/load	2023-05-18 20:14:33 -04:00
ebolam	71aee4dbd8	First concept of model plugins with a conceptual UI. ... Completely breaks UI2 model loading.	2023-05-10 16:30:46 -04:00
0cc4m	a2d01bb9e4	Update to GPTQ module 0.0.2, add support for upstream cuda quantizations, automatic detection	2023-05-09 22:20:35 +02:00
0cc4m	2f7856f0d1	Use GPTQ python module, add MPT quantized support	2023-05-06 20:52:42 +02:00
0cc4m	43b0afc7a8	Add safe MPT support	2023-05-05 20:07:10 +02:00
0cc4m	ef358fdf5a	Merge remote-tracking branch 'origin/united' into model-structure-update	2023-05-04 07:31:13 +02:00
Henk	a87d5d6f23	Remove HF's llama workaround	2023-05-03 20:18:40 +02:00
Llama	35d344b951	Remove torch dependency and more generic dim0 workaround ... Remove torch dependency from hf.py Make workaround for dimension zero values of token_ids more generic to handle every token, not just newlines.	2023-05-03 09:48:16 -07:00
0cc4m	58f0a336cb	Merge upstream changes, fix conflict	2023-05-03 18:33:11 +02:00
Llama	3768848548	Fix tokenization and whitespace issues with llama-derived models ... Work around the 'soft' prefix space behavior of sentencepiece. Override encode to restore the deleted HF support for decode_with_prefix_space. Override decode to skip the soft space and return true decoded tokens. Allow submitting chat messages with embedded newlines. Split sentences between punctuation and whitespace, rather than after whitespace. Also include trailing quotes and brackets after sentence stoppers. This avoids splitting ." and .) into two tokens, for instance. Insert whitespace at the beginning of the author's note, since sentences are split with leading whitespace. Remove spurious newlines at the end of chat responses.	2023-05-03 01:27:11 -07:00
somebody	a0f4ab5c6a	Move bad token grabber until after newlinemode has been deduced	2023-05-02 20:23:36 -05:00
somebody	efe268df60	Move overrides to better places	2023-05-02 20:18:33 -05:00
0cc4m	9c3d578d6c	Work on model download support	2023-05-02 21:32:20 +02:00
somebody	933dbd634a	HFInferenceModel: Make badwordsids not unique to torch	2023-05-01 17:13:33 -05:00
onesome	d4f7b60dc9	Fix for multiple paths	2023-04-26 16:49:12 -05:00
onesome	6776a71532	Add more info to custom model error	2023-04-26 16:36:52 -05:00
onesome	bbf4963d6e	Fix custmodpth stuff for hf loading	2023-04-26 16:18:45 -05:00
somebody	8c8bdfaf6a	Model: Fix assorted bugs ... and ignore warnings in pytest	2023-03-10 20:00:39 -06:00
somebody	54cecd4d5d	Model: And another refactor	2023-03-01 19:16:35 -06:00