3869 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
0cc4m	e742083703	Fix multi-gpu-offloading	2023-04-02 11:17:29 +02:00
0cc4m	2729b77640	Add offload.py adapted from llama_inference_offload.py, with multi-gpu support and some improvements. Not yet functional, and still just supports Llama	2023-04-02 10:32:19 +02:00
Henk	4a8b099888	Model loading fix	2023-04-02 00:29:56 +02:00
0cc4m	110f8229c5	Add cudatoolkit-dev for compilation, compatible gcc 9 and update transformers to fix error in gptq	2023-04-01 21:33:05 +02:00
0cc4m	bf0c999412	Update GPTQ to support AMD	2023-04-01 14:19:51 +02:00
0cc4m	d3a5ca6505	Update gptq submodule to latest	2023-04-01 08:52:08 +00:00
0cc4m	6eae457479	Fix 4bit groupsize param letter ... Use g instead of b for groupsize name, for example 4bit-128g.safetensors	2023-03-31 15:36:03 +02:00
henk717	72b4669563	Fix the chex dependency	2023-03-30 23:41:35 +02:00
0cc4m	aa2292b3a4	Enable multi-gpu support	2023-03-30 19:40:49 +02:00
0cc4m	61b13604b6	Fix bug in 4-bit load fallback	2023-03-30 10:57:04 +02:00
henk717	943d0fe68a	Merge branch 'KoboldAI:main' into united	2023-03-30 00:51:17 +02:00
henk717	ab779efe0e	Merge pull request #276 from YellowRoseCx/stable-branch ... Update README and remove unavailable model from gpu.ipynb	2023-03-30 00:50:15 +02:00
YellowRoseCx	3c48a77a52	Update README.md ... changed Colab GPU models listed to their higher quality counter parts	2023-03-29 17:44:44 -05:00
YellowRoseCx	f826930c02	Update GPU.ipynb ... removed litv2-6B-rev3	2023-03-29 17:41:01 -05:00
0cc4m	9d0477f5f7	Fix bug where it picks old model despite new one available	2023-03-29 22:05:44 +00:00
0cc4m	73d5ec0e5d	Pull latest gptq-changes	2023-03-29 20:07:26 +00:00
0cc4m	a0bc770426	Add basic groupsize support ... Write groupsize into filename, for example 4bit-128b.safetensors for groupsize 128	2023-03-29 19:49:05 +00:00
0cc4m	f6f7687cc0	Add 4bit safetensor support, improve loading code	2023-03-29 14:47:59 +00:00
0cc4m	8d008b87a6	Add OPT support	2023-03-29 13:27:11 +00:00
Digitous	e698f22706	Update README.md	2023-03-28 19:14:46 -04:00
0cc4m	ef6fe680a9	Fix high VRAM usage caused by workaround for scalar type error	2023-03-28 06:30:02 +00:00
henk717	66264d38c4	Add Mixes	2023-03-28 00:23:10 +02:00
0cc4m	0f1fc46078	Fix errors during inference	2023-03-27 21:30:43 +00:00
0cc4m	d1a2005a27	Add support for old and new 4-bit format. Old one needs 4bit-old.pt file to launch	2023-03-27 20:45:21 +00:00
0cc4m	2e7a8a1a66	Adapt KoboldAI to latest gptq changes	2023-03-27 04:48:21 +00:00
henk717	37c3fd00b9	Merge pull request #315 from jojorne/jojorne-patch-enable-renaming-deleting-wi-root-folder ... Enable renaming/deleting wi root folder by creating a new one	2023-03-26 16:30:41 +02:00
henk717	bbb554efd3	Merge branch 'KoboldAI:main' into united	2023-03-26 01:45:52 +01:00
0cc4m	9dcba38978	Pin transformers to a working Llama-compatible version	2023-03-24 19:07:28 +00:00
0cc4m	026eb3205e	Fix 4-bit loading error when not loading in 4-bit	2023-03-22 22:12:06 +00:00
0cc4m	8941428c66	Fix Kobold loading to CPU in 4-bit, causing CUDA ASSERT error	2023-03-22 06:22:34 +00:00
0cc4m	c7edc764b9	Fix llama loading	2023-03-21 21:58:31 +00:00
0cc4m	ecd065a881	Overhaul 4-bit support to load with a toggle	2023-03-21 21:40:59 +00:00
0cc4m	4cfc1219d4	Add gptq as submodule	2023-03-20 19:13:46 +00:00
0cc4m	3b7505dc28	Merge remote-tracking branch 'united/united' into 4bit	2023-03-20 19:06:40 +00:00
0cc4m	858657f669	Fix zipfile folder identification fix for Windows	2023-03-20 09:16:30 +01:00
0cc4m	60acf59316	Improve 4-bit llama support, add 4-bit gptj and gptneox support	2023-03-19 21:20:13 +00:00
henk717	94eb8ff825	TPU Message	2023-03-19 14:52:14 +01:00
Henk	90a7eb6153	LLama tokenizer settings	2023-03-17 12:40:08 +01:00
Henk	1235b71bb5	Merge branch 'main' into united	2023-03-17 01:48:10 +01:00
Henk	219b824b9b	SocketIO Requirements Pin	2023-03-17 01:28:59 +01:00
henk717	86c87b23c0	Merge branch 'KoboldAI:main' into united	2023-03-17 01:17:58 +01:00
0cc4m	5d17692c79	Remove except Exception so that errors actually show up	2023-03-16 05:24:58 +00:00
YellowRoseCx	b3b454bbe4	Update huggingface.yml	2023-03-15 00:03:43 -05:00
YellowRoseCx	bf677a32f6	Merge remote-tracking branch 'catboxanon/test/4bit' into yr4bit	2023-03-14 17:09:06 -05:00
YellowRoseCx	2909910bcc	Merge branch 'henk717:united' into dev-yr	2023-03-14 17:04:33 -05:00
jojorne	8378ff9e26	Fix typo	2023-03-14 04:20:32 -03:00
jojorne	8d5a581d5d	Enable renaming deleting wi root folder by creating a new one	2023-03-14 04:04:01 -03:00
henk717	db7b53f52d	Merge pull request #310 from nkpz/united ... Fix out of range error after editing actions	2023-03-14 01:22:14 +01:00
catboxanon	5f3770bb58	Merge branch 'henk717:united' into test/4bit	2023-03-13 19:34:17 -04:00
henk717	5249045c35	Merge pull request #304 from YellowRoseCx/united-yr ... added local rng_states variable and fixed minor typo	2023-03-13 22:46:00 +01:00