110 Commits

Author	SHA1	Message	Date
ebolam	1500d07c38	Merge commit 'refs/pull/352/head' of https://github.com/ebolam/KoboldAI into UI2	2022-12-20 21:24:06 -05:00
Henk	eeb1774d42	Cleaner implementation of zipfolder	2022-12-10 19:23:08 +01:00
Henk	9a8e8a0005	New pytorch zipfile support	2022-12-10 19:11:07 +01:00
ebolam	c1cea71ed6	Merge commit 'refs/pull/331/head' of https://github.com/ebolam/KoboldAI into UI2	2022-12-07 12:04:42 -05:00
vfbd	9a3f0eaab2	Only enable TPU transpose optimization if loading from HF model	2022-11-21 13:47:18 -05:00
Henk	eb52ebd082	Merge branch 'main' into united	2022-11-03 00:22:30 +01:00
vfbd	b20d80ca2a	Add vocab padding to embedding bias in gptj.json	2022-11-02 19:02:09 -04:00
ebolam	c3ccdb0b50	Potential Colab URL fix	2022-10-28 14:19:50 -04:00
ebolam	c3180fb06f	Move display of Colab link to later in the load process for TPUs	2022-10-25 09:41:59 -04:00
ebolam	c83642dbbc	Better Colab status. Disconnects due to long running something still	2022-10-24 20:43:34 -04:00
ebolam	405578f2b3	Colab Update	2022-10-24 20:28:56 -04:00
ebolam	25dbe7ff81	Colab Fixes	2022-10-24 19:40:09 -04:00
ebolam	c2eadcddf6	Typo	2022-10-24 19:33:28 -04:00
ebolam	c1df2c7862	Colab Fix	2022-10-24 19:32:17 -04:00
ebolam	693e7500cf	Debug	2022-10-24 19:28:56 -04:00
ebolam	f0dfde9a64	Colab debug	2022-10-24 19:27:48 -04:00
ebolam	1950eb6b4f	Colab Debug	2022-10-24 19:18:40 -04:00
ebolam	db2fbbb17a	Colab Debug	2022-10-24 19:16:03 -04:00
ebolam	46ac4ec009	Slight fix	2022-10-24 19:01:11 -04:00
ebolam	47c832fde4	Add Original UI status for TPU connection/loading	2022-10-24 18:52:53 -04:00
ebolam	b01713875c	Add status bar message and status bar for TPU loading	2022-10-24 18:34:45 -04:00
ebolam	e101e43132	Merge commit 'refs/pull/243/head' of https://github.com/ebolam/KoboldAI into UI2	2022-10-23 12:03:40 -04:00
ebolam	0a81727fc2	Merge commit 'refs/pull/227/head' of https://github.com/ebolam/KoboldAI into UI2	2022-10-19 10:32:04 -04:00
vfbd	bdc73ef393	Decrease TPU loading times by eliminating a transpose operation	2022-10-12 14:31:18 -04:00
ebolam	dfb05968c9	Merge commit 'refs/pull/188/head' of https://github.com/ebolam/KoboldAI into UI2	2022-10-07 14:31:59 -04:00
vfbd	323f593a96	Custom unpickler to avoid pickle's arbitrary code execution vulnerability	2022-10-06 20:08:08 -04:00
vfbd	bdfa6d86b7	Seed has to be a 64-bit unsigned int or PyTorch will throw an error ... tpu_mtj_backend's seed can be an integer of arbitrary size but we will limit it to a 64-bit unsigned integer anyways for consistency.	2022-10-02 17:50:32 -04:00
vfbd	1a59a4acea	Allow changing sampler seed and sampler order from API	2022-10-02 16:25:51 -04:00
ebolam	c28ad27c1d	Merge commit 'refs/pull/181/head' of https://github.com/ebolam/KoboldAI into UI2	2022-09-28 12:59:53 -04:00
Henk	6c32bc18d7	GPT2Tokenizer for TPU	2022-09-27 18:33:31 +02:00
Henk	11455697ef	Tokenizer Fixes (Slow first to keep coherency)	2022-09-27 17:57:18 +02:00
Henk	07896867b2	Revert Tokenizer Change	2022-09-27 15:36:08 +02:00
Henk	82a250aa1b	Revert "Fix tokenizer selection code" ... This reverts commit 7fba1fd28a.	2022-09-27 15:33:08 +02:00
ebolam	3afd617cb4	Merge commit 'refs/pull/179/head' of https://github.com/ebolam/KoboldAI into UI2	2022-09-27 08:21:05 -04:00
vfbd	79ae0f17ec	Merge branch 'main' into merge	2022-09-26 16:10:10 -04:00
vfbd	7fba1fd28a	Fix tokenizer selection code	2022-09-26 14:37:25 -04:00
ebolam	740e6592ce	Added proper tpu load status to the UI	2022-09-25 20:37:25 -04:00
ebolam	55771c1edf	Giving proper TPU load progress in UI	2022-09-25 20:03:15 -04:00
ebolam	d1562af472	Merge commit 'refs/pull/138/head' of https://github.com/ebolam/KoboldAI into UI2	2022-09-18 19:20:20 -04:00
vfbd	943614b5e6	Merge branch 'main' into dependency-fix	2022-09-15 17:33:48 -04:00
vfbd	7bf6c9a23f	Remove TPU Colab's dependency on optax and chex	2022-09-15 13:47:48 -04:00
ebolam	e9678ceea3	Merge commit 'refs/pull/82/head' of https://github.com/ebolam/KoboldAI into UI2	2022-08-30 14:50:28 -04:00
vfbd	cbacfbdfac	Fix error that occurs when using dynamic TPU backend	2022-08-27 17:42:49 -04:00
vfbd	938e1eddf3	Fix jax.lax.cond call	2022-08-23 18:13:46 -04:00
vfbd	6ffaf43548	Repetition penalty is now sampler #6 in the sampler order	2022-08-23 15:10:21 -04:00
vfbd	74922966bd	Merge branch 'avril' into rep-pen-order	2022-08-23 14:47:29 -04:00
ebolam	21c08353b9	renamed vars to koboldai_vars	2022-08-17 09:26:45 -04:00
henk717	9e140e3ba9	Merge branch 'KoboldAI:main' into united	2022-07-05 21:35:53 +02:00
vfbd	2a78b66932	Fix base OPT-125M and finetuned OPT models in Colab TPU instances	2022-07-05 15:28:58 -04:00
vfbd	048bd0ff3b	Add support for setting the RNG seed and full determinism	2022-06-28 13:21:05 -04:00