65 Commits

Author	SHA1	Message	Date
Cohee	c8b9b62d8a	Update built-in extensions to use fixed order in extensions menu	2024-06-24 22:15:08 +03:00
Cohee	8564d6faa8	Debug function to purge all vectors	2024-06-22 17:41:02 +03:00
Cohee	b8830e34d3	Add ollama download shortcut to vector storage	2024-06-22 16:38:00 +03:00
Cohee	bba16f5263	Custom vector chunk boundary	2024-06-16 02:16:27 +03:00
Cohee	b22bc47c4f	Update ext.macro calls in built-in extensions	2024-06-15 01:40:16 +03:00
Cohee	4e822eeebb	Add VLLM as vector source	2024-06-09 01:03:22 +03:00
Cohee	bcfc4d5c64	Split overlap size in two	2024-06-06 23:00:49 +03:00
Cohee	39721b6a8f	Add file chunks overlap control	2024-06-06 21:45:47 +03:00
Cohee	716366070b	Clamp /db-search threshold arg	2024-05-30 17:15:17 +03:00
Cohee	2c911a3ea2	Add more Data Bank script commands	2024-05-30 14:49:57 +03:00
Cohee	2b3dfc5ae2	Add ollama and llamacpp as vector sources	2024-05-28 22:54:50 +03:00
Cohee	967a7980f5	Add vector retrieval score threshold	2024-05-23 17:28:43 +03:00
Cohee	5147233391	Fix file attachment chunk size inconsistency	2024-05-13 00:27:32 +03:00
Cohee	d9d76ba16d	#2164 Add error toasts to VecStore	2024-04-30 00:17:39 +03:00
Cohee	6ac6c7cfda	#2159 Move debounce constants to a separate module	2024-04-28 19:47:53 +03:00
Cohee	de2bb7938a	Utilize import for vector store	2024-04-28 14:35:35 +03:00
Wolfsblvt	d7ade487b8	Refactor common enum for debounce timeouts	2024-04-28 06:21:47 +02:00
Cohee	75372ad0cc	Use Map for caches instead of objects	2024-04-23 16:15:54 +03:00
Cohee	4370db6bdc	Implement World Info activation using Vector Storage	2024-04-23 03:09:52 +03:00
Cohee	ca89be8930	Add experimental setting for file translation	2024-04-21 03:24:01 +03:00
Cohee	a3f6ce52e4	Fix manual vectorization of files	2024-04-19 18:43:35 +03:00
Cohee	80de3fdd4c	Add buttons to process and purge file vectors for current chat	2024-04-19 00:16:23 +03:00
Cohee	25cb598694	Add Cohere as embedding source	2024-04-19 00:07:12 +03:00
Cohee	2eafa2a212	Clean-up vectors upon deleting a file from Data Bank	2024-04-18 23:07:16 +03:00
Cohee	8434f6e6cf	Clear toast upon inserting file	2024-04-18 22:59:42 +03:00
Cohee	16785ae005	Merge branch 'staging' into neo-server	2024-04-18 22:57:27 +03:00
Kristan Schlikow	10692523f0	Improve settings, add settings for prompt, sent message summarize toggle	2024-04-17 20:35:50 +02:00
Kristan Schlikow	4cfd0c71cf	Add extras summarization API, fix missing substitution of placeholders	2024-04-17 20:35:49 +02:00
Kristan Schlikow	a1473dedd0	Make summarization toggleable	2024-04-17 20:35:49 +02:00
Kristan Schlikow	59abee3043	Summarize the chat messages handed to the embedding beforehand	2024-04-17 20:35:48 +02:00
Cohee	9a1ea7f226	Implement Data Bank vectors querying	2024-04-17 02:09:22 +03:00
Cohee	4665db62f4	#1954 Remove backtick wrapping for inserted files	2024-04-16 22:28:10 +03:00
Cohee	123131074c	Change built-in extensions template render to async	2024-04-11 23:38:44 +03:00
Cohee	5efdecc825	Add new OpenAI embedding models for selection	2024-03-12 21:41:30 +02:00
Kristian Schlikow	44a7dd3d74	Add NomicAI for vectorization (#1922) ... * Crudely add NomicAi for vectorization * Move NomicAI to its own endpoint, properly handle API key * Adjust clear button html * Remove leftover nomicai http header code * Revert changes to openai-vectors.js * Fix UI issues * Revert change to settings, fix UI --------- Co-authored-by: Cohee <18619528+Cohee1207@users.noreply.github.com>	2024-03-12 21:10:25 +02:00
Cohee	b490978142	Refactor vector models code	2024-03-02 23:16:18 +02:00
Kristan Schlikow	adfb9c5097	Implement TogetherAI as vectorization provider	2024-03-01 23:52:49 +01:00
Cohee	7b8ac8f4c4	Properly use vector insert setting	2024-02-24 15:57:26 +02:00
Cohee	737a0bd3ae	Fix purge extras and mistral vectors	2024-02-23 22:37:00 +02:00
Cohee	3cf01e765a	#1741 Filter non-unique chunks	2024-01-26 04:18:15 +02:00
Cohee	0ad756c923	Add check for "embeddings" module.	2024-01-24 16:51:57 +02:00
Juha Jeronen	8b43535352	Initial support for Extras vectorizer, for Vector Storage	2024-01-24 15:16:55 +02:00
Cohee	a2e4dc2950	Add chunking of vector storage messages	2023-12-31 04:00:04 +02:00
based	5dd2e8cd88	added mistral vector support (off the back of oai's)	2023-12-16 08:37:39 +10:00
Cohee	a6bb754567	Fix API key access	2023-12-14 19:56:43 +02:00
Cohee	1b11ddc26a	Add vector storage to WI scanning	2023-12-11 22:47:26 +02:00
valadaptive	a37f874e38	Require single quotes	2023-12-02 13:04:51 -05:00
valadaptive	c893e2165e	Enable no-prototype-builtins lint	2023-12-02 12:10:31 -05:00
Cohee	bb727d5f9c	Insert files before user text, not after	2023-12-01 03:54:28 +02:00
Cohee	19df1f52cd	Vector storage file retrieval	2023-11-30 00:01:59 +02:00