277 lines
20 KiB
Plaintext
277 lines
20 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "view-in-github",
|
||
"colab_type": "text"
|
||
},
|
||
"source": [
|
||
"<a href=\"https://colab.research.google.com/github/KoboldAI/KoboldAI-Client/blob/main/colab/GPU.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "kX9y5koxa58q"
|
||
},
|
||
"source": [
|
||
"## [You can get faster generations and higher context with our Koboldcpp Notebook](https://koboldai.org/colabcpp)\n",
|
||
"\n",
|
||
"# Welcome to KoboldAI on Google Colab, GPU Edition!\n",
|
||
"KoboldAI is a powerful and easy way to use a variety of AI based text generation experiences. You can use it to write stories, blog posts, play a text adventure game, use it like a chatbot and more! In some cases it might even help you with an assignment or programming task (But always make sure the information the AI mentions is correct, it loves to make stuff up).\n",
|
||
"\n",
|
||
"For more information about KoboldAI check our our Github readme : https://github.com/KoboldAI/KoboldAI-Client/blob/main/readme.md\n",
|
||
"\n",
|
||
"---\n",
|
||
"## How to load KoboldAI: Everything you need to know\n",
|
||
"1. On a phone? First put your browser in desktop mode because of a Google Colab bug. Otherwise nothing will happen when you click the play button. Then tap the play button next to \"<-- Tap This if you play on Mobile\", you will see an audio player. Keep the audio player playing so Colab does not get shut down in the background.\n",
|
||
"2. Select the desired model, you will find a description of all the available models further down the page.\n",
|
||
"3. Click the play button next to \"<-- Select your model below and then click this to start KoboldAI\".\n",
|
||
"4. Got a message saying no accelerator is available? Click cancel, and try again in a few minutes. If you do not manage to get a session when you frequently try again try at a different time of day, colab can be busy or your priority may have been lowered by frequent usage.\n",
|
||
"5. After everything is done loading you will get a link that you can use to open KoboldAI. In case of Localtunnel you will also be warned that some people are abusing Localtunnel for phishing, once you acknowledge this warning you will be taken to KoboldAI's interface. If you picked Cloudflare and get a 1033 error refresh the error page after waiting one minute.\n",
|
||
"\n",
|
||
"---\n",
|
||
"\n",
|
||
"Further down the page you can find descriptions of the models, and tips to get the most out of your Google Colab experience.\n",
|
||
"\n",
|
||
"Make sure to keep this page open while you are using KoboldAI, and check back regularly to see if you got a Captcha. Failure to complete the captcha's in time can result in termination of your session or a lower priority towards the TPUs.\n",
|
||
"\n",
|
||
"Firefox users need to disable the enhanced tracking protection or use a different browser in order to be able to use Google Colab without errors (This is not something we can do anything about, the cookie blocker breaks the Google Drive integration because it uses different domains)."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"id": "ewkXkyiFP2Hq"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"#@title <-- Tap this if you play on Mobile { display-mode: \"form\" }\n",
|
||
"%%html\n",
|
||
"<b>Press play on the music player to keep the tab alive, then start KoboldAI below (Uses only 13MB of data)</b><br/>\n",
|
||
"<audio src=\"https://raw.githubusercontent.com/KoboldAI/KoboldAI-Client/main/colab/silence.m4a\" controls>"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"cellView": "form",
|
||
"id": "lVftocpwCoYw"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"#@title <b><-- Select your model below and then click this to start KoboldAI</b>\n",
|
||
"#@markdown You can find a description of the models below along with instructions on how to start KoboldAI.\n",
|
||
"\n",
|
||
"Model = \"Nerys V2 6B\" #@param [\"Tiefighter 13B (United)\", \"Echidna 13B (United)\", \"HoloMax 13B (United)\", \"Emerhyst 13B (United)\", \"MythoMax 13B (United)\", \"Huginn 13B (United)\", \"Chronos 13B (United)\", \"Airoboros M2.0 13B (United)\", \"Holodeck 13B (United)\", \"Spring Dragon 13B (United)\", \"Nerys V2 6B\", \"Skein 6B\", \"Janeway 6B\", \"Adventure 6B\", \"Nerys 2.7B\", \"AID 2.7B\", \"Janeway 2.7B\", \"Picard 2.7B\", \"OPT 2.7B\", \"Fairseq Dense 2.7B\", \"Neo 2.7B\"] {allow-input: true}\n",
|
||
"Revision = \"\" #@param [\"\"]{allow-input: true}\n",
|
||
"Version = \"Official\" #@param [\"Official\", \"United\"] {allow-input: true}\n",
|
||
"Provider = \"Cloudflare\" #@param [\"Localtunnel\", \"Cloudflare\"]\n",
|
||
"use_google_drive = True #@param {type:\"boolean\"}\n",
|
||
"\n",
|
||
"import os\n",
|
||
"if not os.path.isfile(\"/opt/bin/nvidia-smi\"):\n",
|
||
" raise RuntimeError(\"⚠️Colab did not give you a GPU due to usage limits, this can take a few hours before they let you back in. Check out https://lite.koboldai.net for a free alternative (that does not provide an API link but can load KoboldAI saves and chat cards) or subscribe to Colab Pro for immediate access.⚠️\")\n",
|
||
"\n",
|
||
"!nvidia-smi\n",
|
||
"from google.colab import drive\n",
|
||
"if use_google_drive:\n",
|
||
" drive.mount('/content/drive/')\n",
|
||
"else:\n",
|
||
" import os\n",
|
||
" if not os.path.exists(\"/content/drive\"):\n",
|
||
" os.mkdir(\"/content/drive\")\n",
|
||
" if not os.path.exists(\"/content/drive/MyDrive/\"):\n",
|
||
" os.mkdir(\"/content/drive/MyDrive/\")\n",
|
||
"\n",
|
||
"if Model == \"Nerys V2 6B\":\n",
|
||
" Model = \"KoboldAI/OPT-6B-nerys-v2\"\n",
|
||
" path = \"\"\n",
|
||
" download = \"\"\n",
|
||
"elif Model == \"Skein 6B\":\n",
|
||
" Model = \"KoboldAI/GPT-J-6B-Skein\"\n",
|
||
" path = \"\"\n",
|
||
" download = \"\"\n",
|
||
"elif Model == \"Janeway 6B\":\n",
|
||
" Model = \"KoboldAI/GPT-J-6B-Janeway\"\n",
|
||
" path = \"\"\n",
|
||
" download = \"\"\n",
|
||
"elif Model == \"Adventure 6B\":\n",
|
||
" Model = \"KoboldAI/GPT-J-6B-Adventure\"\n",
|
||
" path = \"\"\n",
|
||
" download = \"\"\n",
|
||
"elif Model == \"Shinen 6B\":\n",
|
||
" Model = \"KoboldAI/GPT-J-6B-Shinen\"\n",
|
||
" path = \"\"\n",
|
||
" download = \"\"\n",
|
||
"elif Model == \"Nerys 2.7B\":\n",
|
||
" Model = \"KoboldAI/fairseq-dense-2.7B-Nerys\"\n",
|
||
" path = \"\"\n",
|
||
" download = \"\"\n",
|
||
"elif Model == \"Janeway 2.7B\":\n",
|
||
" Model = \"KoboldAI/GPT-Neo-2.7B-Janeway\"\n",
|
||
" path = \"\"\n",
|
||
" download = \"\"\n",
|
||
"elif Model == \"Picard 2.7B\":\n",
|
||
" Model = \"KoboldAI/GPT-Neo-2.7B-Picard\"\n",
|
||
" path = \"\"\n",
|
||
" download = \"\"\n",
|
||
"elif Model == \"AID 2.7B\":\n",
|
||
" Model = \"KoboldAI/GPT-Neo-2.7B-AID\"\n",
|
||
" path = \"\"\n",
|
||
" download = \"\"\n",
|
||
"elif Model == \"Fairseq Dense 2.7B\":\n",
|
||
" Model = \"KoboldAI/fairseq-dense-2.7B\"\n",
|
||
" path = \"\"\n",
|
||
" download = \"\"\n",
|
||
"elif Model == \"OPT 2.7B\":\n",
|
||
" Model = \"facebook/opt-2.7b\"\n",
|
||
" path = \"\"\n",
|
||
" download = \"\"\n",
|
||
"elif Model == \"Neo 2.7B\":\n",
|
||
" Model = \"EleutherAI/gpt-neo-2.7B\"\n",
|
||
" path = \"\"\n",
|
||
" download = \"\"\n",
|
||
"elif Model == \"Tiefighter 13B (United)\":\n",
|
||
" Model = \"KoboldAI/LLaMA2-13B-Tiefighter\"\n",
|
||
" path = \"\"\n",
|
||
" download = \"\"\n",
|
||
" Version = \"United\"\n",
|
||
"elif Model == \"Echidna 13B (United)\":\n",
|
||
" Model = \"NeverSleep/Echidna-13b-v0.3\"\n",
|
||
" path = \"\"\n",
|
||
" download = \"\"\n",
|
||
" Version = \"United\"\n",
|
||
"elif Model == \"Huginn 13B (United)\":\n",
|
||
" Model = \"The-Face-Of-Goonery/Huginn-13b-v1.2\"\n",
|
||
" path = \"\"\n",
|
||
" download = \"\"\n",
|
||
" Version = \"United\"\n",
|
||
"elif Model == \"Chronos 13B (United)\":\n",
|
||
" Model = \"elinas/chronos-13b-v2\"\n",
|
||
" path = \"\"\n",
|
||
" download = \"\"\n",
|
||
" Version = \"United\"\n",
|
||
"elif Model == \"Airoboros M2.0 13B (United)\":\n",
|
||
" Model = \"jondurbin/airoboros-l2-13b-gpt4-m2.0\"\n",
|
||
" path = \"\"\n",
|
||
" download = \"\"\n",
|
||
" Version = \"United\"\n",
|
||
"elif Model == \"Emerhyst 13B (United)\":\n",
|
||
" Model = \"Undi95/Emerhyst-13B\"\n",
|
||
" path = \"\"\n",
|
||
" download = \"\"\n",
|
||
" Version = \"United\"\n",
|
||
"elif Model == \"MythoMax 13B (United)\":\n",
|
||
" Model = \"Gryphe/MythoMax-L2-13b\"\n",
|
||
" Revision = \"\"\n",
|
||
" path = \"\"\n",
|
||
" download = \"\"\n",
|
||
" Version = \"United\"\n",
|
||
"elif Model == \"Spring Dragon 13B (United)\":\n",
|
||
" Model = \"Henk717/spring-dragon\"\n",
|
||
" path = \"\"\n",
|
||
" download = \"\"\n",
|
||
" Version = \"United\"\n",
|
||
"elif Model == \"Holodeck 13B (United)\":\n",
|
||
" Model = \"KoboldAI/LLAMA2-13B-Holodeck-1\"\n",
|
||
" path = \"\"\n",
|
||
" download = \"\"\n",
|
||
" Version = \"United\"\n",
|
||
"elif Model == \"HoloMax 13B (United)\":\n",
|
||
" Model = \"KoboldAI/LLaMA2-13B-Holomax\"\n",
|
||
" path = \"\"\n",
|
||
" download = \"\"\n",
|
||
" Version = \"United\"\n",
|
||
"\n",
|
||
"if Provider == \"Localtunnel\":\n",
|
||
" tunnel = \"--localtunnel yes\"\n",
|
||
"else:\n",
|
||
" tunnel = \"\"\n",
|
||
"\n",
|
||
"!wget https://koboldai.org/ckds -O - | bash /dev/stdin -m $Model -g $Version $Revision $tunnel"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {
|
||
"id": "Lrm840I33hkC"
|
||
},
|
||
"source": [
|
||
"# GPU Edition Model Descriptions\n",
|
||
"| Model | Style | Description |\n",
|
||
"| --- | --- | --- |\n",
|
||
"| [Nerys](https://huggingface.co/KoboldAI/fairseq-dense-2.7B-Nerys) by Mr Seeker | Novel/Adventure | Nerys is a hybrid model based on Pike (A newer Janeway), on top of the Pike dataset you also get some Light Novels, Adventure mode support and a little bit of Shinen thrown in the mix. The end result is a very diverse model that is heavily biased towards SFW novel writing, but one that can go beyond its novel training and make for an excellent adventure model to. Adventure mode is best played from a second person perspective, but can be played in first or third person as well. Novel writing can be done best from the first or third person. |\n",
|
||
"| [Tiefighter 13B by KoboldAI](https://huggingface.co/KoboldAI/LLaMA2-13B-Tiefighter) | Hybrid | Tiefighter 13B is a very versitile fiction Hybrid, it can write, chat and play adventure games and can also answer regular instructions (Although we do not recommend this model for factual use due to its fictional nature). This is an excellent starting model, for the best results avoid using Second person writing in your chats unless you are wanting it to become a text adventure.|\n",
|
||
"| [Janeway](https://huggingface.co/KoboldAI/GPT-Neo-2.7B-Janeway) by Mr Seeker | Novel | Janeway is a model created from Picard's dataset combined with a brand new collection of ebooks. This model is trained on 20% more content than Picard and has been trained on literature from various genres. Although the model is mainly focussed on SFW, romantic scenes might involve a degree of nudity. |\n",
|
||
"| [Picard](https://huggingface.co/KoboldAI/GPT-Neo-2.7B-Picard) by Mr Seeker | Novel | Picard is a model trained for SFW Novels based on Neo 2.7B. It is focused on Novel style writing without the NSFW bias. While the name suggests a sci-fi model this model is designed for Novels of a variety of genre's. It is meant to be used in KoboldAI's regular mode. |\n",
|
||
"| [AID](https://huggingface.co/KoboldAI/GPT-Neo-2.7B-AID) by melastacho | Adventure | Also know as Adventure 2.7B this is a clone of the AI Dungeon Classic model and is best known for the epic wackey adventures that AI Dungeon Classic players love. |\n",
|
||
"| [OPT](https://huggingface.co/facebook/opt-2.7b) by Metaseq | Generic | OPT is considered one of the best base models as far as content goes, its behavior has the strengths of both GPT-Neo and Fairseq Dense. Compared to Neo duplicate and unnecessary content has been left out, while additional literature was added in similar to the Fairseq Dense model. The Fairseq Dense model however lacks the broader data that OPT does have. The biggest downfall of OPT is its license, which prohibits any commercial usage, or usage beyond research purposes. |\n",
|
||
"| [Fairseq Dense](https://huggingface.co/KoboldAI/fairseq-dense-2.7B) | Generic | Trained by Facebook Researchers this model stems from the MOE research project within Fairseq. This particular version has been converted by us for use in KoboldAI. It is known to be on par with the larger models from EleutherAI and considered as better for pop culture and language tasks. Because the model has never seen a new line (enter) it may perform worse on formatting and paragraphing. Compared to other models the dataset focuses primarily on literature and contains little else. |\n",
|
||
"| [MythoMax 13B](https://huggingface.co/TheBloke/MythoMax-L2-13B-GPTQ) by Gryphe | Roleplay | An improved, potentially even perfected variant of MythoMix, my MythoLogic-L2 and Huginn merge using a highly experimental tensor type merge technique¹. |\n",
|
||
"| [Holomax 13B by KoboldAI](https://huggingface.co/KoboldAI/LLaMA2-13B-Holomax) | Adventure | This is an expansion merge to the well-praised MythoMax model from Gryphe (60%) using MrSeeker's KoboldAI Holodeck model (40%). The goal of this model is to enhance story-writing capabilities while preserving the desirable traits of the MythoMax model as much as possible (It does limit chat reply length). |\n",
|
||
"| [Airoboros 13B](https://huggingface.co/jondurbin/airoboros-13b) by Jon Durbin | Generic | This is an instruction fine-tuned llama-2 model, using synthetic instructions generated by airoboros⁵. |\n",
|
||
"| [Emerhyst 13B](https://huggingface.co/Undi95/Emerhyst-13B) by Undi | Roleplay | An attempt using BlockMerge_Gradient to get better result. In addition, LimaRP v3 was used⁷. |\n",
|
||
"| [Chronos 13B](https://huggingface.co/elinas/chronos-13b) by Elinas | Generic | This model is primarily focused on chat, roleplay, and storywriting, but can accomplish other tasks such as simple reasoning and coding. Chronos generates very long outputs with coherent text, largely due to the human inputs it was trained on. |\n",
|
||
"| [Spring Dragon by Henk717](https://huggingface.co/Henk717/spring-dragon) | Adventure | This model is a recreation attempt of the AI Dungeon 2 Dragon model. To achieve this, the \"text_adventures.txt\" dataset was used, which was bundled with the original AI Dungeon 2 GitHub release prior to the online service. It is worth noting that the same dataset file was used to create the Dragon model, where Dragon is a GPT-3 175B Davinci model from 2020. |\n",
|
||
"| [Holodeck By KoboldAI](https://huggingface.co/KoboldAI/LLAMA2-13B-Holodeck-1) | Adventure |LLAMA2 13B-Holodeck is a finetune created using Meta's llama 2 model.The training data contains around 3000 ebooks in various genres. Most parts of the dataset have been prepended using the following text: [Genre: <genre1>, <genre2>|\n",
|
||
"| [Neo](https://huggingface.co/EleutherAI/gpt-neo-2.7B) by EleutherAI | Generic | This is the base model for all the other 2.7B models, it is best used when you have a use case that we have no other models available for, such as writing blog articles or programming. It can also be a good basis for the experience of some of the softprompts if your softprompt is not about a subject the other models cover. |\n",
|
||
"\n",
|
||
"\n",
|
||
"| Style | Description |\n",
|
||
"| --------- | ------------------------------------------------------------ |\n",
|
||
"| Novel | For regular story writing, not compatible with Adventure mode or other specialty modes. |\n",
|
||
"| Adventure | These models are excellent for people willing to play KoboldAI like a Text Adventure game and are meant to be used with Adventure mode enabled. Even if you wish to use it as a Novel style model you should always have Adventure mode on and set it to story. These models typically have a strong bias towards the use of the word You and without Adventure mode enabled break the story flow and write actions on your behalf. |\n",
|
||
"| Generic | Generic models are not trained towards anything specific, typically used as a basis for other tasks and models. They can do everything the other models can do, but require much more handholding to work properly. Generic models are an ideal basis for tasks that we have no specific model for, or for experiencing a softprompt in its raw form. |\n",
|
||
"\n",
|
||
"---\n",
|
||
"# How to start KoboldAI in 7 simple steps\n",
|
||
"Using KoboldAI on Google Colab is easy! Simply follow these steps to get started:\n",
|
||
"1. Mobile phone? Tap the play button below next to \"<--- Tap this if you play on mobile\" to reveal an audio player, play the silent audio to keep the tab alive so Google will not shut you down when your using KoboldAI. If no audio player is revealed your phone browser does not support Google Colab in the mobile view, go to your browser menu and enable Desktop mode before you continue.\n",
|
||
"2. Select the model that most describes what you would like to do, by default we have the most recommended model for people willing to try out KoboldAI selected. If you are an advanced user you can also type any GPT model name from Huggingface.co to load this up (Unlisted Models may or may not work depending on Colab's hardware limitations).\n",
|
||
"3. Click the play button next to \"<--- Click this to start KoboldAI\".\n",
|
||
"4. Allow Google Drive access, this typically happens trough a popup but sometimes Google Drive access may be requested trough the older method by asking you to click on a link and copy a code. This is normal behavior for Colab and only you will get access to your files, nothing is shared with us.\n",
|
||
"5. Now the automatic installation and Download process starts, for most models in the GPU edition expect this to take 7 minutes on average depending on the current Colab download speeds. These downloads happen trough Google's internet connection, you will not be billed by your internet provider and it will not count towards any download limits.\n",
|
||
"6. After waiting a Trycloudflare link appears, click the link to enjoy KoboldAI. If you get a 1033 error Cloudflare is not done loading, in that case keep refreshing until it goes away. (If it keeps happening after 2 minutes Cloudflare has an issue, in that case you can use Runtime -> Restart and Run All to get a new link).\n",
|
||
"7. As you play KoboldAI, keep this Colab tab open in the background and check occationally for Captcha's so they do not shut your instance down. If you do get shut down you can always download a copy of your gamesave in the Save menu inside KoboldAI. Stories are never lost as long as you keep KoboldAI open in your browser.\n",
|
||
"\n",
|
||
"Get a error message saying you do not have access to a GPU/TPU instance? Do not continue and try again later, KoboldAI will not run correctly without them."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {
|
||
"cellView": "form",
|
||
"id": "5k8fK4F6UiTs"
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"#@title <b>Model Cleaner</b>\n",
|
||
"#@markdown Out of space? Run this to remove all cached models (Google Drive models are not effected).\n",
|
||
"!rm -rf /content/KoboldAI-Client/cache/*\n"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"accelerator": "GPU",
|
||
"colab": {
|
||
"name": "ColabKobold GPU",
|
||
"private_outputs": true,
|
||
"provenance": [],
|
||
"include_colab_link": true
|
||
},
|
||
"kernelspec": {
|
||
"display_name": "Python 3",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"name": "python"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 0
|
||
} |