80 lines
4.4 KiB
Plaintext
80 lines
4.4 KiB
Plaintext
Thanks for checking out the KoboldAI Client! Get support and updates on the subreddit:
|
|
https://www.reddit.com/r/KoboldAI/
|
|
|
|
[ABOUT]
|
|
|
|
This is a browser-based front-end for AI-assisted writing with multiple local & remote AI models.
|
|
It offers the standard array of tools, including Memory, Author's Note, World Info, Save & Load,
|
|
adjustable AI settings, formatting options, and the ability to import exising AI Dungeon adventures.
|
|
Current UI Snapshot: https://imgur.com/mjk5Yre
|
|
|
|
For local generation, KoboldAI uses Transformers (https://huggingface.co/transformers/) to interact
|
|
with the AI models. This can be done either on CPU, or GPU with sufficient hardware. If you have a
|
|
high-end GPU with sufficient VRAM to run your model of choice, see
|
|
(https://www.tensorflow.org/install/gpu) for instructions on enabling GPU support.
|
|
|
|
Transformers/Tensorflow can still be used on CPU if you do not have high-end hardware, but generation
|
|
times will be much longer. Alternatively, KoboldAI also supports utilizing remotely-hosted models.
|
|
The currently supported remote APIs are InferKit and Google Colab, see the dedicated sections below
|
|
for more info on these.
|
|
|
|
[SETUP]
|
|
|
|
1. Install a 64-bit version of Python.
|
|
(Development was done on 3.7, I have not tested newer versions)
|
|
Windows download link: https://www.python.org/ftp/python/3.7.9/python-3.7.9-amd64.exe
|
|
2. When installing Python make sure "Add Python to PATH" is selected.
|
|
(If pip isn't working, run the installer again and choose Modify to choose Optional features.)
|
|
3. Run install_requirements.bat.
|
|
(This will install the necessary python packages via pip)
|
|
4. Run play.bat
|
|
5. Select a model from the list. Flask will start and give you a message that it's ready to connect.
|
|
6. Open a web browser and enter http://127.0.0.1:5000/
|
|
|
|
[ENABLE COLORS IN WINDOWS 10 COMMAND LINE]
|
|
|
|
If you see strange numeric tags in the console output, then your console of choice does not have
|
|
color support enabled. On Windows 10, you can enable color support by lanching the registry editor
|
|
and adding the REG_DWORD key VirtualTerminalLevel to Computer\HKEY_CURRENT_USER\Console and setting
|
|
its value to 1.
|
|
|
|
[ENABLE GPU FOR SUPPORTED VIDEO CARDS]
|
|
|
|
1. Install NVidia CUDA toolkit from https://developer.nvidia.com/cuda-10.2-download-archive
|
|
2. Visit PyTorch's website(https://pytorch.org/get-started/locally/) and select Pip under "Package"
|
|
and your version of CUDA under "Compute Platform" (I linked 10.2) to get the pip3 command.
|
|
3. Copy and paste pip3 command into command prompt to install torch with GPU support
|
|
|
|
Be aware that when using GPU mode, inference will be MUCH faster but if your GPU doesn't have enough
|
|
VRAM to load the model it will crash the application.
|
|
|
|
[IMPORT AI DUNGEON GAMES]
|
|
|
|
To import your games from AI Dungeon, first grab CuriousNekomimi's AI Dungeon Content Archive Toolkit:
|
|
https://github.com/CuriousNekomimi/AIDCAT
|
|
Follow the video instructions for getting your access_token, and run aidcat.py in command prompt.
|
|
Choose option [1] Download your saved content.
|
|
Choose option [2] Download your adventures.
|
|
Save the JSON file to your computer using the prompt.
|
|
Run KoboldAI, and after connecting to the web GUI, press the Import button at the top.
|
|
Navigate to the JSON file exported from AIDCAT and select it. A prompt will appear in the GUI
|
|
presenting you with all Adventures scraped from your AI Dungeon account.
|
|
Select an Adventure and click the Accept button.
|
|
|
|
[HOST GPT-NEO ON GOOGLE COLAB]
|
|
|
|
If your computer does not have an 8GB GPU to run GPT-Neo locally, you can now run a Google Colab
|
|
notebook hosting a GPT-Neo-2.7B model remotely and connect to it using the KoboldAI client.
|
|
See the instructions on the Colab at the link below:
|
|
https://colab.research.google.com/drive/1uGe9f4ruIQog3RLxfUsoThakvLpHjIkX?usp=sharing
|
|
|
|
[FOR INFERKIT INTEGRATION]
|
|
|
|
If you would like to use InferKit's Megatron-11b model, sign up for a free account on their website.
|
|
https://inferkit.com/
|
|
After verifying your email address, sign in and click on your profile picture in the top right.
|
|
In the drop down menu, click "API Key".
|
|
On the API Key page, click "Reveal API Key" and copy it. When starting KoboldAI and selecting the
|
|
InferKit API model, you will be asked to paste your API key into the terminal. After entering,
|
|
the API key will be stored in the client.settings file for future use.
|
|
You can see your remaining budget for generated characters on their website under "Billing & Usage". |