Multigen

SillyTavern tries to create faster and longer responses by chaining the generation using smaller batches.

Default settings:

First batch = 50 tokens

Next batches = 30 tokens

Algorithm:

1. Generate the first batch (if amount of generation setting is more than batch length).

2. Generate next batch of tokens until one of the stopping conditions is reached.

3. Append the generated text to the next cycle's prompt.

Stopping conditions:

1. Generated enough text.

2. Character starts speaking for You.

3. <|endoftext|> token reached.

4. No text generated.