Enable Tokenizer Parralism

Has proven to be safe in my internal testing and does help with the interface lag at boot. Enabling this so it can get wider testing.
2022-02-01 12:00:53 +01:00 · 2022-02-01 12:00:53 +01:00 · b8e08cdd63
parent ecd7b328ec
commit b8e08cdd63
1 changed files with 1 additions and 1 deletions
--- a/aiserver.py
+++ b/aiserver.py
@ -11,7 +11,7 @@ eventlet.monkey_patch(all=True, thread=False)
 import os
 os.system("")
 os.environ['EVENTLET_THREADPOOL_SIZE'] = '1'
-os.environ['TOKENIZERS_PARALLELISM'] = 'false'
+os.environ['TOKENIZERS_PARALLELISM'] = 'true'
 from eventlet import tpool

 from os import path, getcwd