rixty/KoboldAI-Client
1
1
Fork 0
mirror of https://github.com/KoboldAI/KoboldAI-Client.git synced 2025-06-05 21:59:24 +02:00
Code Issues Projects Releases Wiki Activity

Add API for generated tokens and output text

Browse Source
This commit is contained in:
Gnome Ann
2021-12-12 19:27:20 -05:00
parent ceabd2ef7b
commit fbf3e7615b
3 changed files with 261 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
tpu_mtj_backend.py
View File

@ -305,7 +305,7 @@ def infer(
soft_embeddings=soft_embeddings,
)[0]
for o in output:
samples.append(tokenizer.decode(o[0][0, 0, params["seq"] : params["seq"] + gen_len]))
samples.append(o[0][0, 0, params["seq"] : params["seq"] + gen_len])
return samples