Fix soft prompt length calculation in `calcsubmitbudget()`
In TPU instances, `vars.sp.shape[0]` is not always the actual number of tokens in the soft prompt. We have to use `vars.sp_length` to get an accurate token count.
This commit is contained in:
parent
74f79081d1
commit
9594b2db1c
|
@ -2494,7 +2494,7 @@ def calcsubmitbudget(actionlen, winfo, mem, anotetxt, actions, submission=None,
|
|||
anotetkns = [] # Placeholder for Author's Note tokens
|
||||
lnanote = 0 # Placeholder for Author's Note length
|
||||
|
||||
lnsp = vars.sp.shape[0] if vars.sp is not None else 0
|
||||
lnsp = vars.sp_length
|
||||
|
||||
if("tokenizer" not in globals()):
|
||||
from transformers import GPT2TokenizerFast
|
||||
|
|
Loading…
Reference in New Issue