Use GPT-4o tokenizer for Gemini

Use GPT-4o tokenizer for Gemini, as Gemini tokenizer is more similar to GPT-4o's
This commit is contained in:
NijikaMyWaifu
2024-05-25 13:26:11 +08:00
committed by GitHub
parent 439ef0dc5e
commit 33b22bd4f8

View File

@ -308,6 +308,10 @@ function getTokenizerModel(requestModel) {
return 'yi';
}
if (requestModel.includes('gemini')) {
return 'gpt-4o';
}
// default
return 'gpt-3.5-turbo';
}