somebody 91bb433b5f GenericTokenizer: Fall back to defined tokenizer ...

Shouldn't be relied on for model-agnostic code, but for loading
processes where you know the tokenizer class used it should be okie
dokie

2023-03-19 19:03:20 -05:00

..

inference_models

GenericTokenizer: Fall back to defined tokenizer

2023-03-19 19:03:20 -05:00

inference_model.py

Modeling: Fix logits processors (probs, biasing, lua)

2023-03-17 16:56:47 -05:00

logits_processors.py

Modeling: Fix logits processors (probs, biasing, lua)

2023-03-17 16:56:47 -05:00

patches.py

Model: And another refactor

2023-03-01 19:16:35 -06:00

post_token_hooks.py

Model: Respect model lazyload over kaivars

2023-03-10 20:00:39 -06:00

stoppers.py

Model: Reformat and clean up

2023-03-10 20:00:39 -06:00

test_generation.py

Model: More Jax import fixes and formatting

2023-03-17 15:36:44 -05:00

tokenizer.py

GenericTokenizer: Fall back to defined tokenizer

2023-03-19 19:03:20 -05:00

warpers.py

Model: Lazyload backends

2023-03-13 20:29:29 -05:00