rixty/KoboldAI-Client
1
1
Fork 0
mirror of https://github.com/KoboldAI/KoboldAI-Client.git synced 2025-03-02 10:27:44 +01:00
Code Issues Projects Releases Wiki Activity

Merge pull request #180 from VE-FORBRYDERNE/patch

Browse Source 
Only enable TPU transpose optimization if loading from HF model
This commit is contained in:
henk717 2022-11-21 20:02:14 +01:00 committed by GitHub
commit 04d9172fcd
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
tpu_mtj_backend.py
View File

@ -1149,7 +1149,8 @@ def load_model(path: str, driver_version="tpu_driver0.1_dev20210607", hf_checkpo
params[param] = default_params[param]
# Use an optimization that will allow us to avoid one extra transpose operation
params["transposed_linear"] = True
if hf_checkpoint:
params["transposed_linear"] = True
# Load tokenizer
if vars.model == "TPUMeshTransformerGPTNeoX":