rixty
/
KoboldAI-Client
mirror of https://github.com/KoboldAI/KoboldAI-Client.git
1
1
Fork 0
Code Issues Projects Releases Wiki Activity

Remove device=0 because of incompatibility 

Device=0 breaks some of the pytorch implementations, removed to restore hardware compatibility to 0.16 levels.
This commit is contained in:
henk717 2021-11-14 02:33:27 +01:00 committed by GitHub
parent 4bcffc614e
commit 8a916116e3
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
aiserver.py
View File

@ -638,7 +638,7 @@ if(not vars.model in ["InferKit", "Colab", "OAI", "ReadOnly"]):
tokenizer = GPT2Tokenizer.from_pretrained(vars.model)
if(vars.hascuda):
if(vars.usegpu):
model = AutoModelForCausalLM.from_pretrained(vars.model, device=0)
model = AutoModelForCausalLM.from_pretrained(vars.model)
vars.modeldim = get_hidden_size_from_model(model)
model = model.to(0)
generator = model.generate