rixty/KoboldAI-Client
1
1
Fork 0
mirror of https://github.com/KoboldAI/KoboldAI-Client.git synced 2025-06-05 21:59:24 +02:00
Code Issues Projects Releases Wiki Activity

Remove legacy no accelerate fallback code

Browse Source 
Was causing issues with disk cache the old code had a
`and not utils.HAS_ACCELERATE` preceding it (a variable which no longer
exists), and since disk cache is accelerate only, there was no disk
handling code in here. Anyway its bad so blast it
This commit is contained in:
somebody
2023-04-15 10:47:31 -05:00
parent b2e6fcfe3a
commit 2b950f08d3
1 changed files with 0 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
modeling/inference_models/hf_torch.py
View File

@@ -302,13 +302,6 @@ class HFTorchInferenceModel(HFInferenceModel):
return True
def _move_to_devices(self) -> None:
if not utils.koboldai_vars.breakmodel:
if utils.koboldai_vars.usegpu:
self.model = self.model.half().to(utils.koboldai_vars.gpu_device)
else:
self.model = self.model.to("cpu").float()
return
for key, value in self.model.state_dict().items():
target_dtype = (
torch.float32 if breakmodel.primary_device == "cpu" else torch.float16