rixty
/
KoboldAI-Client
mirror of https://github.com/KoboldAI/KoboldAI-Client.git
1
1
Fork 0
Code Issues Projects Releases Wiki Activity

move_model_to_devices no longer crashes if you don't have accelerate

This commit is contained in:
Gnome Ann 2022-06-21 13:15:46 -04:00
parent f2c5bb5cb7
commit 6b172306f6
1 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
aiserver.py
View File

@ -633,8 +633,9 @@ def move_model_to_devices(model):
generator = model.generate generator = model.generate
return return
import breakmodel
if(utils.HAS_ACCELERATE): if(utils.HAS_ACCELERATE):
import breakmodel
disk_blocks = breakmodel.disk_blocks disk_blocks = breakmodel.disk_blocks
gpu_blocks = breakmodel.gpu_blocks gpu_blocks = breakmodel.gpu_blocks
ram_blocks = len(utils.layers_module_names) - sum(gpu_blocks) ram_blocks = len(utils.layers_module_names) - sum(gpu_blocks)