A rather embarassing way to spend an hour debugging after I told myself
"I'd better remember to add this important thing to the torch side".
Samplers were being applied when in their "off values" causing
boring mathmatical operations to take place (ie anything x 0 is always
0)