mirror of
https://github.com/KoboldAI/KoboldAI-Client.git
synced 2025-06-05 21:59:24 +02:00
Typical sampling needs to use nansum instead of sum
If `probs` is zero then `log_probs` will be negative infinity, and the calculation of `neg_entropy` would then give NaN because zero times infinity is a mathematically indeterminate value. We need to use nansum so that those NaN values are treated as zeros to ignore them in the entropy calculation.
This commit is contained in:
@@ -123,7 +123,7 @@ class TypicalLogitsWarper(LogitsWarper):
|
||||
|
||||
# Compute the negative of entropy, which is the sum of p*ln(p) for all p
|
||||
# in the set of softmax probabilities of the logits
|
||||
neg_entropy = (probs * log_probs).sum(dim=-1, keepdim=True)
|
||||
neg_entropy = (probs * log_probs).nansum(dim=-1, keepdim=True)
|
||||
|
||||
# Determine absolute difference between the negative entropy and the
|
||||
# log probabilities
|
||||
|
Reference in New Issue
Block a user