class MaskedAccuracy[source]

MaskedAccuracy(ignore:int=-100, dim:int=-1) :: AvgMetric

Computes accuracy skipping values where targ == ignore

pred = torch.tensor([[ 0, 0, 0, 0, 0, 0, 0, 0, 1, 0],
                     [ 0, 0, 0, 0, 0, 1, 0, 1, 0, 0],
                     [ 1, 1, 1, 1, 1, 0, 1, 0, 0, 1]]).t()
targ = torch.tensor([[-1,-1,-1,-1,-1, 1, 2, 1, 0, 0]])
acc = MaskedAccuracy(-1)
assert acc.func(pred[None], targ) == 0.8

class BPC[source]

BPC() :: AvgLoss

Bit per character for Language Models

class BPC(AvgLoss):
    "Bit per character for Language Models"
    @property
    def value(self): return self.total/self.count/math.log(2) if self.count != 0 else None
    @property
    def name(self):  return "bpc"
    
bpc = BPC()

Bits-Per-Character was used as a metric for the language modelling tasks, where a character-level tokenizer was used. Calculated as CrossEntropy/log(2)