fastdebug

一个有助于改进torch和fastai错误的库

安装

pip install fastdebug

如何使用

fastdebug围绕提高处理Pytorch和fastai错误时的生活质量而设计，同时还包括一些新的健全性检查（仅限fastai）

Pytorch

Pytorch现在有

• device_error
• layer_error

两者都可以通过以下方式导入

from fastdebug.error.torch import device_error, layer_error

device_error在两个张量不在同一设备上时打印出更易读的错误信息

inp = torch.rand().cuda()
model = model.cpu()
try:
    _ = model(inp)
except Exception as e:
    device_error(e, 'Input type', 'Model weights')

还有我们新的日志

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-28-981e0ace9c38> in <module>()
      2     model(x)
      3 except Exception as e:
----> 4     device_error(e, 'Input type', 'Model weights')

10 frames
/usr/local/lib/python3.7/dist-packages/torch/tensor.py in __torch_function__(cls, func, types, args, kwargs)
    993 
    994         with _C.DisableTorchFunction():
--> 995             ret = func(*args, **kwargs)
    996             return _convert(ret, cls)
    997 

RuntimeError: Mismatch between weight types

Input type has type: 		 (torch.cuda.FloatTensor)
Model weights have type: 	 (torch.FloatTensor)

Both should be the same.

使用layer_error，如果存在形状不匹配，它将尝试找到正确的层

inp = torch.rand(5,2, 3)
try:
    m(inp)
except Exception as e:
    layer_error(e, m)

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-84-d4ab91131841> in <module>()
      3     m(inp)
      4 except Exception as e:
----> 5     layer_error(e, m)

<ipython-input-83-ca2dc02cfff4> in layer_error(e, model)
      8     i, layer = get_layer_by_shape(model, shape)
      9     e.args = [f'Size mismatch between input tensors and what the model expects\n\n{args}\n\tat layer {i}: {layer}']
---> 10     raise e

<ipython-input-84-d4ab91131841> in <module>()
      1 inp = torch.rand(5,2, 3)
      2 try:
----> 3     m(inp)
      4 except Exception as e:
      5     layer_error(e, m)

/mnt/d/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    725             result = self._slow_forward(*input, **kwargs)
    726         else:
--> 727             result = self.forward(*input, **kwargs)
    728         for hook in itertools.chain(
    729                 _global_forward_hooks.values(),

/mnt/d/lib/python3.7/site-packages/torch/nn/modules/container.py in forward(self, input)
    115     def forward(self, input):
    116         for module in self:
--> 117             input = module(input)
    118         return input
    119 

/mnt/d/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    725             result = self._slow_forward(*input, **kwargs)
    726         else:
--> 727             result = self.forward(*input, **kwargs)
    728         for hook in itertools.chain(
    729                 _global_forward_hooks.values(),

/mnt/d/lib/python3.7/site-packages/torch/nn/modules/conv.py in forward(self, input)
    421 
    422     def forward(self, input: Tensor) -> Tensor:
--> 423         return self._conv_forward(input, self.weight)
    424 
    425 class Conv3d(_ConvNd):

/mnt/d/lib/python3.7/site-packages/torch/nn/modules/conv.py in _conv_forward(self, input, weight)
    418                             _pair(0), self.dilation, self.groups)
    419         return F.conv2d(input, weight, self.bias, self.stride,
--> 420                         self.padding, self.dilation, self.groups)
    421 
    422     def forward(self, input: Tensor) -> Tensor:

RuntimeError: Size mismatch between input tensors and what the model expects

Model expected 4-dimensional input for 4-dimensional weight [3, 3, 1, 1], but got 3-dimensional input of size [5, 2, 3] instead
	at layer 1: Conv2d(3, 3, kernel_size=(1, 1), stride=(1, 1))

fastai

除了上述添加（并在fit期间使用）之外，fastai现在还有一个Learner.sanity_check函数，它允许您快速执行基本检查以确保您的fit调用不会引发任何异常。它们在CPU上进行部分时代检查，以确保可以预防性地找到CUDA设备辅助错误。

要使用它，只需这样做

from fastdebug.fastai import *
from fastai.vision.all import *

learn = Learner(...)
learn.sanity_check()

这现在是Learner的一个参数，默认设置为False，以确保在创建Learner后进行快速检查。

learn = Learner(..., sanity_check=True)