Kaggle运行报错排查
今天跑深度学习实验代码刚接触到Kaggle这个平台，发现意外的好用，比起Colab来说还具备自带数据集的功能，我会在以后慢慢发掘Kaggle的。
不过今天初跑实验的时候报了一个小错，一开始我百思不得其解，后来自己依据报错位置检查了一下运行选项才发觉问题所在。
报错内容如下：
Time
#
Log Message
37.1s 1 /opt/conda/lib/python3.10/site-packages/torch/utils/data/dataloader.py:561: UserWarning: This DataLoader will create 4 worker processes in total. Our suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
37.1s 2 warnings.warn(_create_warning_msg(
37.1s 3 /opt/conda/lib/python3.10/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
37.1s 4 warnings.warn(
37.1s 5 /opt/conda/lib/python3.10/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=DenseNet201_Weights.IMAGENET1K_V1`. You can also use `weights=DenseNet201_Weights.DEFAULT` to get the most up-to-date weights.
37.1s 6 warnings.warn(msg)
37.5s 7 Downloading: "https://download.pytorch.org/models/densenet201-c1103571.pth" to /root/.cache/torch/hub/checkpoints/densenet201-c1103571.pth
61.3s 8 Traceback (most recent call last):
61.3s 9 File "", line 1, in
61.3s 10 File "/opt/conda/lib/python3.10/site-packages/papermill/execute.py", line 128, in execute_notebook
61.3s 11 raise_for_execution_errors(nb, output_path)
61.3s 12 File "/opt/conda/lib/python3.10/site-packages/papermill/execute.py", line 232, in raise_for_execution_errors
61.3s 13 raise error
61.3s 14 papermill.exceptions.PapermillExecutionError:
61.3s 15 ---------------------------------------------------------------------------
61.3s 16 Exception encountered at "In [1]":
61.3s 17 ---------------------------------------------------------------------------
61.3s 18 gaierror Traceback (most recent call last)
61.3s 19 File /opt/conda/lib/python3.10/urllib/request.py:1348, in AbstractHTTPHandler.do_open(self, http_class, req, **http_conn_args)
61.3s 20 1347 try:
61.3s 21 -> 1348 h.request(req.get_method(), req.selector, req.data, headers,
61.3s 22 1349 encode_chunked=req.has_header('Transfer-encoding'))
61.3s 23 1350 except OSError as err: # timeout error
61.3s 24
61.3s 25 File /opt/conda/lib/python3.10/http/client.py:1282, in HTTPConnection.request(self, method, url, body, headers, encode_chunked)
61.3s 26 1281 """Send a complete request to the server."""
61.3s 27 -> 1282 self._send_request(method, url, body, headers, encode_chunked)
61.3s 28
61.3s 29 File /opt/conda/lib/python3.10/http/client.py:1328, in HTTPConnection._send_request(self, method, url, body, headers, encode_chunked)
61.3s 30 1327 body = _encode(body, 'body')
61.3s 31 -> 1328 self.endheaders(body, encode_chunked=encode_chunked)
61.3s 32
61.3s 33 File /opt/conda/lib/python3.10/http/client.py:1277, in HTTPConnection.endheaders(self, message_body, encode_chunked)
61.3s 34 1276 raise CannotSendHeader()
61.3s 35 -> 1277 self._send_output(message_body, encode_chunked=encode_chunked)
61.3s 36
61.3s 37 File /opt/conda/lib/python3.10/http/client.py:1037, in HTTPConnection._send_output(self, message_body, encode_chunked)
61.3s 38 1036 del self._buffer[:]
61.3s 39 -> 1037 self.send(msg)
61.3s 40 1039 if message_body is not None:
61.3s 41 1040
61.3s 42 1041 # create a consistent interface to message_body
61.3s 43
61.3s 44 File /opt/conda/lib/python3.10/http/client.py:975, in HTTPConnection.send(self, data)
61.3s 45 974 if self.auto_open:
61.3s 46 --> 975 self.connect()
61.3s 47 976 else:
61.3s 48
61.3s 49 File /opt/conda/lib/python3.10/http/client.py:1447, in HTTPSConnection.connect(self)
61.3s 50 1445 "Connect to a host on a given (SSL) port."
61.3s 51 -> 1447 super().connect()
61.3s 52 1449 if self._tunnel_host:
61.3s 53
61.3s 54 File /opt/conda/lib/python3.10/http/client.py:941, in HTTPConnection.connect(self)
61.3s 55 940 sys.audit("http.client.connect", self, self.host, self.port)
61.3s 56 --> 941 self.sock = self._create_connection(
61.3s 57 942 (self.host,self.port), self.timeout, self.source_address)
61.3s 58 943 # Might fail in OSs that don't implement TCP_NODELAY
61.3s 59
61.3s 60 File /opt/conda/lib/python3.10/socket.py:824, in create_connection(address, timeout, source_address)
61.3s 61 823 err = None
61.3s 62 --> 824 for res in getaddrinfo(host, port, 0, SOCK_STREAM):
61.3s 63 825 af, socktype, proto, canonname, sa = res
61.3s 64
61.3s 65 File /opt/conda/lib/python3.10/socket.py:955, in getaddrinfo(host, port, family, type, proto, flags)
61.3s 66 954 addrlist = []
61.3s 67 --> 955 for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
61.3s 68 956 af, socktype, proto, canonname, sa = res
61.3s 69
61.3s 70 gaierror: [Errno -3] Temporary failure in name resolution
61.3s 71
61.3s 72 During handling of the above exception, another exception occurred:
61.3s 73
61.3s 74 URLError Traceback (most recent call last)
61.3s 75 Cell In[1], line 103
61.3s 76 100 out = self.fc(out_flat)
61.3s 77 101 return out
61.3s 78 --> 103 net = DenseNet().to(device)
61.3s 79 105 # 定义损失函数和优化器
61.3s 80 106 criterion = nn.CrossEntropyLoss(ignore_index=256)
61.3s 81
61.3s 82 Cell In[1], line 91, in DenseNet.__init__(self)
61.3s 83 89 def __init__(self):
61.3s 84 90 super(DenseNet, self).__init__()
61.3s 85 ---> 91 self.densenet = torchvision.models.densenet201(pretrained=True)
61.3s 86 92 self.fc = nn.Linear(1920, 257)
61.3s 87
61.3s 88 File /opt/conda/lib/python3.10/site-packages/torchvision/models/_utils.py:142, in kwonly_to_pos_or_kw..wrapper(*args, **kwargs)
61.3s 89 135 warnings.warn(
61.3s 90 136 f"Using {sequence_to_str(tuple(keyword_only_kwargs.keys()), separate_last='and ')} as positional "
61.3s 91 137 f"parameter(s) is deprecated since 0.13 and may be removed in the future. Please use keyword parameter(s) "
61.3s 92 138 f"instead."
61.3s 93 139 )
61.3s 94 140 kwargs.update(keyword_only_kwargs)
61.3s 95 --> 142 return fn(*args, **kwargs)
61.3s 96
61.3s 97 File /opt/conda/lib/python3.10/site-packages/torchvision/models/_utils.py:228, in handle_legacy_interface..outer_wrapper..inner_wrapper(*args, **kwargs)
61.3s 98 225 del kwargs[pretrained_param]
61.3s 99 226 kwargs[weights_param] = default_weights_arg
61.3s 100 --> 228 return builder(*args, **kwargs)
61.3s 101
61.3s 102 File /opt/conda/lib/python3.10/site-packages/torchvision/models/densenet.py:448, in densenet201(weights, progress, **kwargs)
61.3s 103 428 r"""Densenet-201 model from
61.3s 104 429 `Densely Connected Convolutional Networks `_.
61.3s 105 430
61.3s 106 (...)
61.3s 107 444 :members:
61.3s 108 445 """
61.3s 109 446 weights = DenseNet201_Weights.verify(weights)
61.3s 110 --> 448 return _densenet(32, (6, 12, 48, 32), 64, weights, progress, **kwargs)
61.3s 111
61.3s 112 File /opt/conda/lib/python3.10/site-packages/torchvision/models/densenet.py:254, in _densenet(growth_rate, block_config, num_init_features, weights, progress, **kwargs)
61.3s 113 251 model = DenseNet(growth_rate, block_config, num_init_features, **kwargs)
61.3s 114 253 if weights is not None:
61.3s 115 --> 254 _load_state_dict(model=model, weights=weights, progress=progress)
61.3s 116 256 return model
61.3s 117
61.3s 118 File /opt/conda/lib/python3.10/site-packages/torchvision/models/densenet.py:230, in _load_state_dict(model, weights, progress)
61.3s 119 221 def _load_state_dict(model: nn.Module, weights: WeightsEnum, progress: bool) -> None:
61.3s 120 222 # '.'s are no longer allowed in module names, but previous _DenseLayer
61.3s 121 223 # has keys 'norm.1', 'relu.1', 'conv.1', 'norm.2', 'relu.2', 'conv.2'.
61.3s 122 224 # They are also in the checkpoints in model_urls. This pattern is used
61.3s 123 225 # to find such keys.
61.3s 124 226 pattern = re.compile(
61.3s 125 227 r"^(.*denselayer\d+\.(?:norm|relu|conv))\.((?:[12])\.(?:weight|bias|running_mean|running_var))$"
61.3s 126 228 )
61.3s 127 --> 230 state_dict = weights.get_state_dict(progress=progress)
61.3s 128 231 for key in list(state_dict.keys()):
61.3s 129 232 res = pattern.match(key)
61.3s 130
61.3s 131 File /opt/conda/lib/python3.10/site-packages/torchvision/models/_api.py:89, in WeightsEnum.get_state_dict(self, progress)
61.3s 132 88 def get_state_dict(self, progress: bool) -> Mapping[str, Any]:
61.3s 133 ---> 89 return load_state_dict_from_url(self.url, progress=progress)
61.3s 134
61.3s 135 File /opt/conda/lib/python3.10/site-packages/torch/hub.py:746, in load_state_dict_from_url(url, model_dir, map_location, progress, check_hash, file_name)
61.3s 136 744 r = HASH_REGEX.search(filename) # r is Optional[Match[str]]
61.3s 137 745 hash_prefix = r.group(1) if r else None
61.3s 138 --> 746 download_url_to_file(url, cached_file, hash_prefix, progress=progress)
61.3s 139 748 if _is_legacy_zip_format(cached_file):
61.3s 140 749 return _legacy_zip_load(cached_file, model_dir, map_location)
61.3s 141
61.3s 142 File /opt/conda/lib/python3.10/site-packages/torch/hub.py:611, in download_url_to_file(url, dst, hash_prefix, progress)
61.3s 143 609 file_size = None
61.3s 144 610 req = Request(url, headers={"User-Agent": "torch.hub"})
61.3s 145 --> 611 u = urlopen(req)
61.3s 146 612 meta = u.info()
61.3s 147 613 if hasattr(meta, 'getheaders'):
61.3s 148
61.3s 149 File /opt/conda/lib/python3.10/urllib/request.py:216, in urlopen(url, data, timeout, cafile, capath, cadefault, context)
61.3s 150 214 else:
61.3s 151 215 opener = _opener
61.3s 152 --> 216 return opener.open(url, data, timeout)
61.3s 153
61.3s 154 File /opt/conda/lib/python3.10/urllib/request.py:519, in OpenerDirector.open(self, fullurl, data, timeout)
61.3s 155 516 req = meth(req)
61.3s 156 518 sys.audit('urllib.Request', req.full_url, req.data, req.headers, req.get_method())
61.3s 157 --> 519 response = self._open(req, data)
61.3s 158 521 # post-process response
61.3s 159 522 meth_name = protocol+"_response"
61.3s 160
61.3s 161 File /opt/conda/lib/python3.10/urllib/request.py:536, in OpenerDirector._open(self, req, data)
61.3s 162 533 return result
61.3s 163 535 protocol = req.type
61.3s 164 --> 536 result = self._call_chain(self.handle_open, protocol, protocol +
61.3s 165 537 '_open', req)
61.3s 166 538 if result:
61.3s 167 539 return result
61.3s 168
61.3s 169 File /opt/conda/lib/python3.10/urllib/request.py:496, in OpenerDirector._call_chain(self, chain, kind, meth_name, *args)
61.3s 170 494 for handler in handlers:
61.3s 171 495 func = getattr(handler, meth_name)
61.3s 172 --> 496 result = func(*args)
61.3s 173 497 if result is not None:
61.3s 174 498 return result
61.3s 175
61.3s 176 File /opt/conda/lib/python3.10/urllib/request.py:1391, in HTTPSHandler.https_open(self, req)
61.3s 177 1390 def https_open(self, req):
61.3s 178 -> 1391 return self.do_open(http.client.HTTPSConnection, req,
61.3s 179 1392 context=self._context, check_hostname=self._check_hostname)
61.3s 180
61.3s 181 File /opt/conda/lib/python3.10/urllib/request.py:1351, in AbstractHTTPHandler.do_open(self, http_class, req, **http_conn_args)
61.3s 182 1348 h.request(req.get_method(), req.selector, req.data, headers,
61.3s 183 1349 encode_chunked=req.has_header('Transfer-encoding'))
61.3s 184 1350 except OSError as err: # timeout error
61.3s 185 -> 1351 raise URLError(err)
61.3s 186 1352 r = h.getresponse()
61.3s 187 1353 except:
61.3s 188
61.3s 189 URLError:
61.3s 190
63.2s 191 /opt/conda/lib/python3.10/site-packages/traitlets/traitlets.py:2930: FutureWarning: --Exporter.preprocessors=["remove_papermill_header.RemovePapermillHeader"] for containers is deprecated in traitlets 5.0. You can pass `--Exporter.preprocessors item` ... multiple times to add items to a list.
63.2s 192 warn(
63.2s 193 [NbConvertApp] WARNING | Config option `kernel_spec_manager_class` not recognized by `NbConvertApp`.
63.3s 194 [NbConvertApp] Converting notebook __notebook__.ipynb to notebook
63.6s 195 [NbConvertApp] Writing 38712 bytes to __notebook__.ipynb
65.2s 196 /opt/conda/lib/python3.10/site-packages/traitlets/traitlets.py:2930: FutureWarning: --Exporter.preprocessors=["nbconvert.preprocessors.ExtractOutputPreprocessor"] for containers is deprecated in traitlets 5.0. You can pass `--Exporter.preprocessors item` ... multiple times to add items to a list.
65.2s 197 warn(
65.2s 198 [NbConvertApp] WARNING | Config option `kernel_spec_manager_class` not recognized by `NbConvertApp`.
65.2s 199 [NbConvertApp] Converting notebook __notebook__.ipynb to html
66.2s 200 [NbConvertApp] Writing 349898 bytes to __results__.html
可以看到是下载网上的模型时报错。
检查Setting，发现Internet选项没有勾选，如果不勾选的话应该是离线运行。勾选之后成功运行，问题解决了
