Computational Mirrors: Blind Inverse Light Transport by Deep Matrix Factorization 代码运行记录

Computational Mirrors: Blind Inverse Light Transport by Deep Matrix Factorization

Miika Aittala, Prafull Sharma, Lukas Murmann, Adam Yedidia, Gregory Wornell, William T. Freeman, and Frédo Durand

 

论文Github地址：https://github.com/prafull7/compmirrors

论文原文：https://arxiv.org/abs/1912.02314

论文Project: http://compmirrors.csail.mit.edu/

 

Table of Contents

矩阵分解部分(factorization_1d.py)

Error1 AttributeError: module 'scipy.misc' has no attribute 'imresize'

重建部分

数据集

参数设置

Error2 ImportError: cannot import name 'imsave'

Error3 ConnectionError: HTTPConnectionPool(host='localhost', port=8097): Max retries exceeded with url

Error4 IndexError: list index out of range  

总结

---

 

 

矩阵分解部分(factorization_1d.py)

Error1 AttributeError: module 'scipy.misc' has no attribute 'imresize'

Traceback (most recent call last):
  File "factorization_1d.py", line 74, in
    T0 = scipy.misc.imresize(T0, [h, q]).astype(float) / 255
AttributeError: module 'scipy.misc' has no attribute 'imresize'

解决方案：https://blog.csdn.net/WUDIxi/article/details/100060055

更改代码：

# T0 = scipy.misc.imresize(T0, [h, q]).astype(float) / 255
# L0 = scipy.misc.imresize(L0, [q, w]).astype(float) / 255
T0 = resize(T0, [h, q]).astype(float) / 255
L0 = resize(L0, [q, w]).astype(float) / 255

即可

 

至此，single picture的矩阵分解部分代码调通。明天再调Reconstruction部分的代码。

重建部分

数据集

Reconstruction部分的数据从http://compmirrors.csail.mit.edu/ 下载；

并按照说明处理：

Place the folder in './data/light_transport/' to be consistent with instructions on github.

 

参数设置

根据github的instruction，不妨将参数按如下进行设置：

parser.add_argument('-d', '--data_dir', type=str,
    default=os.environ.get('FACTORIZE_DATA_DIR', './data/light_transport/'), help='Location for data directory')
parser.add_argument('-o', '--out_dir', type=str,
    default=os.environ.get('FACTORIZE_OUT_DIR', './result/'), help='Location for output directory')
parser.add_argument('-f', '--folder', type=str,
    default='dots_data', help='Name of the folder')
parser.add_argument('-ds', '--dataset', type=str,
    default='dots_sequence', help='Name of the dataset')
parser.add_argument('-s', '--seq_name', type=str,
    default='dots', help='Name of the sequence')

同时注意visdom的端口，为了防止和别人冲突改为9103

parser.add_argument('-vis_p', '--visdom_port', type=int,
    default=9103, help='Port of the visdom server')

并通过下面代码开启端口：

python -m visdom.server -port 9103

同时运行代码可改为：

python factorization_light_transport.py -dev DEVICE_NUMBER

 

Error2 ImportError: cannot import name 'imsave'

ImportError: cannot import name 'imsave'

参考：https://blog.csdn.net/chao2016/article/details/77971577

并且scipy版本不能达到1.5.0，因此通过conda降级到1.2.1：

conda install scipy==1.2.1

问题解决

 

Error3 ConnectionError: HTTPConnectionPool(host='localhost', port=8097): Max retries exceeded with url

Traceback (most recent call last):
  File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/urllib3/connection.py", line 160, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/urllib3/util/connection.py", line 84, in create_connection
    raise err
  File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/urllib3/util/connection.py", line 74, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/urllib3/connectionpool.py", line 392, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/urllib3/connection.py", line 187, in connect
    conn = self._new_conn()
  File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/urllib3/connection.py", line 172, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: : Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/urllib3/connectionpool.py", line 725, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/urllib3/util/retry.py", line 439, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=8097): Max retries exceeded with url: /env/2020-08-12-1051--dots_dots_sequence_gpu01_0 (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/visdom/__init__.py", line 484, in _send
    data=json.dumps(msg),
  File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/requests/api.py", line 119, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=8097): Max retries exceeded with url: /env/2020-08-12-1051--dots_dots_sequence_gpu01_0 (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused',))
Device ID: 0
Traceback (most recent call last):
  File "factorization_light_transport.py", line 101, in 
    frames_dict = json.load(open(os.path.join(args.data_dir, args.folder, "frames.txt"),'r'))
  File "/share2/home/ruixu/anaconda3/envs/python36/lib/python3.6/posixpath.py", line 80, in join
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

前面所说的port没有改...开的是9103，用的却是没有开的8097，故出现错误。

 

Error4 IndexError: list index out of range  

Traceback (most recent call last):
  File "factorization_light_transport.py", line 488, in 
    factorize()
  File "factorization_light_transport.py", line 147, in factorize
    Z, overexp_mask, gt_imgs = load_observations(data_dir, folder_name, dataset, seq_name, skip_frames, n_downsample=args.num_downsample)
  File "/share2/home/ruixu/DeepLearningLCT/DeblurGAN-master/compmirrors-master/compmirrors-master/data_loading.py", line 117, in load_observations
    blacks, blacks_oe = load_pgm(os.path.join(data_dir, folder_name, dataset + "_reflector"), frames_dict['black'][0], frames_dict['black'][1], n_downsample)
  File "/share2/home/ruixu/DeepLearningLCT/DeblurGAN-master/compmirrors-master/compmirrors-master/data_loading.py", line 51, in load_pgm
    img = cv2.imread(files[i], cv2.IMREAD_UNCHANGED)
IndexError: list index out of range

经过调试可以发现，data_loading.py中load_pgm的files是None，即path不对，从而导致了上面的错误。

修改方法：

step1: 将data_loading.py中load_observations代码的的116-120行中的reflector后面加上/，改为：

    # Read black frames if available
    if 'black' in frames_dict:  # {'black': [0, 99], 'impulse': [100, 675], 'black2': [676, 707], 'disc': [708, 4290]}
        blacks, blacks_oe = load_pgm(os.path.join(data_dir, folder_name, dataset + "_reflector/"), frames_dict['black'][0], frames_dict['black'][1], n_downsample)

    imgs, imgs_oe = load_pgm(os.path.join(data_dir, folder_name, dataset + "_reflector/"),
                         frames_dict[seq_name][0], frames_dict[seq_name][1], n_downsample)

读取ft图片时同理：

    gt_imgs = load_gt(os.path.join(data_dir, folder_name, dataset + "_target/"), frames_dict[seq_name][0], frames_dict[seq_name][1], ext="")

step2: 将文件结构改为：

其中，第一个reflector文件夹中是文件名包含reflector的图像(即相机observe到的图片)；第二个文件夹则是gt.

 

这里有一个问题，即读取gt时后缀为什么是""？这是因为708正好分割开了target_img的两个部分。

=================================================================================

至此，程序可以正常运行：

 

总结

遇到的问题基本都是因为版本问题 或 数据集的文件目录问题导致的。