python制作图片数据集 h5py,使用python中的h5py调整和保存.h5格式的数据集的大小

I am trying to resize dataset and store new values using h5py package in python. My dataset size keeps increasing at every time instance, and I would like to append the .h5 file using the resize function. However, I run into errors using my approach. The variable dset is an array of datasets.

import os

import h5py

import numpy as np

path = './out.h5'

os.remove(path)

def create_h5py(path):

with h5py.File(path, "a") as hf:

grp = hf.create_group('left')

dset = []

dset.append(grp.create_dataset('voltage', (10**4,3), maxshape=(None,3), dtype='f', chunks=(10**4,3)))

dset.append(grp.create_dataset('current', (10**4,3), maxshape=(None,3), dtype='f', chunks=(10**4,3)))

return dset

if __name__ == '__main__':

dset = create_h5py(path)

for i in range(3):

if i == 0:

dset[0][:] = np.random.random(dset[0].shape)

dset[1][:] = np.random.random(dset[1].shape)

else:

dset[0].resize(dset[0].shape[0]+10**4, axis=0)

dset[0][-10**4:] = np.random.random((10**4,3))

dset[1].resize(dset[1].shape[0]+10**4, axis=0)

dset[1][-10**4:] = np.random.random((10**4,3))

EDIT

Thanks to tel I was able to solve this. Replace with h5py.File(path, "a") as hf: with hf = h5py.File(path, "a").

解决方案

@tel provided an elegant solution to the problem. I outlined a simpler approach in my comments below his answer. It is simpler for a beginner to code (and understand). Basically, it there a few minor changes to @Maxtron's original code. Modifications are:

move with h5py.File(path, "a") as hf: to __main__ routine

pass hf in create_h5py(hf)

I also added a test before os.remove() to avoid errors if the h5 file

doesn't exist

My suggested modifications below:

import h5py, os

import numpy as np

path = './out.h5'

# test existence of H5 file before deleting

if os.path.isfile(path):

os.remove(path)

def create_h5py(hf):

grp = hf.create_group('left')

dset = []

dset.append(grp.create_dataset('voltage', (10**4,3), maxshape=(None,3), dtype='f', chunks=(10**4,3)))

dset.append(grp.create_dataset('current', (10**4,3), maxshape=(None,3), dtype='f', chunks=(10**4,3)))

return dset

if __name__ == '__main__':

with h5py.File(path, "a") as hf:

dset = create_h5py(hf)

for i in range(3):

if i == 0:

dset[0][:] = np.random.random(dset[0].shape)

dset[1][:] = np.random.random(dset[1].shape)

else:

dset[0].resize(dset[0].shape[0]+10**4, axis=0)

dset[0][-10**4:] = np.random.random((10**4,3))

dset[1].resize(dset[1].shape[0]+10**4, axis=0)

dset[1][-10**4:] = np.random.random((10**4,3))