huggingface之datasets将数据集下载到本地

下载数据集使用代理会下载快一点.
但是我想放在本地,下载一次不用下载了.虽然他自己可以缓存,但是感觉每次还是重新下载,好像有bug似的.

参考:[https://blog.csdn.net/PolarisRisingWar/article/details/124042709#:~:text=%E5%9C%A8%E6%9C%AC%E5%9C%B0%E5%8A%A0%E8%BD%BD%E6%95%B0%E6%8D%AE%E9%9B%86%E5%B9%B6%E5%82%A8%E5%AD%98%E5%88%B0%E6%9C%AC%E5%9C%B0%E7%A3%81%E7%9B%98%EF%BC%88%E6%B3%A8%E6%84%8F%E8%BF%99%E4%B8%AA%E8%B7%AF%E5%BE%84%E6%98%AFWindows%E7%B3%BB%E7%BB%9F%E7%9A%84%E8%B7%AF%E5%BE%84%EF%BC%89%EF%BC%9A%20import%20datasets,dataset%3Ddatasets.load_dataset%28%22yelp_review_full%22%2Ccache_dir%3D%27mypathdata%5B%26hu%26%5Dggingfacedatasetscache%27%29%20dataset.save_to_disk%28%27mypath%5Cdata%5Cyelp_review_full_disk%27%29%201]

#数据集下载到硬盘disk
from datasets import load_dataset
dataset = load_dataset("neulab/conala", cache_dir="./dataset")
dataset.save_to_disk('dataset/neulab/conala')

#加载数据集.
import datasets
dataset = datasets.load_from_disk("dataset/neulab/conala")