minio操作手册
minio
文章目录
- minio
-
- 1. 基础介绍
-
- 1.1 简介
- 1.2 基础概念
- 1.3 纠删码EC
- 1.4 存储形式
- 2. minio部署
-
-
- non-erasure code mode
- **erasure code mode**
- 2.1 用户名密码设置
- 2.2 服务启动
- 2.3 登录控制台
-
- 3. 基础操作
-
- 3.1 控制台基础操作
-
- 3.1.1 创建bucket
- 3.1.2 上传文件
- 3.1.3 文件操作
- 3.2 客户端的使用
-
- 3.2.1 服务配置
- 3.2.2 文件操作
- 3.2.3 bucket管理
- 3.2.4 http跟踪
- 3.2.5 扫描修复
- 3.2.6 文件查找
- 4. JAVA
-
- 4.1 环境准备
- 4.2 代码示例
- 4.3 API操作
-
- 4.3.1 bucket操作
-
- 4.3.1.1 bucket是否存在
- 4.3.1.2 bucket创建
- 4.3.1.3 bucket移除
- 4.3.1.4 设置存储周期
- 4.3.1.5 多版本设置
- 4.3.2 对象操作
-
- 4.3.2.1 上传对象
- **4.3.2.2 获取对象**
- 4.3.2.3 复制对象
- 4.3.2.4 删除对象
- 4.3.2.5 对象信息查询
- 4.4 Springboot集成
-
- 4.4.1 环境搭建
- 4.4.2 代码示例
-
- 4.4.2.1 配置参数
- 4.4.2.2 配置文件修改
- 4.4.2.3 MinioClient创建
- 4.4.2.4 Minio API操作
- 4.4.2.5 接口提供
- 4.4.2.6 调用示例
1. 基础介绍
1.1 简介
MinIO 是一个基于Apache License v2.0开源协议的对象存储服务。它兼容亚马逊S3云存储服务接口，非常适合于存储大容量非结构化的数据，例如图片、视频、日志文件、备份数据和容器/虚拟机镜像等，而一个对象文件可以是任意大小，从几kb到最大5T不等。
MinIO是一个非常轻量的服务,可以很简单的和其他应用的结合，类似 NodeJS, Redis 或者 MySQL。目前支持JavaScript 、Java、Python、Golang、.NET。
官网：https://min.io/ http://www.minio.org.cn/
MinIO的优点如下：
- 部署简单: 一个 single 二进制文件即是一切，还可支持各种平台。
- minio 支持海量存储，可按 zone 扩展(原 zone 不受任何影响)，支持单个对象最大 5TB;
- 低冗余且磁盘损坏高容忍，标准且最高的数据冗余系数为 2(即存储一个 1M 的数据对象，实际占用磁盘空间为 2M)。但在任意 n/2 块 disk 损坏的情况下依然可以读出数据(n 为一个纠删码集合(Erasure Coding Set)中的 disk 数量)。并且这种损坏恢复是基于单个对象的，而不是基于整个存储卷的。
- 读写性能优异
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-YjEABJWK-1671378607578)(E:\Typora\548e9d729925e15a05c21c314e8e3c79.png)]
1.2 基础概念
MinIO中涉及以下几个基础概念：
Object：存储到MinIO的基本对象，如文件、字节流等
Bucket：用来存储Object的逻辑空间。每个Bucket之间的数据是相互隔离的。对于客户端而言，就相当于一个存放文件的顶层文件夹
Drive：即存储数据的磁盘，在MinIO启动时，以参数的方式传入。MinIO中的所有对象数据都会存储在Drive里
Set：即一组Drive的集合。分布式部署根据集群规模自动划分一个或多个Set，每个Set中的Drive分布在不同位置。
- 一个对象存储在一个Set上
- 一个Object存储在一个Set上
- 一个集群划分为多个Set
- 一个Set包含的Drive数量是固定的，默认由系统根据集群规模自动计算得出
- 一个Set中的Drive尽可能分布在不同节点上
1.3 纠删码EC
纠删码（erasure coding，EC）是一种数据保护方法，它将数据分割成片段，把冗余数据块扩展、编码，并将其存储在不同的位置，比如磁盘、存储节点或者其它地理位置。
Minio采用Reed-Solomon code将对象拆分成N/2数据和N/2 奇偶校验块。 这就意味着如果是12块盘，一个对象会被分成6个数据块、6个奇偶校验块，你可以丢失任意6块盘（不管其是存放的数据块还是奇偶校验块），你仍可以从剩下的盘中的数据进行恢复。
1.4 存储形式
文件对象上传到 MinIO ，会在对应的数据存储磁盘中，以 Bucket 名称为目录，文件名称为下一级目录，文件名下是 part.1 和 xl.meta(老版本，最新版本如下图)，前者是编码数据块及检验块，后者是元数据文件。
2. minio部署
minio 支持多种 server 启动模式：
non-erasure code mode
在此启动模式下，对于每一份对象数据，minio 直接在 data 下面存储这份数据，不会建立副本，也不会启用纠删码机制。因此，这种模式无论是服务实例还是磁盘都是“单点”，无任何高可用保障，磁盘损坏就表示数据丢失。
erasure code mode
此模式为 minio server 实例传入多个本地磁盘参数。一旦遇到多于一个磁盘参数，minio server 会自动启用 erasure code mode。erasure code 对磁盘的个数是有要求的，如不满足要求，实例启动将失败。 erasure code 启用后，要求传给 minio server 的 endpoint(standalone 模式下，即本地磁盘上的目录)至少为 4 个。
2.1 用户名密码设置
set MINIO_ROOT_USER=minio //用户名
set MINIO_ROOT_PASSWORD=minio //密码
2.2 服务启动
单节点单目录部署
minio.exe server ./data
单节点多目录部署
minio.exe server ./data{1...64} [生成data1-data64 共64个文件存储数据]
多节点部署
//顺序部署需要节点名，目录名顺序化
minio.exe server http://node{1...32}.example.com/mnt/export{1...32}
//自定义ip，目录部署
minio.exe server
http://192.168.1.1/opt/minio/data1 http://192.168.1.1/opt/minio/data2 \
http://192.168.1.2/opt/minio/data1 http://192.168.1.2/opt/minio/data2 \
http://192.168.1.3/opt/minio/data1 http://192.168.1.3/opt/minio/data2 \
http://192.168.1.4/opt/minio/data1 http://192.168.1.4/opt/minio/data2
参数设置：
--address value bind to a specific ADDRESS:PORT, ADDRESS can be an IP or hostname (default: ":9000") [%MINIO_ADDRESS%]
--console-address value bind to a specific ADDRESS:PORT for embedded Console UI, ADDRESS can be an IP or hostname [%MINIO_CONSOLE_ADDRESS%]
--certs-dir value, -S value path to certs directory (default: "C:\\Users\\ZF\\.minio\\certs")
--quiet disable startup and info messages
--anonymous hide sensitive information from logging
--json output logs in JSON format
--help, -h show help
启动成功输出以下日志：
2.3 登录控制台
3. 基础操作
3.1 控制台基础操作
3.1.1 创建bucket
3.1.2 上传文件
点击右上角选择【上传文件||上传文件夹】，进行上传
3.1.3 文件操作
可进行下载，分享，预览，删除等操作
3.2 客户端的使用
MinIO提供了客户端，可以让我们以类似linux命令的方式操作MinIO
PS F:\MINIO> ./mc.exe -help
────────────────────────────────────────────────────────────
NAME:
mc - MinIO Client for object storage and filesystems.
USAGE:
mc [FLAGS] COMMAND [COMMAND FLAGS | -h] [ARGUMENTS...]
COMMANDS:
alias manage server credentials in configuration file
ls list buckets and objects
mb make a bucket
rb remove a bucket
cp copy objects
mv move objects
rm remove object(s)
mirror synchronize object(s) to a remote site
cat display object contents
head display first 'n' lines of an object
pipe stream STDIN to an object
find search for objects
sql run sql queries on objects
stat show object metadata
tree list buckets and objects in a tree format
du summarize disk usage recursively
retention set retention for object(s)
legalhold manage legal hold for object(s)
support support related commands
license license related commands
share generate URL for temporary access to an object
version manage bucket versioning
ilm manage bucket lifecycle
encrypt manage bucket encryption config
event manage object notifications
watch listen for object notification events
undo undo PUT/DELETE operations
anonymous manage anonymous access to buckets and objects
tag manage tags for bucket and object(s)
diff list differences in object name, size, and date between two buckets
replicate configure server side bucket replication
admin manage MinIO servers
update update mc to latest release
ready checks if the cluster is ready or not
ping perform liveness check
od measure single stream upload and download
batch manage batch jobs
GLOBAL FLAGS:
--autocompletion install auto-completion for your shell
--config-dir value, -C value path to configuration folder (default: "C:\\Users\\ZF\\mc")
────────────────────────────────────────────────────────────
MinIO默认配置了以下服务列表
PS F:\MINIO> ./mc.exe config host ls
gcs
URL : https://storage.googleapis.com
AccessKey : YOUR-ACCESS-KEY-HERE
SecretKey : YOUR-SECRET-KEY-HERE
API : S3v2
Path : dns
local
URL : http://localhost:9000
AccessKey :
SecretKey :
API :
Path : auto
play
URL : https://play.min.io
AccessKey : Q3AM3UQ867SPQQA43P2F
SecretKey : zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG
API : S3v4
Path : auto
s3
URL : https://s3.amazonaws.com
AccessKey : YOUR-ACCESS-KEY-HERE
SecretKey : YOUR-SECRET-KEY-HERE
API : S3v4
Path : dns
3.2.1 服务配置
PS F:\MINIO> ./mc.exe config -h
────────────────────────────────────────────────────────────
NAME:
mc config - configure MinIO client
USAGE:
mc config COMMAND [COMMAND FLAGS | -h] [ARGUMENTS...]
COMMANDS:
host add, remove and list hosts in configuration file
FLAGS:
--config-dir value, -C value path to configuration folder (default: "C:\\Users\\ZF\\mc")
--quiet, -q disable progress bar display
--no-color disable color theme
--json enable JSON lines formatted output
--debug enable debug output
--insecure disable SSL certificate verification
--help, -h show help
示例【增加服务】：
PS F:\MINIO> ./mc.exe config host a local http://127.0.0.1:9000 minio 123ABCdef*
Added `local` successfully.
3.2.2 文件操作
文件预览
PS F:\MINIO> ./mc.exe ls local
[2022-12-18 16:53:47 CST] 0B 202211/
[2022-12-18 16:23:48 CST] 0B 202212/
PS F:\MINIO> ./mc.exe ls local/202212
[2022-12-18 16:32:11 CST] 106KiB STANDARD erasure-code1.jpg
文件上传下载
上传
PS F:\MINIO> ./mc.exe cp .\mc.exe local/202211
F:\MINIO\mc.exe: 24.86 MiB / 24.86 MiB [==========================================================] 244.25 MiB/s 0s
下载
PS F:\MINIO> ./mc.exe cp .\mc.exe local/202211
F:\MINIO\mc.exe: 24.86 MiB / 24.86 MiB [==========================================================] 244.25 MiB/s 0s
文件删除
PS F:\MINIO> ./mc.exe cp .\mc.exe local/202211
F:\MINIO\mc.exe: 24.86 MiB / 24.86 MiB [==========================================================] 244.25 MiB/s 0s
3.2.3 bucket管理
//创建
PS F:\MINIO> .\mc.exe mb local/202210
Bucket created successfully `local/202210`.
//无文件时删除
PS F:\MINIO> .\mc.exe rb local/202210
Removed `local/202210` successfully.
//有文件时删除
PS F:\MINIO> .\mc.exe --force rb local/202210
Removed `local/202210` successfully.
3.2.4 http跟踪
PS F:\MINIO> .\mc.exe admin trace local
2022-12-18T20:51:14.994 [200 OK] s3.GetBucketLocation 127.0.0.1:9000/202212/?location= 127.0.0.1 37µs ↑ 98 B ↓ 128 B
2022-12-18T20:51:14.994 [200 OK] s3.ListObjectsV2 127.0.0.1:9000/202212/?delimiter=%2F&encoding-type=url&fetch-owner=true&list-type=2&prefix= 127.0.0.1 1.134ms ↑ 98 B ↓ 626 B
2022-12-18T20:51:14.997 [200 OK] s3.GetBucketLocation 127.0.0.1:9000/202212/?location= 127.0.0.1 0s ↑ 98 B ↓ 128 B
2022-12-18T20:51:14.997 [200 OK] s3.GetBucketVersioning 127.0.0.1:9000/202212/?versioning= 127.0.0.1 0s ↑ 98 B ↓ 99 B
2022-12-18T20:51:15.010 [200 OK] s3.GetBucketLocation 127.0.0.1:9000/202212/?location= 127.0.0.1 0s ↑ 98 B ↓ 128 B
2022-12-18T20:51:15.011 [404 Not Found] s3.GetBucketPolicy 127.0.0.1:9000/202212/?policy= 127.0.0.1 0s ↑ 98 B ↓ 288 B
2022-12-18T20:51:15.012 [404 Not Found] s3.GetBucketTagging 127.0.0.1:9000/202212/?tagging= 127.0.0.1 514µs ↑ 98 B ↓ 275 B
2022-12-18T20:51:15.030 [200 OK] s3.ListObjectsV2 127.0.0.1:9000/202212/?delimiter=%2F&encoding-type=url&fetch-owner=true&list-type=2&prefix=%2F 127.0.0.1 1.026ms ↑ 98 B ↓ 626 B
2022-12-18T20:51:21.205 [200 OK] s3.GetBucketLocation 127.0.0.1:9000/202212/?location= 127.0.0.1 511µs ↑ 98 B ↓ 128 B
2022-12-18T20:51:21.206 [200 OK] s3.PutObject 127.0.0.1:9000/202212/erasure-code1.jpg 127.0.0.1 32.825ms ↑ 106 KiB ↓ 0 B
2022-12-18T20:51:21.251 [200 OK] s3.ListObjectsV2 127.0.0.1:9000/202212/?delimiter=%2F&encoding-type=url&fetch-owner=true&list-type=2&prefix= 127.0.0.1 11.408ms ↑ 98 B ↓ 626 B
3.2.5 扫描修复
mc admin heal 命令用来扫描损坏的对象并修复这些对象。mc admin heal 是资源密集型的，即使在磁盘故障或损坏事件之后通常也不需要。
相反，MinIO会自动修复因bitrot损坏、磁盘故障或其他POST/GET问题而损坏的对象。MinIO 还执行周期性的后台对象修复。
语法如下：
C:\>mc admin heal -h
NAME:
mc admin heal - [DEPRECATED] heal disks, buckets and objects on MinIO server
USAGE:
mc admin heal [FLAGS] TARGET
FLAGS:
--scan value [DEPRECATED] select the healing scan mode (normal/deep) (default: "normal")
--recursive, -r [DEPRECATED] heal recursively
--dry-run, -n [DEPRECATED] only inspect data, but do not mutate
--force-start, -f [DEPRECATED] force start a new heal sequence
--force-stop, -s [DEPRECATED] force stop a running heal sequence
--remove [DEPRECATED] remove dangling objects in heal sequence
--config-dir value, -C value path to configuration folder (default: "C:\\Users\\Administrator\\mc")
--quiet, -q disable progress bar display
--no-color disable color theme
--json enable JSON lines formatted output
--debug enable debug output
--insecure disable SSL certificate verification
--help, -h show help
SCAN MODES:
normal (default): Heal objects which are missing on one or more disks.
deep : Heal objects which are missing or with silent data corruption on one or more disks.
DEPRECATED:
MinIO server now supports auto-heal, this command will be removed in future.
命令应在存储桶或存储桶前缀的完整路径上执行对象修复，将已配置的 MinIO 部署的alias指定为路径的前缀。例如：
mc admin heal play``/mybucket/myprefix
如果 TARGET 存储桶或存储桶前缀具有活动的修复扫描，则该命令将返回该扫描的状态
3.2.6 文件查找
mc find 命令用来查询对象。语法如下：
PS D:\minio> .\mc.exe find -h
NAME:
mc.exe find - search for objects
USAGE:
mc.exe find [FLAGS] TARGET
FLAGS:
--exec value spawn an external process for each matching object (see FORMAT)
--ignore value exclude objects matching the wildcard pattern
--name value find object names matching wildcard pattern
--newer-than value match all objects newer than value in duration string (e.g. 7d10h31s)
--older-than value match all objects older than value in duration string (e.g. 7d10h31s)
--path value match directory names matching wildcard pattern
--print value print in custom format to STDOUT (see FORMAT)
--regex value match directory and object name with PCRE regex pattern
--larger value match all objects larger than specified size in units (see UNITS)
--smaller value match all objects smaller than specified size in units (see UNITS)
--maxdepth value limit directory navigation to specified depth (default: 0)
--watch monitor a specified path for newly created object(s)
--config-dir value, -C value path to configuration folder (default: "C:\\Users\\admin\\mc")
--quiet, -q disable progress bar display
--no-color disable color theme
--json enable JSON lines formatted output
--debug enable debug output
--insecure disable SSL certificate verification
--help, -h show help
UNITS
--smaller, --larger flags accept human-readable case-insensitive number
suffixes such as "k", "m", "g" and "t" referring to the metric units KB,
MB, GB and TB respectively. Adding an "i" to these prefixes, uses the IEC
units, so that "gi" refers to "gibibyte" or "GiB". A "b" at the end is
also accepted. Without suffixes the unit is bytes.
--older-than, --newer-than flags accept the string for days, hours and minutes
i.e. 1d2h30m states 1 day, 2 hours and 30 minutes.
FORMAT
Support string substitutions with special interpretations for following keywords.
Keywords supported if target is filesystem or object storage:
{} --> Substitutes to full path.
{base} --> Substitutes to basename of path.
{dir} --> Substitutes to dirname of the path.
{size} --> Substitutes to object size of the path.
{time} --> Substitutes to object modified time of the path.
Keywords supported if target is object storage:
{url} --> Substitutes to a shareable URL of the path.
EXAMPLES:
01. Find all "foo.jpg" in all buckets under "s3" account.
C:\> mc.exe find s3 --name "foo.jpg"
02. Find all objects with ".txt" extension under "s3/mybucket".
C:\> mc.exe find s3/mybucket --name "*.txt"
03. Find only the object names without the directory component under "s3/mybucket".
C:\> mc.exe find s3/mybucket --name "*" -print {base}
04. Find all images with ".jpg" extension under "s3/photos", prefixed with "album".
C:\> mc.exe find s3/photos --name "*.jpg" --path "*/album*/*"
05. Find all images with ".jpg", ".png", and ".gif" extensions, using regex under "s3/photos".
C:\> mc.exe find s3/photos --regex "(?i)\.(jpg|png|gif)$"
06. Find all images with ".jpg" extension under "s3/bucket" and copy to "play/bucket" *continuously*.
C:\> mc.exe find s3/bucket --name "*.jpg" --watch --exec "mc cp {} play/bucket"
07. Find and generate public URLs valid for 7 days, for all objects between 64 MB, and 1 GB in size under "s3" account.
C:\> mc.exe find s3 --larger 64MB --smaller 1GB --print {url}
08. Find all objects created in the last week under "s3/bucket".
C:\> mc.exe find s3/bucket --newer-than 7d
09. Find all objects which were created are older than 2 days, 5 hours and 10 minutes and exclude the ones with ".jpg"
extension under "s3".
C:\> mc.exe find s3 --older-than 2d5h10m --ignore "*.jpg"
10. List all objects up to 3 levels sub-directory deep under "s3/bucket".
C:\> mc.exe find s3/bucket --maxdepth 3
PS D:\minio>
4. JAVA
4.1 环境准备
最低要求:
Java 1.8 或更高版本。
Maven:
io.minio
minio
8.4.6
jar包资源：
Central Repository: io/minio/minio/8.4.6 (maven.org)
4.2 代码示例
import io.minio.*;
import io.minio.errors.MinioException;
import java.io.IOException;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
public class MinIOExamples {
public final static String bucketName = "my-bucketname";
public static void main(String[] args)
throws IOException, NoSuchAlgorithmException, InvalidKeyException {
try {
// create minioClient
MinioClient minioClient =
MinioClient.builder()
// minio服务端地址URL
.endpoint("http://127.0.0.1:9000")
// 用户名及密码（访问密钥/密钥）
.credentials("minio", "123ABCdef*")
.build();
// 检查 bucket 'my-bucketname' 是否存在不存在进行创建
boolean found =
minioClient.bucketExists(BucketExistsArgs.builder().bucket(bucketName).build());
if (!found) {
minioClient.makeBucket(MakeBucketArgs.builder()
.bucket(bucketName)
.build());
System.out.println("my-bucketname no found , create success");
}
// 上传文件
minioClient.uploadObject(
UploadObjectArgs.builder()
.bucket(bucketName)
.object("minio-8.4.6.jar")
.filename("F:\\lib\\minio-8.4.6.jar")
.build());
System.out.println("上传文件成功");
// 下载文件
minioClient.downloadObject(
DownloadObjectArgs.builder()
.bucket(bucketName)
.object("minio-8.4.6.jar")
.filename("F:\\minio-8.4.6.jar")
.build());
// 删除文件
minioClient.removeObject(
RemoveObjectArgs.builder().bucket(bucketName).object("minio-8.4.6.jar").build());
// 删除bucket
minioClient.removeBucket(RemoveBucketArgs.builder().bucket(bucketName).build());
} catch (MinioException e) {
System.out.println("Error occurred: " + e);
}
}
}
4.3 API操作
官方API参考文档：Java Client API Reference — MinIO Object Storage for Linux
4.3.1 bucket操作
4.3.1.1 bucket是否存在
bucketExists(BucketExistsArgs args)
public boolean bucketExists(BucketExistsArgs args) [Javadoc]
Checks if a bucket exists.
Parameters
| Parameter | Type | Description |
|---|---|---|
bucketName |
BucketExistsArgs | Arguments. |
| Returns |
|---|
| boolean - True if the bucket exists. |
Example
// Check whether 'my-bucketname' exists or not.
boolean found =
minioClient.bucketExists(BucketExistsArgs.builder().bucket("my-bucketname").build());
if (found) {
System.out.println("my-bucketname exists");
} else {
System.out.println("my-bucketname does not exist");
}
4.3.1.2 bucket创建
makeBucket(MakeBucketArgs args)
public void makeBucket(MakeBucketArgs args) [Javadoc]
Creates a bucket with given region and object lock feature enabled.
Parameters
| Parameter | Type | Description |
|---|---|---|
args |
MakeBucketArgs | Arguments to create bucket |
Example
// Create bucket with default region.
minioClient.makeBucket(
MakeBucketArgs.builder()
.bucket("my-bucketname")
.build());
// Create bucket with specific region.
minioClient.makeBucket(
MakeBucketArgs.builder()
.bucket("my-bucketname")
.region("us-west-1")
.build());
// Create object-lock enabled bucket with specific region.
minioClient.makeBucket(
MakeBucketArgs.builder()
.bucket("my-bucketname")
.region("us-west-1")
.objectLock(true)
.build());
4.3.1.3 bucket移除
removeBucket(RemoveBucketArgs args)
public void removeBucket(RemoveBucketArgs args) [Javadoc]
Removes an empty bucket.
Parameters
| Parameter | Type | Description |
|---|---|---|
args |
RemoveBucketArgs | Arguments. |
Example
minioClient.removeBucket(RemoveBucketArgs.builder().bucket(bucketName).build());
4.3.1.4 bucket查看
listBuckets()
public List [Javadoc]
Lists bucket information of all buckets.
| Returns |
|---|
| List |
Example
List bucketList = minioClient.listBuckets();
for (Bucket bucket : bucketList) {
System.out.println(bucket.creationDate() + ", " + bucket.name());
}
4.3.1.4 设置存储周期
setBucketLifecycle(SetBucketLifecycleArgs args)
public void setBucketLifecycle(SetBucketLifecycleArgs args) [Javadoc]
Sets lifecycle configuration to a bucket.
Parameters
| Parameter | Type | Description |
|---|---|---|
args |
SetBucketLifecycleArgs | Arguments. |
Example
// 配置生命周期规则
rules.add(
new LifecycleRule(
Status.ENABLED, // 开启状态
null,
new Expiration((ZonedDateTime) null, 365, null), // 保存365天
new RuleFilter("logs/"), // 目录配置
"rule2",
null,
null,
null));
LifecycleConfiguration lifecycleConfiguration = new LifecycleConfiguration(rules);
// 添加生命周期配置
minioClient.setBucketLifecycle(
SetBucketLifecycleArgs.builder().bucket("my-bucketname").config(lifecycleConfiguration).build());
4.3.1.5 多版本设置
若开启了多版本控制，上传对象时，OBS自动为每个对象创建唯一的版本号。上传同名的对象将以不同的版本号同时保存在OBS中。
若未开启多版本控制，向同一个文件夹中上传同名的对象时，新上传的对象将覆盖原有的对象。
某些功能（例如版本控制、对象锁定和存储桶复制）需要使用擦除编码分布式部署 MinIO。开启了版本控制后，允许在同一密钥下保留同一对象的多个版本。
设置存储桶的版本控制配置。
public void setBucketVersioning(SetBucketVersioningArgs args)
获取存储桶的版本控制配置。
public VersioningConfiguration getBucketVersioning(GetBucketVersioningArgs args)
示例代码：
// 版本配置
// 'my-bucketname'启用版本控制
minioClient.setBucketVersioning(
SetBucketVersioningArgs.builder()
.bucket("my-bucketname")
.config(new VersioningConfiguration(VersioningConfiguration.Status.ENABLED, null))
.build());
System.out.println("Bucket versioning is enabled successfully");
// 'my-bucketname'暂停版本控制
minioClient.setBucketVersioning(
SetBucketVersioningArgs.builder()
.bucket("my-bucketname")
.config(new VersioningConfiguration(VersioningConfiguration.Status.SUSPENDED, null))
.build());
System.out.println("Bucket versioning is suspended successfully");
4.3.2 对象操作
4.3.2.1 上传对象
- PutObject 调用PutObject接口上传文件（Object）。
public ObjectWriteResponse putObject(PutObjectArgs args)
注意事项：
添加的Object大小不能超过5 GB。
默认情况下，如果已存在同名Object且对该Object有访问权限，则新添加的Object将覆盖原有的Object，并返回200 OK。OSS没有文件夹的概念，所有资源都是以文件来存储，但您可以通过创建一个以正斜线（/）结尾，大小为0的Object来创建模拟文件夹。
示例1，InputStream上传：
// 1. 创建InputStream上传
File file = new File("D:\\deploy\\nacos-server-2.0.3.tar.gz");
InputStream bais = new FileInputStream(file);
long start = System.currentTimeMillis();
minioClient.putObject(
PutObjectArgs.builder().bucket("my-bucketname").object(file.getName()).stream(
bais, bais.available(), -1)
.build());
bais.close();
System.out.println("my-objectname is uploaded successfully 耗时：" + (System.currentTimeMillis() - start));
- uploadObject 将文件中的内容作为存储桶中的对象上传。
public void uploadObject(UploadObjectArgs args)
示例：
//将本地文件上传到minio
minioClient.uploadObject(UploadObjectArgs.builder() .bucket("my-bucketname")
.object("start.sh").filename("D:\\deploy\\service\\general-task\\start.sh")
.build());
System.out.println("my-filename is uploaded to my-objectname successfully");
4.3.2.2 获取对象
- getObject 用于获取某个文件（Object）。此操作需要对此Object具有读权限。获取对象的数据。InputStream使用后返回必须关闭以释放网络资源。
public InputStream getObject(GetObjectArgs args)
示例：
// 2. 获取对象的InputStream,并保存为文件
InputStream stream =minioClient.getObject(GetObjectArgs.builder().bucket("my-bucketname").object("my-objectname").build());
// 读流
File targetFile = new File("D:\\deploy\\targetFile.tmp");
FileUtils.copyInputStreamToFile(stream, targetFile);
stream.close();
- downloadObject 将对象的数据下载到文件。
public void downloadObject(DownloadObjectArgs args)
示例：
// 4. 下载对象到本地文件
minioClient.downloadObject(
DownloadObjectArgs.builder()
.bucket("my-bucketname")
.object("my-objectname")
.filename("D:\\deploy\\service\\general-task\\aaa.tmp")
.build());
System.out.println("my-objectname is successfully downloaded to my-filename");
- getPresignedObjectUrl
获取一个指定了 HTTP 方法、到期时间和自定义请求参数的对象URL地址，也就是返回带签名的URL，这个地址可以提供给没有登录的第三方共享访问或者上传对象。
public String getPresignedObjectUrl(GetPresignedObjectUrlArgs args)
示例：
// 指定一个GET请求，返回获取文件对象的URL，此URL过期时间为一天
String url =minioClient.getPresignedObjectUrl(
GetPresignedObjectUrlArgs.builder()
.method(Method.GET)
.bucket("my-bucketname")
.object("start.sh")
.expiry(60 * 60 * 24)
.build());
System.out.println(url);
4.3.2.3 复制对象
- copyObject 通过服务器端从另一个对象复制数据来创建一个对象。
public ObjectWriteResponse copyObject(CopyObjectArgs args)
示例：
// 5. 将my-bucketname中的aa.tmp文件，复制到aaaaa桶下的bb.tmp对象
minioClient.copyObject(
CopyObjectArgs.builder()
.bucket("aaaaa")
.object("bb.tmp")
.source(
CopySource.builder()
.bucket("my-bucketname")
.object("aa.tmp")
.build())
.build());
System.out.println("successfully");
4.3.2.4 删除对象
- removeObject 移除一个对象。
public void removeObject(RemoveObjectArgs args)
示例：
// 6. 删除单个对象
minioClient.removeObject(
RemoveObjectArgs.builder().bucket("my-bucketname").object("my-objectname").build());
// 删除指定版本号的对象
minioClient.removeObject(
RemoveObjectArgs.builder()
.bucket("my-bucketname")
.object("my-versioned-objectname")
.versionId("my-versionid")
.build());
- removeObjects 懒惰地删除多个对象。它需要迭代返回的 Iterable 以执行删除。
public Iterable> removeObjects(RemoveObjectsArgs args)
示例：
// 7. 删除多个文件
List objects = new LinkedList<>();
objects.add(new DeleteObject("aa.tmp"));
objects.add(new DeleteObject("my-objectname"));
objects.add(new DeleteObject("nacos-server-2.0.3.tar.gz"));
Iterable> results =
minioClient.removeObjects(
RemoveObjectsArgs.builder().bucket("my-bucketname").objects(objects).build());
for (Result result : results) {
DeleteError error = result.get();
System.out.println(
"Error in deleting object " + error.objectName() + "; " + error.message());
}
4.3.2.5 对象信息查询
- listObjects列出桶的对象信息。
public Iterable> listObjects(ListObjectsArgs args)
示例1，查询存储桶下文件信息：
// 8. 查询存储桶下文件信息
Iterable> results =
minioClient.listObjects(ListObjectsArgs.builder().bucket("my-bucketname").build());
for (Result- result : results) {
Item item = result.get();
System.out.println(item.lastModified() + "\t" + item.size() + "\t" + item.objectName());
}
示例2，递归查询存储桶下文件信息【存在文件夹时，使用】：
Iterable> results =
minioClient.listObjects(
ListObjectsArgs.builder().bucket("my-bucketname").recursive(true).build());
for (Result- result : results) {
Item item = result.get();
System.out.println(item.lastModified() + "\t" + item.size() + "\t" + item.objectName());
}
示例2， 条件查询，指定前缀、后缀、最大数量：
// 条件查询，指定前缀、后缀、最大数量
Iterable> results =
minioClient.listObjects(
ListObjectsArgs.builder()
.bucket("my-bucketname")
.startAfter("ExampleGuide.pdf")
.prefix("E")
.maxKeys(100)
.build());
for (Result- result : results) {
Item item = result.get();
System.out.println(item.lastModified() + "\t" + item.size() + "\t" + item.objectName());
}
- 标签
public void setObjectTags(SetObjectTagsArgs args) 为对象设置标签。
示例：
Map map = new HashMap<>();
map.put("Project", "Project One");
map.put("User", "jsmith");
minioClient.setObjectTags(
SetObjectTagsArgs.builder()
.bucket("my-bucketname")
.object("my-objectname")
.tags(map)
.build());
public Tags getObjectTags(GetObjectTagsArgs args) 获取对象的标签。
示例：
Tags tags = minioClient.getObjectTags(
GetObjectTagsArgs.builder().bucket("my-bucketname").object("my-objectname").build());
System.out.println("Object tags: " + tags.get());
private void deleteObjectTags(DeleteObjectTagsArgs args) 删除对象的标签。
示例：
minioClient.deleteObjectTags(
DeleteObjectTagsArgs.builder().bucket("my-bucketname").object("my-objectname").build());
System.out.println("Object tags deleted successfully");
4.4 Springboot集成
4.4.1 环境搭建
org.jetbrains.kotlin
kotlin-stdlib
1.3.70
com.squareup.okhttp3
okhttp
4.9.0
io.minio
minio
8.2.0
com.squareup.okhttp3
okhttp
4.4.2 代码示例
4.4.2.1 配置参数
package com.minio.learning.springminio.config;
import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.stereotype.Component;
/**
* @author Jiang
*/
@Data
@Component
@ConfigurationProperties(prefix = "minio")
public class MinioProperties {
private String endpoint;
private String accessKey;
private String secretKey;
}
4.4.2.2 配置文件修改
server:
port: 8080
minio:
endpoint: http://localhost:9000
accessKey: minio
secretKey: 123ABCdef*
bucketName: 202212
# byYear byMonth byDate
storagePolicy: no
4.4.2.3 MinioClient创建
package com.minio.learning.springminio.config;
import io.minio.MinioClient;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import javax.annotation.Resource;
/**
* @author Jiang
*/
@Configuration
public class MinioConfig {
@Resource
private MinioProperties minioProperties;
@Bean
public MinioClient minioClient(){
return MinioClient.builder()
.endpoint(minioProperties.getEndpoint())
.credentials(minioProperties.getAccessKey(),minioProperties.getSecretKey())
.build();
}
}
4.4.2.4 Minio API操作
package com.minio.learning.springminio.service;
import com.minio.learning.springminio.entity.FileInfo;
import com.minio.learning.springminio.entity.FileVO;
import com.minio.learning.springminio.entity.RemoveFailVO;
import org.springframework.web.multipart.MultipartFile;
import javax.servlet.http.HttpServletResponse;
import java.util.List;
/**
* @author Jiang
* @date 2022-12-01 17:29
*/
public interface MinioService {
/**
* 创建一个桶
* @param bucketName bucketName
*/
boolean createBucket(String bucketName);
/**
* 上传一个文件
* @param bucketName bucketName
* @param files files
*
*/
List uploadFile(String bucketName, MultipartFile[] files);
/**
* 列出所有的桶
*
*/
List listBuckets();
/**
* 列出一个桶中的所有文件和目录
*/
List listFiles(String bucketName, String prefix);
/**
* 下载一个文件
*/
void download(String bucketName, String filePath, HttpServletResponse response);
/**
* 删除一个桶
* @param bucketName bucketName
*/
boolean deleteBucket(String bucketName);
/**
* 删除一个对象
* @param bucketName bucketName
* @param filePath filePath
* @return 删除结果
*/
boolean deleteObject(String bucketName, String filePath);
/**
* 删除一个或多个对象
* @param bucketName bucketName
* @param filePaths 文件路径
* @return 删除失败列表
*/
List deleteObjects(String bucketName, List filePaths);
}
实现类
package com.minio.learning.springminio.service.impl;
import com.minio.learning.springminio.entity.FileInfo;
import com.minio.learning.springminio.entity.FileVO;
import com.minio.learning.springminio.entity.RemoveFailVO;
import com.minio.learning.springminio.exception.ServiceException;
import com.minio.learning.springminio.service.MinioService;
import io.minio.*;
import io.minio.messages.Bucket;
import io.minio.messages.DeleteError;
import io.minio.messages.DeleteObject;
import io.minio.messages.Item;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.apache.tomcat.util.http.fileupload.IOUtils;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import org.springframework.web.multipart.MultipartFile;
import javax.annotation.Resource;
import javax.servlet.http.HttpServletResponse;
import java.io.InputStream;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.util.ArrayList;
import java.util.List;
/**
* @author Jiang
* @date 2022-12-01 17:30
*/
@Service
@Slf4j
public class MinioServiceImpl implements MinioService {
@Resource
private MinioClient minioClient;
@Value("${minio.storagePolicy}")
private String storagePolicy;
@Value("${minio.endpoint}")
private String endpoint;
/**
* 创建一个桶
*
* @param bucketName bucketName
*/
@Override
public boolean createBucket(String bucketName) {
boolean found = bucketIsExist(bucketName);
try {
if (!found) {
minioClient.makeBucket(MakeBucketArgs.builder().bucket(bucketName).build());
}
} catch (Exception e) {
log.error("minio创建一个桶失败：{}", e.getMessage());
throw new ServiceException("创建存储桶失败！");
}
return true;
}
/**
* 获取文件上传前缀
*
* @return String
*/
private String getFilePrefix() {
String prefix = null;
LocalDateTime dateTime = LocalDateTime.now();
switch (storagePolicy) {
case "byDate":
DateTimeFormatter byDate = DateTimeFormatter.ofPattern("yyyy/MM/dd/");
prefix = dateTime.format(byDate);
break;
case "byMonth":
DateTimeFormatter byMonth = DateTimeFormatter.ofPattern("yyyy/MM/");
prefix = dateTime.format(byMonth);
break;
case "byYear":
prefix = dateTime.getYear() + "/";
break;
default:
prefix="";
break;
}
return prefix;
}
/**
* 上传一个文件
*
* @param bucketName bucketName
* @param files files
*/
@Override
public List uploadFile(String bucketName, MultipartFile[] files) {
int length = files.length;
boolean result = bucketIsExist(bucketName);
if (!result) {
throw new ServiceException("存储桶不存在!");
}
List fileList = new ArrayList<>(length);
for (MultipartFile file : files) {
String uploadFileName = file.getOriginalFilename();
try {
PutObjectArgs args = PutObjectArgs.builder()
.bucket(bucketName)
.object(getFilePrefix() + uploadFileName)
.stream(file.getInputStream(), file.getSize(), -1)
.contentType(file.getContentType())
.build();
minioClient.putObject(args);
String url = endpoint + "/" + bucketName + "/" + getFilePrefix() + uploadFileName;
FileVO fileVO = new FileVO();
fileVO.setFileName(uploadFileName);
fileVO.setUrl(url);
fileList.add(fileVO);
} catch (Exception e) {
log.error("minio上传文件失败：{}", e.getMessage());
}
}
return fileList;
}
/**
* 列出所有的桶
*/
@Override
public List listBuckets() {
List list;
try {
list = minioClient.listBuckets();
} catch (Exception e) {
throw new ServiceException("列出所有的桶失败！");
}
List names = new ArrayList<>();
list.forEach(b -> names.add(b.name()));
return names;
}
/**
* 判断存储桶是否存在
*
* @param bucket bucket
* @return boolean
*/
private boolean bucketIsExist(String bucket) {
try {
return minioClient.bucketExists(BucketExistsArgs.builder().bucket(bucket).build());
} catch (Exception e) {
throw new ServiceException("系统中断，稍后重试！");
}
}
/**
* 列出一个桶中的所有文件和目录
*
* @param bucketName bucketName
*/
@Override
public List listFiles(String bucketName, String prefix) {
boolean result = bucketIsExist(bucketName);
if (!result) {
throw new ServiceException("存储桶不存在!");
}
Iterable> results;
if (StringUtils.isNotBlank(prefix)) {
results = minioClient.listObjects(ListObjectsArgs.builder().bucket(bucketName).prefix(prefix).recursive(false).build());
} else {
results = minioClient.listObjects(ListObjectsArgs.builder().bucket(bucketName).recursive(false).build());
}
DateTimeFormatter fmt = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
List infos = new ArrayList<>();
results.forEach(r -> {
FileInfo info = new FileInfo();
try {
Item item = r.get();
info.setFilename(item.objectName());
boolean dir = item.isDir();
info.setDirectory(dir);
// 文件夹没有修改时间
if (!dir) {
info.setLastModifiedTime(item.lastModified().format(fmt));
}
infos.add(info);
} catch (Exception e) {
log.info("获取文件失败！{}", e.getMessage());
}
});
return infos;
}
/**
* 下载一个文件
*
* @param bucketName bucketName
* @param filePath filePath
*/
@Override
public void download(String bucketName, String filePath, HttpServletResponse response) {
try {
String fileName = getFileName(filePath, bucketName);
// 获取对象信息
StatObjectResponse statObjectResponse = minioClient.statObject(
StatObjectArgs.builder().bucket(bucketName).object(fileName).build());
response.setContentType(statObjectResponse.contentType());
response.setHeader("Content-Disposition", URLEncoder.encode(fileName, String.valueOf(StandardCharsets.UTF_8)));
InputStream stream = minioClient.getObject(GetObjectArgs.builder().bucket(bucketName).object(fileName).build());
IOUtils.copy(stream, response.getOutputStream());
} catch (Exception e) {
throw new ServiceException("下载失败!");
}
}
/**
* 删除一个桶
*
* @param bucketName bucketName
*/
@Override
public boolean deleteBucket(String bucketName) {
boolean result = bucketIsExist(bucketName);
if (!result) {
throw new ServiceException("存储桶不存在!");
}
try {
minioClient.removeBucket(RemoveBucketArgs.builder().bucket(bucketName).build());
} catch (Exception e) {
log.error("minio删除存储桶失败：{}", e.getMessage());
throw new ServiceException("删除存储桶失败!");
}
return true;
}
/**
* 删除一个对象
*
* @param bucketName bucketName
* @param filePath filePath
* @return 删除结果
*/
@Override
public boolean deleteObject(String bucketName, String filePath) {
String fileName = getFileName(bucketName, filePath);
try {
minioClient.removeObject(RemoveObjectArgs.builder().bucket(bucketName).object(fileName).build());
} catch (Exception e) {
log.error("minio删除文件失败：{}", e.getMessage());
throw new ServiceException("删除文件失败!");
}
return true;
}
/**
* 获取文件名称
*
* @param filePath 文件路径
* @param bucketName 存储桶名称
* @return 获取文件名称
*/
private String getFileName(String filePath, String bucketName) {
String prefix = endpoint + "/" + bucketName + "/";
int length = prefix.length();
return filePath.substring(length);
}
/**
* 删除一个或多个对象
* 删除文件夹下文件 /fileName/fileName/sss.txt
*
* @param bucketName bucketName
* @param filePaths 文件路径
* @return 删除失败列表
*/
@Override
public List deleteObjects(String bucketName, List filePaths) {
List list = new ArrayList<>();
for (String filePath : filePaths) {
String fileName = getFileName(filePath,bucketName);
list.add(new DeleteObject(fileName));
}
Iterable> iterable = minioClient.removeObjects(RemoveObjectsArgs.builder().bucket(bucketName).objects(list).build());
List failList = new ArrayList<>();
for (Result result : iterable) {
DeleteError error = null;
try {
error = result.get();
} catch (Exception e) {
log.error("minio删除对象失败：{}", e.getMessage());
}
assert error != null;
log.info("minio删除错误->bucketName={},objectName={},message={}", error.bucketName(), error.objectName(), error.message());
RemoveFailVO failVO = new RemoveFailVO();
failVO.setObjectName(error.objectName());
failVO.setBucketName(error.bucketName());
failList.add(failVO);
}
return failList;
}
}
4.4.2.5 接口提供
package com.minio.learning.springminio.controller;
import com.minio.learning.springminio.entity.FileInfo;
import com.minio.learning.springminio.entity.FileVO;
import com.minio.learning.springminio.entity.RemoveFailVO;
import com.minio.learning.springminio.resp.CommonResult;
import com.minio.learning.springminio.service.MinioService;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.validation.annotation.Validated;
import org.springframework.web.bind.annotation.*;
import org.springframework.web.multipart.MultipartFile;
import javax.annotation.Resource;
import javax.servlet.http.HttpServletResponse;
import javax.validation.constraints.NotBlank;
import javax.validation.constraints.NotEmpty;
import java.util.List;
/**
* @author Jiang
* @date 2022-12-12 17:08
*/
@Slf4j
@Validated
@RestController
@RequestMapping("/minio")
public class MinioController {
@Resource
private MinioService minioService;
@Value("${minio.bucketName}")
private String bucket;
@GetMapping("/createBucket")
public CommonResult createBucket(@RequestParam(name = "bucketName")
@NotBlank(message = "存储桶名称不能为空！") String bucketName) {
boolean result = minioService.createBucket(bucketName);
if (result) {
return CommonResult.ok("创建桶成功！");
} else {
return CommonResult.error("创建桶失败！");
}
}
@GetMapping("/deleteBucket")
public CommonResult deleteBucket(@RequestParam(name = "bucketName")
@NotBlank(message = "存储桶名称不能为空！") String bucketName) {
boolean result = minioService.deleteBucket(bucketName);
if (result) {
return CommonResult.ok("删除桶成功！");
} else {
return CommonResult.error("删除桶失败！");
}
}
@PostMapping("/uploadFile")
public CommonResult> uploadFile(@RequestParam(name = "file")
@NotEmpty(message = "上传文件不能为空！") MultipartFile[] files,
@RequestParam(name = "bucketName",required = false)
String bucketName) {
if(StringUtils.isBlank(bucketName)){
bucketName = bucket;
}
List fileVOS = minioService.uploadFile(bucketName, files);
if (fileVOS.size() > 0) {
return CommonResult.ok("上传成功！",fileVOS);
} else {
return CommonResult.error("上传失败！");
}
}
@GetMapping("/download")
public void uploadFile(@RequestParam(name = "filePath") @NotBlank(message = "下载路径不能为空！") String filePath,
HttpServletResponse response,
@RequestParam(name = "bucketName",required = false) String bucketName) {
if(StringUtils.isBlank(bucketName)){
bucketName = bucket;
}
minioService.download(bucketName, filePath, response);
}
@GetMapping("/deleteObjects")
public CommonResult> deleteObjects(@RequestParam(name = "bucketName",required = false)
String bucketName,
@RequestParam("filePaths") List filePaths) {
if(StringUtils.isBlank(bucketName)){
bucketName = bucket;
}
List removeFailVOS = minioService.deleteObjects(bucketName, filePaths);
if (removeFailVOS.size() > 0) {
return CommonResult.error("部分删除失败！", removeFailVOS);
} else {
return CommonResult.ok("删除成功");
}
}
@GetMapping("/listBuckets")
public CommonResult> listBuckets() {
List buckets = minioService.listBuckets();
return CommonResult.ok("查询成功！", buckets);
}
@GetMapping("/listFiles")
public CommonResult> listBuckets(@RequestParam(name = "bucketName",required = false) String bucketName,
@RequestParam(name = "prefix", required = false) String prefix) {
if(StringUtils.isBlank(bucketName)){
bucketName = bucket;
}
List fileInfos = minioService.listFiles(bucketName, prefix);
return CommonResult.ok("查询成功！", fileInfos);
}
}
4.4.2.6 调用示例
http://127.0.0.1:8080/minio/createBucket?bucketName=202213
bucketName = bucket;
}
minioService.download(bucketName, filePath, response);
}
@GetMapping("/deleteObjects")
public CommonResult> deleteObjects(@RequestParam(name = "bucketName",required = false)
String bucketName,
@RequestParam("filePaths") List filePaths) {
if(StringUtils.isBlank(bucketName)){
bucketName = bucket;
}
List removeFailVOS = minioService.deleteObjects(bucketName, filePaths);
if (removeFailVOS.size() > 0) {
return CommonResult.error("部分删除失败！", removeFailVOS);
} else {
return CommonResult.ok("删除成功");
}
}
@GetMapping("/listBuckets")
public CommonResult> listBuckets() {
List buckets = minioService.listBuckets();
return CommonResult.ok("查询成功！", buckets);
}
@GetMapping("/listFiles")
public CommonResult> listBuckets(@RequestParam(name = "bucketName",required = false) String bucketName,
@RequestParam(name = "prefix", required = false) String prefix) {
if(StringUtils.isBlank(bucketName)){
bucketName = bucket;
}
List fileInfos = minioService.listFiles(bucketName, prefix);
return CommonResult.ok("查询成功！", fileInfos);
}
}
###### 4.4.2.6 调用示例
http://127.0.0.1:8080/minio/createBucket?bucketName=202213