【分布式云储存】Springboot微服务接入MinIO实现文件服务
文章目录
-
- 前言
- 技术回顾
- 准备工作
-
- 申请accessKey\secretKey
- 创建数据存储桶
- 公共资源直接访问测试
- 接入springboot实现文件服务
-
- 依赖引入
- 配置文件
- MinIO配置
- MinIO工具类
- OkHttpSSLSocketClient兼容ssl
- 静态资源预览解决方案
- 资源上传预览测试
- 测试结果
前言
上篇博客我们介绍了分布式云存储MinIO作业环境的搭建，以及分布式云储存MinIO在实际的文件服务中的优势。那么，今天我们就小试牛刀来将MinIO接入我们的微服务项目，实现一个分布式的文件服务器。
技术回顾
MinIO 提供高性能、与S3 兼容的对象存储系统，让你自己能够构建自己的私有云储存服务。
MinIO原生支持 Kubernetes，它可用于每个独立的公共云、每个 Kubernetes 发行版、私有云和边缘的对象存储套件。
MinIO是软件定义的，不需要购买其他任何硬件，在 GNU AGPL v3 下是 100% 开源的。
准备工作
申请accessKey\secretKey
MinIO控制台申请accessKey\secretKey
http://your_hostname:18001~18004/login minio/minio123
创建数据存储桶
创建数据存储桶
点击创建的桶我们进行访问策略配置
访问策略有private\custom\public
public 公共的桶，任何人都可以访问资源，直接映射为静态资源，可直接提供预览和下载
custom 自定义桶，用户根据自身需求定义访问规则
private 私有的桶，需要授权才能访问
根据一般的生产需求，我们定义一个private，一个custom桶。private保存私有资源，custom保存公共资源并禁止目录访问。
private 规则：
custom 规则：
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": [
"*"
]
},
"Action": [
"s3:GetBucketLocation",
"s3:ListBucketMultipartUploads"
],
"Resource": [
"arn:aws:s3:::sacpublic"
]
},
{
"Effect": "Allow",
"Principal": {
"AWS": [
"*"
]
},
"Action": [
"s3:AbortMultipartUpload",
"s3:DeleteObject",
"s3:GetObject",
"s3:ListMultipartUploadParts",
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::sacpublic/*"
]
}
]
}
公共资源直接访问测试
私有资源代码博文最后测试
公共资源直接访问测试：
接入springboot实现文件服务
依赖引入
maven引入依赖
io.minio
minio
8.2.0
配置文件
配置文件
minio.url = https://10.10.22.91:9100
minio.accessKey = fUIXbkBZ9UQTHOOZXNGW
minio.secretKey = sy1RAgItAOk9pk1gE7FbrPYzsZI87CfpGkuoY0KW
minio.buckets.public = sacpublic
minio.buckets.private = sacprivate
注意：接口调用minio url 为nginx映射出来的9000端口
http://your_hostname:9000
MinIO配置
MinIO配置
/**
* MinioConfig
* @author senfel
* @version 1.0
* @date 2023/9/14 11:37
*/
@Configuration
@ConditionalOnProperty(name = "oss.file.service", havingValue = "minio", matchIfMissing = true)
public class MinioConfig {
@Value("${minio.url}")
private String minioUrl;
@Value("${minio.accessKey}")
private String accessKey;
@Value("${minio.secretKey}")
private String secretKey;
@Bean
public MinioClient getMinioClient() {
return MinioClient.builder().endpoint(url)
.credentials(accessKey, secretKey).build();
}
}
MinIO工具类
MinIO工具类
/**
* MinioUtil
* @author senfel
* @version 1.0
* @date 2023/9/14 10:26
*/
@Component
@ConditionalOnProperty(name = "oss.file.service", havingValue = "minio", matchIfMissing = true)
public class MinioUtil {
@Resource
private MinioClient minioClient;
/**
* 创建一个桶
*/
public void createBucket(String bucket) throws Exception {
boolean found = minioClient.bucketExists(BucketExistsArgs.builder().bucket(bucket).build());
if (!found) {
minioClient.makeBucket(MakeBucketArgs.builder().bucket(bucket).build());
}
}
/**
* 上传一个文件
*/
public ObjectWriteResponse uploadFile(InputStream stream, String bucket, String objectName) throws Exception {
String prefix = objectName.substring(objectName.lastIndexOf(".") + 1);
//静态资源预览解决方案
//String contentType = ViewContentType.getContentType(prefix);
ObjectWriteResponse objectWriteResponse = minioClient.putObject(PutObjectArgs.builder().bucket(bucket).object(objectName)
.contentType(contentType)
.stream(stream, -1, 10485760).build());
return objectWriteResponse;
}
/**
* 列出所有的桶
*/
public List listBuckets() throws Exception {
List list = minioClient.listBuckets();
List names = new ArrayList<>();
list.forEach(b -> {
names.add(b.name());
});
return names;
}
/**
* 下载一个文件
*/
public InputStream download(String bucket, String objectName) throws Exception {
InputStream stream = minioClient.getObject(
GetObjectArgs.builder().bucket(bucket).object(objectName).build());
return stream;
}
/**
* 删除一个桶
*/
public void deleteBucket(String bucket) throws Exception {
minioClient.removeBucket(RemoveBucketArgs.builder().bucket(bucket).build());
}
/**
* 删除一个对象
*/
public void deleteObject(String bucket, String objectName) throws Exception {
minioClient.removeObject(RemoveObjectArgs.builder().bucket(bucket).object(objectName).build());
}
/**
* 复制文件
*
* @Param: [sourceBucket, sourceObject, targetBucket, targetObject]
* @return: void
* @Date: 2021/11/15
*/
public void copyObject(String sourceBucket, String sourceObject, String targetBucket, String targetObject) throws Exception {
this.createBucket(targetBucket);
minioClient.copyObject(CopyObjectArgs.builder().bucket(targetBucket).object(targetObject)
.source(CopySource.builder().bucket(sourceBucket).object(sourceObject).build()).build());
}
/**
* 获取文件信息
*
* @Param: [bucket, objectName]
* @return: java.lang.String
*/
public String getObjectInfo(String bucket, String objectName) throws Exception {
return minioClient.statObject(StatObjectArgs.builder().bucket(bucket).object(objectName).build()).toString();
}
/**
* 生成一个给HTTP GET请求用的presigned URL。浏览器/移动端的客户端可以用这个URL进行下载，即使其所在的存储桶是私有的。
* @Param: [bucketName, objectName, expires]
* @return: java.lang.String
*/
public String getPresignedObjectUrl(String bucketName, String objectName, Integer expires) throws Exception {
GetPresignedObjectUrlArgs build = GetPresignedObjectUrlArgs
.builder().bucket(bucketName).object(objectName).expiry(expires).method(Method.GET).build();
return minioClient.getPresignedObjectUrl(build);
}
}
OkHttpSSLSocketClient兼容ssl
对于改解决方案一般生产环境都有固定的域名和匹配的ssl证书，如果也不用https我们代码则不用兼容ssl。但是如果我们配置了不可信任的ssl，这里我们则需要进行ssl兼容方案。
/**
* MinioConfig
* @author senfel
* @version 1.0
* @date 2023/9/14 11:37
*/
@Configuration
@ConditionalOnProperty(name = "oss.file.service", havingValue = "minio", matchIfMissing = true)
public class MinioConfig {
@Value("${minio.url}")
private String minioUrl;
@Value("${minio.accessKey}")
private String accessKey;
@Value("${minio.secretKey}")
private String secretKey;
@Bean
public MinioClient getMinioClient() {
OkHttpClient okHttpClient = new OkHttpClient.Builder()
.sslSocketFactory(OkHttpSSLSocketClient.getSSLSocketFactory(),OkHttpSSLSocketClient.getX509TrustManager()) // //通过sslSocketFactory方法设置https证书
.hostnameVerifier(OkHttpSSLSocketClient.getHostnameVerifier())
.build();
return MinioClient.builder().endpoint(minioUrl).httpClient(okHttpClient)
.credentials(accessKey, secretKey).build();
}
}
/**
* OkHttpSSLSocketClient
* @author senfel
* @version 1.0
* @date 2023/9/15 10:07
*/
public class OkHttpSSLSocketClient{
//获取SSLSocketFactory
public static SSLSocketFactory getSSLSocketFactory() {
try {
SSLContext sslContext = SSLContext.getInstance("SSL");
sslContext.init(null, getTrustManager(), new SecureRandom());
return sslContext.getSocketFactory();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
//获取TrustManager
private static TrustManager[] getTrustManager() {
TrustManager[] trustAllCerts = new TrustManager[]{
new X509TrustManager() {
@Override
public void checkClientTrusted(X509Certificate[] chain, String authType) {
}
@Override
public void checkServerTrusted(X509Certificate[] chain, String authType) {
}
@Override
public X509Certificate[] getAcceptedIssuers() {
return new X509Certificate[]{};
}
}
};
return trustAllCerts;
}
//获取HostnameVerifier，验证主机名
public static HostnameVerifier getHostnameVerifier() {
HostnameVerifier hostnameVerifier = (s, sslSession) -> true;
return hostnameVerifier;
}
//X509TrustManager：证书信任器管理类
public static X509TrustManager getX509TrustManager() {
X509TrustManager x509TrustManager = new X509TrustManager() {
//检查客户端的证书是否可信
@Override
public void checkClientTrusted(X509Certificate[] chain, String authType) {
}
//检查服务器端的证书是否可信
@Override
public void checkServerTrusted(X509Certificate[] chain, String authType) {
}
@Override
public X509Certificate[] getAcceptedIssuers() {
return new X509Certificate[0];
}
};
return x509TrustManager;
}
}
静态资源预览解决方案
由于我们使用API上传，minio不能区分我们资源的类型，如果资源类型不对则不能正确提供预览功能，都直接变成为访问资源链接就下载了。
所以，在实际的项目集成中我们在上传资源前需要将资源类型写入请求中，方便minio解析并提供预览和下载功能。
/**
* 上传一个文件
*/
public ObjectWriteResponse uploadFile(InputStream stream, String bucket, String objectName) throws Exception {
String prefix = objectName.substring(objectName.lastIndexOf(".") + 1);
//静态资源预览解决方案
String contentType = ViewContentType.getContentType(prefix);
ObjectWriteResponse objectWriteResponse = minioClient.putObject(PutObjectArgs.builder().bucket(bucket).object(objectName)
.contentType(contentType)
.stream(stream, -1, 10485760).build());
return objectWriteResponse;
}
/**
* ViewContentType
* @author senfel
* @version 1.0
* @date 2023/9/14 17:27
*/
public enum ViewContentType {
DEFAULT("default","application/octet-stream"),
JPG("jpg", "image/jpeg"),
TIFF("tiff", "image/tiff"),
GIF("gif", "image/gif"),
JFIF("jfif", "image/jpeg"),
PNG("png", "image/png"),
TIF("tif", "image/tiff"),
ICO("ico", "image/x-icon"),
JPEG("jpeg", "image/jpeg"),
WBMP("wbmp", "image/vnd.wap.wbmp"),
FAX("fax", "image/fax"),
NET("net", "image/pnetvue"),
JPE("jpe", "image/jpeg"),
RP("rp", "image/vnd.rn-realpix");
private String prefix;
private String type;
public static String getContentType(String prefix){
if(StringUtils.isEmpty(prefix)){
return DEFAULT.getType();
}
prefix = prefix.substring(prefix.lastIndexOf(".") + 1);
for (ViewContentType value : ViewContentType.values()) {
if(prefix.equalsIgnoreCase(value.getPrefix())){
return value.getType();
}
}
return DEFAULT.getType();
}
ViewContentType(String prefix, String type) {
this.prefix = prefix;
this.type = type;
}
public String getPrefix() {
return prefix;
}
public String getType() {
return type;
}
}
资源上传预览测试
MinIO对于图片等资源可以直接预览，对于excel文档等直接提供下载功能
@SpringBootTest
@RunWith(SpringJUnit4ClassRunner.class)
public class CodeDbInfoServiceTests {
@Resource
@Lazy
private MinioUtil minioUtil;
/**
* upload
* @author senfel
* @date 2023/9/27 10:00
* @return void
*/
@Test
public void upload() throws Exception {
FileInputStream fileInputStream = new FileInputStream("C:/Users/dev/Desktop/minio.png");
ObjectWriteResponse sacprivate = minioUtil.uploadFile(fileInputStream, "sacprivate", "test/minio.png");
System.err.println(sacprivate.bucket());
}
/**
* 获取私库连接
* @author senfel
* @date 2023/9/27 10:01
* @return void
*/
@Test
public void getPrivateUrl() throws Exception{
String url = minioUtil.getPresignedObjectUrl("sacprivate", "test/minio.png", 60);
System.err.println(url);
}
}
测试结果
图片成功上传
私库直接预览访问失败
获取私库链接过期时间为60s
访问成功
60s后再次访问该链接，提示链接已经过期
至此minio接入并测试完成，对于公开桶的资源直接可任意访问不用获取有效期链接。
写在最后
Springboot微服务接入MinIO实现文件服务较为简单，我们只要按照官方文档引入依赖调用即可。值得注意的是我们根据需求选择接入ssl兼容方案和静态资源预览功能。当然，minio的private\custom\public可以完全实现我们各种需求，可以完全替代市场上付费和笨重的分布式服务。