FastDFS实战使用-高性能文件服务器
文章目录
- 1.FastDFS介绍
- 2.安装环境
- 1.安装libevent
- 2.安装fastdfs
- 1.解压libfastcommon
- 2.安装Tracker服务
- 进入目录
- 参考以下命令复制文件
- 配置tracker服务
- 启动tracker服务
- 3.安装storage服务
- 修改配置文件
- 修改图片路径
- 修改tracker的配置
- 修改组名
- 启动storage
- 查看进程
- 3.使用client上传图片
- 4.配置nginx
- 1.解压fastdfs-nginx的压缩包
- 2.修改config配置文件
- 3.安装nginx
- 1.安装基本环境
- 2.解压nginx
- 3.配置nginx
- 2.FastDFS整合SpringBoot
- 依赖
- fastdfs启动类
- 配置文件
- 客户端工具类
- File文件工具类
1.FastDFS介绍
2.安装环境
1.安装libevent
yum -y install libevent
2.安装fastdfs
准备三个安装包
[root@localhost fastDFS]# ls -l
total 432
-rw-r--r--. 1 root root 17510 Jan 10 22:20 fastdfs-nginx-module_v1.16.tar.gz
-rw-r--r--. 1 root root 345400 Jan 10 22:20 FastDFS_v5.05.tar.gz
-rw-r--r--. 1 root root 73148 Jan 10 22:20 libfastcommon-1.0.7.tar.gz
[root@localhost fastDFS]#
1.解压libfastcommon
tar -zxvf libfastcommon-1.0.7.tar.gz
make一下
[root@localhost fastDFS]# cd libfastcommon-1.0.7
[root@localhost libfastcommon-1.0.7]# ls
HISTORY INSTALL libfastcommon.spec make.sh README src
[root@localhost libfastcommon-1.0.7]# ./make.sh
如果报错，需要安装
yum install gcc
yum install perl
再make install一下
./make.sh install
执行一下命令
cd /usr/lib64/
ll libfast*
cp libfastcommon.so /usr/lib
2.安装Tracker服务
tar -zxvf FastDFS_v5.05.tar.gz
cd FastDFS...
[make全家桶 略]
cd /usr/bin/
ll fdfs*
进入目录
[root@localhost bin]# cd /etc/fdfs/
[root@localhost fdfs]# ll
total 20
-rw-r--r--. 1 root root 1461 Jan 10 22:44 client.conf.sample
-rw-r--r--. 1 root root 7829 Jan 10 22:44 storage.conf.sample
-rw-r--r--. 1 root root 7102 Jan 10 22:44 tracker.conf.sample
[root@localhost fdfs]# ll fdfs
参考以下命令复制文件
[root@localhost fdfs]# cd ~
[root@localhost ~]# cd fastDFS/
[root@localhost fastDFS]# cd FastDFS
[root@localhost FastDFS]# ls
client conf fastdfs.spec init.d make.sh README.md stop.sh test
common COPYING-3_0.txt HISTORY INSTALL php_client restart.sh storage tracker
[root@localhost FastDFS]# cd conf/
[root@localhost conf]# ls
anti-steal.jpg client.conf http.conf mime.types storage.conf storage_ids.conf tracker.conf
[root@localhost conf]# cp * /etc/fdfs/
配置tracker服务
vim tracker.conf
修改以下内容
# the base path to store data and log files
base_path=/fastdfs/tracker
创建目录
cd /
mkdir /fastdfs/tracker -p
#创建storage文件夹
cd fastdfs/
mkdir storage
mkdir client
启动tracker服务
cd /usr/bin/
#启动
fdfs_trackerd /etc/fdfs/tracker.conf
#重启服务
#fdfs_trackerd /etc/fdfs/tracker.conf restart
3.安装storage服务
修改配置文件
cd /etc/fdfs/
vi storage.conf
修改以下
# the base path to store data and log files
base_path=/fastdfs/storage
修改图片路径
# store_path#, based 0, if store_path0 not exists, it's value is base_path
# the paths must be exist
store_path0=/fastdfs/storage
#store_path1=/home/yuqing/fastdfs2
修改tracker的配置
这里写服务器ip即可
# tracker_server can ocur more than once, and tracker_server format is
# "host:port", host can be hostname or ip address
tracker_server=192.168.232.129:22122
修改组名
# in this case, use_storage_id must set to true in tracker.conf,
# and storage_ids.conf must be configed correctly.
group_name=cxl
启动storage
fdfs_storaged /etc/fdfs/storage.conf
查看进程
ps aux|grep stroage
3.使用client上传图片
cd /usr/fdfs/
vi client.conf
修改内容
# the base path to store log files
base_path=/fastdfs/client
# tracker_server can ocur more than once, and tracker_server format is
# "host:port", host can be hostname or ip address
tracker_server=192.168.232.129:22122
启动
cd /usr/bin/
fdfs_test /etc/fdfs/client.conf upload
# 这里会提示报错
上传一张用来模拟的图片
上传
/usr/bin/fdfs_test /etc/fdfs/client.conf upload 1547195788\(1\).jpg
会返回一堆东西
This is FastDFS client test program v5.05
Copyright (C) 2008, Happy Fish / YuQing
FastDFS may be copied only under the terms of the GNU General
Public License V3, which may be found in the FastDFS source kit.
Please visit the FastDFS Home Page http://www.csource.org/
for more detail.
[2019-01-10 23:12:44] DEBUG - base_path=/fastdfs/client, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0, use_connection_pool=0, g_connection_pool_max_idle_time=3600s, use_storage_id=0, storage server id count: 0
tracker_query_storage_store_list_without_group:
server 1. group_name=, ip_addr=192.168.232.129, port=23000
group_name=cxl, ip_addr=192.168.232.129, port=23000
storage_upload_by_filename
group_name=cxl, remote_filename=M00/00/00/wKjogVw4F7yAOXOjAAHNdSm0V_0973.jpg
source ip address: 192.168.232.129
file timestamp=2019-01-10 23:12:44
file size=118133
file crc32=699684861
example file url: http://192.168.232.129/cxl/M00/00/00/wKjogVw4F7yAOXOjAAHNdSm0V_0973.jpg
storage_upload_slave_by_filename
group_name=cxl, remote_filename=M00/00/00/wKjogVw4F7yAOXOjAAHNdSm0V_0973_big.jpg
source ip address: 192.168.232.129
file timestamp=2019-01-10 23:12:44
file size=118133
file crc32=699684861
example file url: http://192.168.232.129/cxl/M00/00/00/wKjogVw4F7yAOXOjAAHNdSm0V_0973_big.jpg
[root@localhost ~]#
可以看到地址是
remote_filename=M00/00/00/wKjogVw4F7yAOXOjAAHNdSm0V_0973_big.jpg
如果想看到，需要配置nginx
图片存哪了？
4.配置nginx
1.解压fastdfs-nginx的压缩包
tar -zxvf fastdfs-nginx-module_v1.16.tar.gz
2.修改config配置文件
[root@localhost fastDFS]# cd fastdfs-nginx-module
[root@localhost fastdfs-nginx-module]# cd src/
[root@localhost src]# ls
common.c common.h config mod_fastdfs.conf ngx_http_fastdfs_module.c
[root@localhost src]# vim config
删除三个local
3.安装nginx
1.安装基本环境
yum install gcc-c++
yum install pcre pcre-devel
yum install zlib zlib-devel
yum install openssl
2.解压nginx
tar zxvf nginx-1.15.8.tar.gz
3.配置nginx
./configure \
--prefix=/usr/local/nginx \
--pid-path=/var/run/nginx/nginx.pid \
--lock-path=/var/lock/nginx.lock \
--error-log-path=/var/log/nginx/error.log \
--http-log-path=/var/log/nginx/access.log \
--with-http_gzip_static_module \
--http-client-body-temp-path=/var/temp/nginx/client \
--http-proxy-temp-path=/var/temp/nginx/proxy \
--http-fastcgi-temp-path=/var/temp/nginx/uwsgi \
--http-scgi-temp-path=/var/temp/nginx/scgi \
--add-module= {{{你的fastdfs-nginx-module的路径}}}
make一下
make
make install
如果make失败
请参照 https://blog.csdn.net/csdnhadoop/article/details/51112430
再configxxx一下，再make
make好了后，我们需要一个配置文件
复制过去
[root@localhost src]# cp mod_fastdfs.conf /etc/fdfs/
[root@localhost src]# cd /etc/fdfs/
修改配置文件
vi mod_fastdfs.com
修改目录路径
# the base path to store log files
base_path=/fastdfs/tmp
创建目录路径
继续修改配置文件
修改tracker
# valid only when load_fdfs_parameters_from_tracker is true
tracker_server=192.168.232.129:22122
# the group name of the local storage server
group_name=cxl
# default value is false
url_have_group_name = true
配置nginx
vi nginx.conf
server{
listen 88;
server_name 192.168.232.129;
location /cxl/M00 {
ngx_fastdfs_module;
}
}
启动nginx
cd sbin/
mkdir /var/temp/nginx -p
如果这一步出错了
请安装
yum group install Development Tools
后重新执行 进入 fastdfs-nginx-module 目录后执行make,makeInstall后重启，发现nginx日志没错了！
启动nginx
./nginx
关闭防火墙即可
再配置配置文件
cd /etc/fdfs/
ls
vi mod_fastdfs.conf
store_path0=/fastdfs/storage
/usr/bin/fdfs_trackerd /etc/fdfs/tracker.conf restart
/usr/bin/fdfs_storaged /etc/fdfs/storage.conf restart
./nginx -s reload
访问页面显示图片即可
2.FastDFS整合SpringBoot
依赖
<dependency>
<groupId>com.github.tobatogroupId>
<artifactId>fastdfs-clientartifactId>
<version>1.26.2version>
dependency>
<dependency>
<groupId>org.springframeworkgroupId>
<artifactId>spring-testartifactId>
dependency>
fastdfs启动类
package com.imooc;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.EnableMBeanExport;
import org.springframework.context.annotation.Import;
import org.springframework.jmx.support.RegistrationPolicy;
import com.github.tobato.fastdfs.FdfsClientConfig;
/**
* 导入FastDFS-Client组件
*
* @author tobato
*
*/
@Configuration
@Import(FdfsClientConfig.class)
// 解决jmx重复注册bean的问题
@EnableMBeanExport(registration = RegistrationPolicy.IGNORE_EXISTING)
public class FastdfsImporter {
// 导入依赖组件
}
配置文件
fdfs.soTimeout=1501
fdfs.connectTimeout=601
fdfs.thumbImage.width=80
fdfs.thumbImage.height=80
fdfs.trackerList[0]=192.168.1.70:22122
客户端工具类
package com.imooc.utils;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.nio.charset.Charset;
import org.apache.commons.io.FilenameUtils;
import org.apache.commons.lang3.StringUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import org.springframework.web.multipart.MultipartFile;
import com.github.tobato.fastdfs.domain.StorePath;
import com.github.tobato.fastdfs.exception.FdfsUnsupportStorePathException;
import com.github.tobato.fastdfs.service.FastFileStorageClient;
@Component
public class FastDFSClient {
@Autowired
private FastFileStorageClient storageClient;
// @Autowired
// private AppConfig appConfig; // 项目参数配置
/**
* 上传文件
*
* @param file
* 文件对象
* @return 文件访问地址
* @throws IOException
*/
public String uploadFile(MultipartFile file) throws IOException {
StorePath storePath = storageClient.uploadFile(file.getInputStream(), file.getSize(),
FilenameUtils.getExtension(file.getOriginalFilename()), null);
return storePath.getPath();
}
public String uploadFile2(MultipartFile file) throws IOException {
StorePath storePath = storageClient.uploadImageAndCrtThumbImage(file.getInputStream(), file.getSize(),
FilenameUtils.getExtension(file.getOriginalFilename()), null);
return storePath.getPath();
}
public String uploadQRCode(MultipartFile file) throws IOException {
StorePath storePath = storageClient.uploadFile(file.getInputStream(), file.getSize(),
"png", null);
return storePath.getPath();
}
public String uploadFace(MultipartFile file) throws IOException {
StorePath storePath = storageClient.uploadImageAndCrtThumbImage(file.getInputStream(), file.getSize(),
"png", null);
return storePath.getPath();
}
public String uploadBase64(MultipartFile file) throws IOException {
StorePath storePath = storageClient.uploadImageAndCrtThumbImage(file.getInputStream(), file.getSize(),
"png", null);
return storePath.getPath();
}
/**
* 将一段字符串生成一个文件上传
*
* @param content
* 文件内容
* @param fileExtension
* @return
*/
public String uploadFile(String content, String fileExtension) {
byte[] buff = content.getBytes(Charset.forName("UTF-8"));
ByteArrayInputStream stream = new ByteArrayInputStream(buff);
StorePath storePath = storageClient.uploadFile(stream, buff.length, fileExtension, null);
return storePath.getPath();
}
// 封装图片完整URL地址
// private String getResAccessUrl(StorePath storePath) {
// String fileUrl = AppConstants.HTTP_PRODOCOL + appConfig.getResHost() + ":" + appConfig.getFdfsStoragePort()
// + "/" + storePath.getFullPath();
// return fileUrl;
// }
/**
* 删除文件
*
* @param fileUrl
* 文件访问地址
* @return
*/
public void deleteFile(String fileUrl) {
if (StringUtils.isEmpty(fileUrl)) {
return;
}
try {
StorePath storePath = StorePath.praseFromUrl(fileUrl);
storageClient.deleteFile(storePath.getGroup(), storePath.getPath());
} catch (FdfsUnsupportStorePathException e) {
e.getMessage();
}
}
}
File文件工具类
package com.imooc.utils;
import java.io.BufferedOutputStream;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.HttpURLConnection;
import java.net.URL;
import org.springframework.mock.web.MockMultipartFile;
import org.springframework.stereotype.Service;
import org.springframework.util.Base64Utils;
import org.springframework.web.multipart.MultipartFile;
@Service
public class FileUtils {
/**
* 根据url拿取file
*
* @param url
* @param suffix
* 文件后缀名
*/
public static File createFileByUrl(String url, String suffix) {
byte[] byteFile = getImageFromNetByUrl(url);
if (byteFile != null) {
File file = getFileFromBytes(byteFile, suffix);
return file;
} else {
return null;
}
}
/**
* 根据地址获得数据的字节流
*
* @param strUrl
* 网络连接地址
* @return
*/
private static byte[] getImageFromNetByUrl(String strUrl) {
try {
URL url = new URL(strUrl);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
conn.setConnectTimeout(5 * 1000);
InputStream inStream = conn.getInputStream();// 通过输入流获取图片数据
byte[] btImg = readInputStream(inStream);// 得到图片的二进制数据
return btImg;
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
/**
* 从输入流中获取数据
*
* @param inStream
* 输入流
* @return
* @throws Exception
*/
private static byte[] readInputStream(InputStream inStream) throws Exception {
ByteArrayOutputStream outStream = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
int len = 0;
while ((len = inStream.read(buffer)) != -1) {
outStream.write(buffer, 0, len);
}
inStream.close();
return outStream.toByteArray();
}
// 创建临时文件
private static File getFileFromBytes(byte[] b, String suffix) {
BufferedOutputStream stream = null;
File file = null;
try {
file = File.createTempFile("pattern", "." + suffix);
System.out.println("临时文件位置：" + file.getCanonicalPath());
FileOutputStream fstream = new FileOutputStream(file);
stream = new BufferedOutputStream(fstream);
stream.write(b);
} catch (Exception e) {
e.printStackTrace();
} finally {
if (stream != null) {
try {
stream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
return file;
}
public static MultipartFile createImg(String url) {
try {
// File转换成MutipartFile
File file = FileUtils.createFileByUrl(url, "jpg");
FileInputStream inputStream = new FileInputStream(file);
MultipartFile multipartFile = new MockMultipartFile(file.getName(), inputStream);
return multipartFile;
} catch (IOException e) {
e.printStackTrace();
return null;
}
}
public static MultipartFile fileToMultipart(String filePath) {
try {
// File转换成MutipartFile
File file = new File(filePath);
FileInputStream inputStream = new FileInputStream(file);
MultipartFile multipartFile = new MockMultipartFile(file.getName(), "png", "image/png", inputStream);
return multipartFile;
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
return null;
}
}
public static void main(String[] args) {
// WebFileUtils.createFileByUrl("http://122.152.205.72:88/group1/M00/00/01/CpoxxFr7oIaAZ0rOAAC0d3GKDio580.png",
// "png");
// WebFileUtils.createImg("http://122.152.205.72:88/group1/M00/00/01/CpoxxFr7oIaAZ0rOAAC0d3GKDio580.png");
}
public static boolean base64ToFile(String filePath, String base64Data) throws Exception {
String dataPrix = "";
String data = "";
if(base64Data == null || "".equals(base64Data)){
return false;
}else{
String [] d = base64Data.split("base64,");
if(d != null && d.length == 2){
dataPrix = d[0];
data = d[1];
}else{
return false;
}
}
// 因为BASE64Decoder的jar问题，此处使用spring框架提供的工具包
byte[] bs = Base64Utils.decodeFromString(data);
// 使用apache提供的工具类操作流
org.apache.commons.io.FileUtils.writeByteArrayToFile(new File(filePath), bs);
return true;
}
}
整合完毕！
写contoller
/**
* @Description: 上传用户头像
*/
@PostMapping("/uploadFaceBase64")
public IMoocJSONResult uploadFaceBase64(@RequestBody UsersBO userBO) throws Exception {
// 获取前端传过来的base64字符串, 然后转换为文件对象再上传
String base64Data = userBO.getFaceData();
String userFacePath = "C:\\" + userBO.getUserId() + "userface64.png";
FileUtils.base64ToFile(userFacePath, base64Data);
// 上传文件到fastdfs
MultipartFile faceFile = FileUtils.fileToMultipart(userFacePath);
String url = fastDFSClient.uploadBase64(faceFile);
System.out.println(url);
// "dhawuidhwaiuh3u89u98432.png"
// "dhawuidhwaiuh3u89u98432_80x80.png"
// 获取缩略图的url
String thump = "_80x80.";
String arr[] = url.split("\\.");
String thumpImgUrl = arr[0] + thump + arr[1];
// 更细用户头像
Users user = new Users();
user.setId(userBO.getUserId());
user.setFaceImage(thumpImgUrl);
user.setFaceImageBig(url);
Users result = userService.updateUserInfo(user);
return IMoocJSONResult.ok(result);
}