Milvus向量数据库安装手册

目录

一、前言

二、安装

2.1、Docker的安装

2.1.1、解压和配置目录

2.1.2、配置docker环境变量

2.1.3、添加docker服务启动脚本

2.1.4、修改docker镜像源地址

2.1.5、测试docker是否能正常拉取镜像

2.1.6、docker-compose安装

2.2、Python的安装

2.2.1、配置python环境变量

2.3、Milvus的安装

---

一、前言

本文主要是介绍Milvus向量数据库的安装，以及在安装过程中遇到的问题和解决方法。

二、安装

本次安装Milvus向量数据库是基于docker环境进行安装的，所以操作系统的内核版本至少为3.10，否则无法利用docker进行安装。

Milvus向量数据库分为gpu版本和内存版本，一般使用的话内存版本即可，要求操作系统的内存不能低于8G、CPU不低于2C。

2.1、Docker的安装

首先从Index of linux/static/stable/x86_64/ 地址下载docker安装包，推荐使用V20.10以上的版本，本次自己采用的是docker-20.10.6.tgz。

把下载下来的docker安装包下载到自己的服务器对应目录，该目录和安装目录可以根据自己的实际要求进行自定义。

2.1.1、解压和配置目录

[root@Centos7 milvus]# tar -xvf docker-20.10.6.tgz -C /app

[root@Centos7 milvus]# cd /app/docker

[root@Centos7 milvus]# mkdir bin

[root@Centos7 milvus]# mv * ./bin

[root@Centos7 milvus]# mkdir . /data/lib/docker

bin：代表可执行程序的目录，出于好管理，自己单独创建了一个bin目录，并且把docker的可执行文件放到了该目录，可根据自己的实际情况进行配置。

/app/docker/data/lib/docker：是docker引擎在Linux系统中存储docker数据的目录，它包含了docker引擎的运行时数据、容器镜像、容器卷等相关文，默认配置的目录是/var/lib/docker，为了避免该目录引发引起空间不足问题，特意更改了数据存放目录，可根据自已实际情况进行更改。

2.1.2、配置docker环境变量

[root@Centos7 milvus]# vi /etc/profile.d/docker.sh

添加以下内容：

DOCKER_HOME=/app/docker

PATH=$PATH:$DOCKER_HOME/bin

export PATH DOCKER_HOME

[root@Centos7 milvus]# source /etc/profile.d/docker.sh

2.1.3、添加docker服务启动脚本

[root@Centos7 milvus]# vi /usr/lib/systemd/system/docker.service

添加以下内容：

[Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com After=network-online.target firewalld.service Wants=network-online.target [Service] Type=notify #ExecStart=/app/docker/dockerd ExecStart=/app/docker/bin/dockerd --graph /app/docker/data/lib/docker --userland-proxy-path=/app/docker/bin/docker-proxy ExecReload=/bin/kill -s HUP $MAINPID LimitNOFILE=infinity LimitNPROC=infinity TimeoutStartSec=0 Delegate=yes KillMode=process Restart=on-failure StartLimitBurst=3 StartLimitInterval=60s [Install] WantedBy=multi-user.target

[root@Centos7 milvus]# systemctl daemon-reload

[root@Centos7 milvus]# systemctl start docker.service

如果没有启动启来，并且用“journalctl -xe”查看报错日志时发现有如下报错信息：

Failed to start containerd: exec: "containerd": executable file not found in $PATH

是因为少了“containerd”的包导致启动失败，需要安装containerd.io-x.x.x-x.x.e17.x86_64.rpm，需要安装该rpm包，该包下载地址：Index of linux/centos/7/x86_64/stable/Packages/。

安装前一定再检查一下以下的包有没有安装。

[root@Centos7 system]# rpm -qa|grep libseccomp

libseccomp-2.3.1-4.el7.x86_64

[root@Centos7 system]# rpm -qa|grep libtool

libtool-2.4.2-22.el7_3.x86_64

libtool-ltdl-2.4.2-22.el7_3.x86_64

[root@Centos7 system]# rpm -qa|grep lz4

lz4-1.8.3-1.el7.x86_64

如果没有安装的话，需要先安装以上3个包之后，再安装containerd.io的rpm包。

[root@Centos7 milvus]# rpm -ivh containerd.io-1.3.7-3.1.el7.x86_64.rpm

[root@Centos7 milvus]# systemctl start containerd.service

[root@Centos7 milvus]# systemctl start docker.service

[root@Centos7 milvus]# systemctl enable containerd.service

[root@Centos7 milvus]# systemctl enable docker.service

[root@Centos7 milvus]# docker version

[root@Centos7 app]# docker version Client:  Version:           20.10.6  API version:       1.41  Go version:        go1.13.15  Git commit:        370c289  Built:             Fri Apr  9 22:42:10 2021  OS/Arch:           linux/amd64  Context:           default  Experimental:      true Server: Docker Engine - Community  Engine:   Version:          20.10.6   API version:      1.41 (minimum version 1.12)   Go version:       go1.13.15   Git commit:       8728dd2   Built:            Fri Apr  9 22:46:14 2021   OS/Arch:          linux/amd64   Experimental:     false  containerd:   Version:          1.3.7   GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175  runc:   Version:          1.0.0-rc10   GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd

2.1.4、修改docker镜像源地址

如果不修改docker镜像源的话，也可以拉取镜像，但由于默认是国外的镜像源，在拉取的过程中速度有可能会比较慢，所以要把镜像源更改为国内的地址，提高拉取速度。

[root@Centos7 milvus]# vi /etc/docker/daemon.json

添加如下信息：

{ "registry-mirrors": ["http://hub-mirror.c.163.com","https://registry.docker-cn.com","https://docker.mirrors.ustc.edu.cn"] }

地址可以添加一个，也可以添加多个，添加完成之后，需要重启docker.service，并且用docker info查看修改的配置是否生效。

[root@Centos7 milvus]# systemctl restart docker.service

[root@Centos7 milvus]# docker info

[root@Centos7 app]# docker info Client:  Context:    default  Debug Mode: false Server:  Containers: 4   Running: 3   Paused: 0   Stopped: 1  Images: 4  Server Version: 20.10.6  Storage Driver: overlay2   Backing Filesystem: xfs   Supports d_type: true   Native Overlay Diff: true   userxattr: false  Logging Driver: json-file  Cgroup Driver: cgroupfs  Cgroup Version: 1  Plugins:   Volume: local   Network: bridge host ipvlan macvlan null overlay   Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog  Swarm: inactive  Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc  Default Runtime: runc  Init Binary: docker-init  containerd version: 8fba4e9a7d01810a393d5d25a3621dc101981175  runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd  init version: N/A (expected: de40ad007797e0dcd8b7126f27bb87401d224240)  Security Options:   seccomp    Profile: default  Kernel Version: 3.10.0-1160.el7.x86_64  Operating System: CentOS Linux 7 (Core)  OSType: linux  Architecture: x86_64  CPUs: 2  Total Memory: 9.607GiB  Name: Centos7.9  ID: EK4A:NN53:KMFM:IH7C:4WL7:OPDJ:ZMML:GCXL:XZCP:SMFE:VMWU:K26G  Docker Root Dir: /app/docker/data/lib/docker  Debug Mode: false  Registry: https://index.docker.io/v1/  Labels:  Experimental: false  Insecure Registries:   127.0.0.0/8  Registry Mirrors:   http://hub-mirror.c.163.com/   https://registry.docker-cn.com/   https://docker.mirrors.ustc.edu.cn/  Live Restore Enabled: false  Product License: Community Engine WARNING: IPv4 forwarding is disabled WARNING: bridge-nf-call-iptables is disabled WARNING: bridge-nf-call-ip6tables is disabled

标红部分说明配置的daemon.json的镜像源生效了。

2.1.5、测试docker是否能正常拉取镜像

[root@Centos7 docker]# docker pull nginx Using default tag: latest latest: Pulling from library/nginx a378f10b3218: Pull complete 5b5e4b85559a: Pull complete 508092f60780: Pull complete 59c24706ed13: Pull complete 1a8747e4a8f8: Pull complete ad85f053b4ed: Pull complete 3000e3c97745: Pull complete error adding content digest to lease: sha256: 35e16351c585c0721f099bc638423cedc41efcd0f2da986f89ab441989e1768f: unknown method AddResource: not implemented

如果出现以上报错，有可能是containerd.service和docker.service服务需要重启进行解决，但重启之后还是报出同样的错，有可能是containerd.io的rpm包版本和所安装的docker版本不兼容造成，需要更改containerd.io的rpm包的版本，并重新安装，本次采用的是containerd.io-1.3.7-3.1.el7.x86_64.rpm。

更换containerd.io包的版本后，docker可以正常拉取镜像文件，如下所示：

[root@Centos7 docker]# docker pull nginx Using default tag: latest latest: Pulling from library/nginx a378f10b3218: Pull complete 5b5e4b85559a: Pull complete 508092f60780: Pull complete 59c24706ed13: Pull complete 1a8747e4a8f8: Pull complete ad85f053b4ed: Pull complete 3000e3c97745: Pull complete Digest: sha256:35e16351c585c0721f099bc638423cedc41efcd0f2da986f89ab441989e1768f Status: Downloaded newer image for nginx:latest docker.io/library/nginx:latest

2.1.6、docker-compose安装

下载地址：Releases · docker/compose · GitHub，可以下载最新版本，下载完成之后，把该文件改名放到docker的bin目录即可。

[root@Centos7 milvus]# cp docker-compose-linux-x86_64 /app/docker/bin/docker-compose

[root@Centos7 milvus]# source /etc/profile.d/docker.sh

[root@Centos7 milvus]# docker-compose -v

Docker Compose version v2.23.0

2.2、Python的安装

本次采用python3.10以上的版本进行安装，版本太低的话，有可能以后模型与向量结合时会出现未知问题，所以采用python3.10以上的版本进行安装。

下载地址：Download Python | Python.org 本次采用的是3.10.13的版本包Python-3.10.13.tgz。

[root@Centos7 milvus]# tar -xvf Python-3.10.13.tgz

[root@Centos7 milvus]# cd Python-3.10.13

[root@Centos7 milvus]# ./configure –prefix=/app/docker310

[root@Centos7 milvus]# make –j2

[root@Centos7 milvus]# make install

make -j2：主要是为了提高编译速度，其是2是代表cpu的核心个数，一般是核心个数的2倍，可采用可采用，如果不采用，直接执行make即可。

注：如果在编译安装的过程中有报错，有可能是操作系统缺少一些开发包造成，可以根据报错信息用yum安装一下对应有开发包即可，如：缺少zlib的开发包，直接安装zlib-devel对应包即可。

2.2.1、配置python环境变量

[root@Centos7 milvus]# vi /etc/profile.d/python.sh

添加如下内容：

PYTHON_HOME=/app/python310

PATH=$PATH:$PYTHON_HOME/bin

export PATH PYTHON_HOME

[root@Centos7 milvus]# source /etc/profile.d/python.sh

[root@Centos7 milvus]# python3 -V

Python 3.10.13

2.3、Milvus的安装

下载地址：Releases · milvus-io/milvus · GitHub 下载自己想用的版本即可，本次采用的是2.3.2的版本，下载时有2种类型可供选择，gpu版本和内存版本，本次采用的是内存版本，把下载好的milvus-standalone-docker-compose.yml文件改名放到自己规划好的目录下，改成docker-compose.yml执行安装即可。

[root@Centos7 milvus]# cp milvus-standalone-docker-compose.yml /app/milvus

[root@Centos7 milvus]# docker-compse up

[root@Centos7 milvus]# docker-compse up [+] Running 23/23  ✔ minio 6 layers [⣿⣿⣿⣿⣿⣿]      0B/0B      Pulled                                                                                                      302.2s    ✔ c7e856e03741 Pull complete                                                                                                                        117.1s    ✔ c1ff217ec952 Pull complete                                                                                                                         96.6s    ✔ b12cc8972a67 Pull complete                                                                                                                        110.3s    ✔ 4324e307ea00 Pull complete                                                                                                                        127.9s    ✔ 152089595ebc Pull complete                                                                                                                        127.7s    ✔ 05f217fb8612 Pull complete                                                                                                                        184.1s  ✔ standalone 7 layers [⣿⣿⣿⣿⣿⣿⣿]      0B/0B      Pulled                                                                                                404.1s    ✔ d5fd17ec1767 Pull complete                                                                                                                         69.5s    ✔ 50e2e2559689 Pull complete                                                                                                                         57.5s    ✔ 9997e6050d5b Pull complete                                                                                                                        104.0s    ✔ 7061d965979a Pull complete                                                                                                                         69.1s    ✔ 3db6d0e6aa97 Pull complete                                                                                                                        292.8s    ✔ 1f27396f6efc Pull complete                                                                                                                         82.2s    ✔ fe556ec02776 Pull complete                                                                                                                         99.2s  ✔ etcd 7 layers [⣿⣿⣿⣿⣿⣿⣿]      0B/0B      Pulled                                                                                                      159.5s    ✔ dbba69284b27 Pull complete                                                                                                                         75.1s    ✔ 270b322b3c62 Pull complete                                                                                                                         22.0s    ✔ 7c21e2da1038 Pull complete                                                                                                                         19.9s    ✔ cb4f77bfee6c Pull complete                                                                                                                         38.1s    ✔ e5485096ca5d Pull complete                                                                                                                         32.2s    ✔ 3ea3736f61e1 Pull complete                                                                                                                         46.9s    ✔ 1e815a2c4f55 Pull complete                                                                                                                         52.9s [+] Building 0.0s (0/0)                                                                                                                        docker:default [+] Running 4/4  ✔ Network milvus               Created                                                                                                                  0.2s  ✔ Container milvus-minio       Created                                                                                                                  3.1s  ✔ Container milvus-etcd        Created                                                                                                                  3.1s  ✔ Container milvus-standalone  Created

注：初次使用安装环境，有可能会卡住，或下载安装一半就会断掉，此时可以尝试重启docker.service服务后，再次执行docker-compose up。

此时安装过程比较慢，请耐心等待即可。

安装完成之后会直接进行启动，docker-compose up是前台启动，可以使用docker-compose down停掉后，用docker-compose up -d后台启动，启动完成之后，可以使用docker-compose ps 进行查看运行状态。

[root@Centos7 milvus]# docker-compose ps

[root@Centos7 milvus]# docker-compose ps NAME                IMAGE                                      COMMAND                   SERVICE      CREATED        STATUS                  PORTS milvus-etcd         quay.io/coreos/etcd:v3.5.5                 "etcd -advertise-cli…"   etcd         25 hours ago   Up 25 hours (healthy)   2379-2380/tcp milvus-minio        minio/minio:RELEASE.2023-03-20T20-16-18Z   "/usr/bin/docker-ent…"   minio        25 hours ago   Up 25 hours (healthy)   0.0.0.0:9000-9001->9000-9001/tcp, :::9000-9001->9000-9001/tcp milvus-standalone   milvusdb/milvus:v2.3.2                     "/tini -- milvus run…"   standalone   25 hours ago   Up 25 hours (healthy)   0.0.0.0:9091->9091/tcp, :::9091->9091/tcp, 0.0.0.0:19530->19530/tcp, :::19530->19530/tcp

注：使用docker-compose命令执行时，一定要在和yml文件同级目录下执行，否则执行时会报出以下错误：

[root@Centos7 app]# docker-compose ps

no configuration file provided: not found