Ubuntu20.04部署Ollama

在Ubuntu20.04上面安装完RTX4060的Nvidia Cuda和TensorRT环境后，就开始跑些大模型看看。下面是安装使用Ollama的过程：

安装Ollama：

curl -k https://ollama.com/install.sh | sh

执行上面命令，有如下打印：

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0>>> Downloading ollama...
100 10975 0 10975 0 0 13034 0 --:--:-- --:--:-- --:--:-- 13034
######################################################################## 100.0%#=#=-# #
######################################################################## 100.0%
>>> Installing ollama to /usr/local/bin...
[sudo] password for stxinu:
>>> Creating ollama user...
>>> Adding ollama user to render group...
>>> Adding ollama user to video group...
>>> Adding current user to ollama group...
>>> Creating ollama systemd service...
>>> Enabling and starting ollama service...
Created symlink /etc/systemd/system/default.target.wants/ollama.service → /etc/systemd/system/ollama.service.
>>> NVIDIA GPU installed.

上面安装完Ollama后，我们使用如下命令运行Dolphin Phi试试：

ollama run dolphin-phi

此时会有如下打印：

pulling manifest
pulling 4eca7304a07a... 100% ▕██████████████████████████████████████████████████████▏ 1.6 GB
pulling 876a8d805b60... 100% ▕██████████████████████████████████████████████████████▏  10 KB
pulling a47b02e00552... 100% ▕██████████████████████████████████████████████████████▏  106 B
pulling 8b586b146d99... 100% ▕██████████████████████████████████████████████████████▏   40 B
pulling f02dd72bb242... 100% ▕██████████████████████████████████████████████████████▏   59 B
pulling c87a43ded80f... 100% ▕██████████████████████████████████████████████████████▏  555 B
verifying sha256 digest
writing manifest
removing any unused layers
success
>>>

在上面>>>的后面输入hi，会有如下打印：

>>> hi
Hello! How can I assist you today? Is there something specific that I can help with or any questions you have on mind?

上面是命令行方式操作，Ollama还提供了API方式，安装完成后，就有11434端口服务，使用如下命令可以测试：

curl http://localhost:11434/api/generate -d '{   "model": "dolphin-phi",   "prompt":"Why is the sky blue?" }'

执行后，有如下打印信息：

stxinu@tsi:~$ curl http://localhost:11434/api/generate -d '{   "model": "dolphin-phi",   "prompt":"Why is the sky blue?" }'
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.648261246Z","response":"The","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.659150695Z","response":" sky","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.669102772Z","response":" appears","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.678638043Z","response":" blue","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.688013742Z","response":" due","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.697391064Z","response":" to","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.70675917Z","response":" a","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.716161361Z","response":" phenomenon","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.725568899Z","response":" called","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.734978186Z","response":" Ray","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.74427592Z","response":"leigh","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.753544198Z","response":" scattering","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.762828076Z","response":".","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.772079694Z","response":" Sun","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.78134233Z","response":"light","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.790627727Z","response":",","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.799886023Z","response":" which","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.809156449Z","response":" consists","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.818410707Z","response":" of","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.827692402Z","response":" different","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.836960209Z","response":" colors","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.846242649Z","response":",","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.855525997Z","response":" or","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.864790047Z","response":" wavelengths","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.874099897Z","response":",","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.883398974Z","response":" travels","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.892708458Z","response":" in","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.902019318Z","response":" all","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.911329736Z","response":" directions","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.920652441Z","response":" through","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.929966829Z","response":" the","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.939287523Z","response":" Earth","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.948601582Z","response":"'s","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.957940944Z","response":" atmosphere","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.967272475Z","response":".","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.976607597Z","response":" When","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.986003328Z","response":" this","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:12.995415574Z","response":" sunlight","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.004818067Z","response":" encounters","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.014224135Z","response":" molecules","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.023626645Z","response":" and","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.033022806Z","response":" tiny","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.042436623Z","response":" particles","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.051851313Z","response":",","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.06125314Z","response":" such","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.070662162Z","response":" as","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.080072899Z","response":" dust","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.089489766Z","response":" and","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.0988862Z","response":" water","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.108312265Z","response":" dro","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.117729872Z","response":"plets","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.127151292Z","response":",","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.136571427Z","response":" it","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.146000957Z","response":" sc","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.155457867Z","response":"atters","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.164886966Z","response":" in","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.174308459Z","response":" various","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.183774516Z","response":" directions","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.193198026Z","response":",","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.202633956Z","response":" but","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.212071434Z","response":" the","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.221500109Z","response":" shorter","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.230921159Z","response":" wavelength","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.240347899Z","response":" blues","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.249771992Z","response":" and","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.259198304Z","response":" pur","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.268610987Z","response":"ples","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.278028559Z","response":" are","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.287793105Z","response":" scattered","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.297295198Z","response":" more","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.306772418Z","response":" easily","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.31625506Z","response":" than","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.325737729Z","response":" the","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.335204022Z","response":" longer","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.344678835Z","response":" wavelength","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.354144356Z","response":" red","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.36360919Z","response":"s","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.373098808Z","response":" and","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.382567808Z","response":" oranges","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.392045936Z","response":".","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.401514057Z","response":"\n","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.410980952Z","response":"\n","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.42043318Z","response":"The","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.429899224Z","response":" blue","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.439362575Z","response":" light","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.448827261Z","response":" is","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.458297474Z","response":" scattered","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.467765106Z","response":" at","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.477237135Z","response":" a","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.486707277Z","response":" greater","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.496185639Z","response":" angle","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.505653337Z","response":" compared","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.51512901Z","response":" to","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.524609469Z","response":" the","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.534092731Z","response":" other","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.543567309Z","response":" colors","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.553042086Z","response":".","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.56252223Z","response":" This","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.572012807Z","response":" means","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.58150218Z","response":" that","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.591050554Z","response":" our","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.600605829Z","response":" eyes","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.610140945Z","response":" perceive","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.619691651Z","response":" these","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.629240746Z","response":" blue","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.638789393Z","response":" wavelengths","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.648328977Z","response":" to","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.657881337Z","response":" be","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.667421324Z","response":" coming","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.67697396Z","response":" from","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.686525253Z","response":" all","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.696073585Z","response":" directions","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.705619234Z","response":" in","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.715165338Z","response":" the","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.724705771Z","response":" sky","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.734249621Z","response":",","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.743776455Z","response":" creating","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.753319685Z","response":" the","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.762851143Z","response":" impression","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.77238555Z","response":" of","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.781915773Z","response":" a","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.791449973Z","response":" blue","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.800981635Z","response":" color","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.810523931Z","response":" when","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.820061152Z","response":" we","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.829601622Z","response":" look","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.839144193Z","response":" up","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.848690176Z","response":".","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.858226117Z","response":" The","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.867770101Z","response":" deeper","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.877326217Z","response":" blue","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.886859937Z","response":" h","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.896460016Z","response":"ues","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.906056232Z","response":" seen","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.915663679Z","response":" during","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.92527424Z","response":" sunrise","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.934878711Z","response":" or","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.944495539Z","response":" sunset","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.954102376Z","response":" are","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.963711849Z","response":" due","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.973320897Z","response":" to","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.982924943Z","response":" sunlight","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:13.992577554Z","response":" traveling","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.00217487Z","response":" through","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.011772714Z","response":" more","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.021383715Z","response":" of","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.030984392Z","response":" Earth","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.040590204Z","response":"'s","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.050206455Z","response":" atmosphere","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.059815429Z","response":" and","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.069422648Z","response":" being","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.079043339Z","response":" scattered","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.088662793Z","response":" further","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.098279611Z","response":" away","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.107895081Z","response":" from","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.117505689Z","response":" its","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.127112127Z","response":" source","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.136731561Z","response":".","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.146350312Z","response":"\n","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.155955786Z","response":"\n","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.165562568Z","response":"In","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.175188975Z","response":" summary","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.184802713Z","response":",","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.194411983Z","response":" the","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.204083252Z","response":" sky","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.213744876Z","response":" appears","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.223421994Z","response":" blue","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.233087376Z","response":" because","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.242751938Z","response":" the","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.252416416Z","response":" shorter","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.262085525Z","response":" wavelength","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.271761009Z","response":" blue","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.281409218Z","response":" light","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.291072735Z","response":" is","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.300747906Z","response":" scattered","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.310413251Z","response":" more","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.320066961Z","response":" than","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.329735037Z","response":" other","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.339395078Z","response":" colors","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.349064329Z","response":" by","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.358744735Z","response":" the","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.368410591Z","response":" molecules","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.378079285Z","response":" and","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.387752102Z","response":" particles","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.397423861Z","response":" in","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.407114026Z","response":" the","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.416765388Z","response":" Earth","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.42642699Z","response":"'s","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.436090818Z","response":" atmosphere","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.445750321Z","response":".","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.455406682Z","response":" This","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.465079503Z","response":" effect","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.474762345Z","response":" is","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.484410703Z","response":" known","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.494056171Z","response":" as","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.503716432Z","response":" Ray","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.51340459Z","response":"leigh","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.523097105Z","response":" scattering","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.532786Z","response":".","done":false}
{"model":"dolphin-phi","created_at":"2024-06-16T01:55:14.542509635Z","response":"","done":true,"done_reason":"stop","context":[50296,10057,198,1639,389,44576,11,257,7613,9552,8796,13,50295,198,50296,7220,198,5195,318,262,6766,4171,30,50295,198,50296,562,10167,198,464,6766,3568,4171,2233,284,257,10733,1444,7760,42342,45765,13,3825,2971,11,543,10874,286,1180,7577,11,393,45656,11,17781,287,477,11678,832,262,3668,6,82,8137,13,1649,428,19606,16925,17745,290,7009,13166,11,884,355,8977,290,1660,3102,46916,11,340,629,34387,287,2972,11678,11,475,262,12238,28400,25570,290,1308,2374,389,16830,517,3538,621,262,2392,28400,2266,82,290,48389,13,198,198,464,4171,1657,318,16830,379,257,3744,9848,3688,284,262,584,7577,13,770,1724,326,674,2951,19973,777,4171,45656,284,307,2406,422,477,11678,287,262,6766,11,4441,262,10647,286,257,4171,3124,618,356,804,510,13,383,9211,4171,289,947,1775,1141,40048,393,26428,389,2233,284,19606,11300,832,517,286,3668,6,82,8137,290,852,16830,2252,1497,422,663,2723,13,198,198,818,10638,11,262,6766,3568,4171,780,262,12238,28400,4171,1657,318,16830,517,621,584,7577,416,262,17745,290,13166,287,262,3668,6,82,8137,13,770,1245,318,1900,355,7760,42342,45765,13],"total_duration":3719469144,"load_duration":1779468673,"prompt_eval_count":29,"prompt_eval_duration":43560000,"eval_count":200,"eval_duration":1894307000}

然后配置并重启下ollama服务，不然后面安装的Open WebUI无法访问到，需要修改/etc/systemd/system/ollama.service文件，将如下内容：

Environment="PATH=/home/stxinu/.local/bin:/usr/local/cuda-12.5/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin"

修改为：

Environment="PATH=/home/stxinu/.local/bin:/usr/local/cuda-12.5/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" "OLLAMA_HOST=0.0.0.0"

修改完成后再执行如下命令重启下ollama即可：

sudo systemctl daemon-reload
sudo systemctl restart ollama

根据我们的需求，我们使用如下命令安装上llama3:8b模型：

ollama run llama3:8b

接下来安装Open WebUI，给Ollama套层皮，其适配了对应的接口，用Docker安装即可：

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

上面安装完成后，如果在安装ollama的电脑上可使用http://localhost:3000可以通过网页访问，如果是同一局域网的电脑，则用相应的IP和端口3000去访问即可，然后根据自己实际需求使用即可。第一个注册账号为管理员账号。登录后界面如下：