Ollama本地運(yùn)行DeepSeek - Docker方式

安裝Docker

參考：http://www.itdecent.cn/p/62bac0a1060a

Docker支持Nvidia驅(qū)動(dòng)

參考：docker支持nvidia gpu http://www.itdecent.cn/p/ed1e54c2a972

安裝Ollama

# 拉取鏡像
docker pull ollama/ollama:0.5.7

# Docker方式運(yùn)行Ollama
mkdir -p /data/ollama
docker run -d --gpus=all -v /data/ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama:0.5.7

--gpus=all參數(shù)是為了在docker容器內(nèi)可以使用GPU

  • --gpus "device=0": 使用第一個(gè) GPU 設(shè)備
  • --gpus "device=2,3": 使用第三和第四個(gè) GPU 設(shè)備

由于習(xí)慣docker-compose，以docker-compose方式啟動(dòng)的話，上述docker run可以用如下命令代替：

mkdir -p /data/ollama/data
cat > /data/ollama/docker-compose.yml <<"EOF"
services:
  ollama:
    image: ollama/ollama:0.5.7
    container_name: ollama 
    restart: always
    ports:
      - "11434:11434"
    volumes:
      - /data/ollama/data:/root/.ollama
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              # count 與 device_ids 二選一 
              # count指定需要使用的GPU數(shù)量；值為int數(shù)組型或all；
              # device_ids: ["0", "3"]
              count: "all"
              capabilities: ["gpu"]
EOF
# 啟動(dòng)Ollama
docker-compose up -d

Ollama更多環(huán)境變量，可以參考：https://zhuanlan.zhihu.com/p/712642806

運(yùn)行Deepseek方式一：直接運(yùn)行Ollama官方大模型

官方模型：https://ollama.com/search

# 進(jìn)入容器內(nèi)
docker exec -it ollama bash

# -------------以下命令都在容器內(nèi)執(zhí)行-------------
# 運(yùn)行deepseek-r1:7b
ollama run deepseek-r1:7b

效果如下：

[root@bogon ollama]# docker exec -it ollama bash
root@e2809411e208:/# ollama run deepseek-r1:7b
pulling manifest
pulling 96c415656d37... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████ ▏ 4.7 GB/4.7 GB   14 MB/s      0s
pulling 369ca498f347... 100% ▕███████████████████████████████████████████████████████████████████████████████████████████████▏  387 B
pulling 6e4c38e1172f... 100% ▕███████████████████████████████████████████████████████████████████████████████████████████████▏ 1.1 KB
pulling f4d24e9138dd... 100% ▕███████████████████████████████████████████████████████████████████████████████████████████████▏  148 B
pulling 40fb844194b2... 100% ▕███████████████████████████████████████████████████████████████████████████████████████████████▏  487 B
verifying sha256 digest
writing manifest
success
>>> 你是誰(shuí)？
<think>

</think>

您好！我是由中國(guó)的深度求索（DeepSeek）公司開(kāi)發(fā)的智能助手DeepSeek-R1。如您有任何任何問(wèn)題，我會(huì)盡我所能為您提供幫助。

>>> Send a message (/? for help)

運(yùn)行Deepseek方式二：通過(guò)離線gguf文件運(yùn)行大模型

1. 下載gguf文件

文件來(lái)源：

下載方式：

  • wget

wget -b https://www.modelscope.cn/models/unsloth/DeepSeek-R1-Distill-Llama-70B-GGUF/resolve/master/DeepSeek-R1-Distill-Llama-70B-Q4_K_M.gguf

  • modelscope命令（推薦）

參考：http://www.itdecent.cn/p/e06cfe41b7a9?v=1739521146303

2. 編寫(xiě) modelfile


cat > /data/ollama/data/gguf/deepseek-r1-70B-modelfile <<"EOF"
FROM ./DeepSeek-R1-Distill-Llama-70B-Q4_K_M.gguf

# 設(shè)置對(duì)話終止符
PARAMETER stop "<｜begin▁of▁sentence｜>"
PARAMETER stop "<｜end▁of▁sentence｜>"
PARAMETER stop "<｜User｜>"
PARAMETER stop "<｜Assistant｜>"


SYSTEM """
"""

TEMPLATE """{{- if .System }}{{ .System }}{{ end }} 
{{- range $i, $_ := .Messages }} 
{{- $last := eq (len (slice $.Messages $i)) 1}}
{{- if eq .Role "user" }}<｜User｜>{{ .Content }}
{{- else if eq .Role "assistant" }}<｜Assistant｜>{{ .Content }}{{- if not $last }}<｜end▁of▁sentence｜>{{- end }}
{{- end }}
{{- if and $last (ne .Role "assistant") }}<｜Assistant｜>{{- end }} 
{{- end }}"""

EOF

modelfile必須與gguf文件同目錄，否則需要修改modelfile第一行的相對(duì)位置

3. 創(chuàng)建模型

# 通過(guò)gguf創(chuàng)建模型(docker內(nèi)執(zhí)行）
ollama create deepseek-r1-custom:70B -f /root/.ollama/mygguf/deepseek-r1-70B-modelfile

4. 運(yùn)行模型

# docker內(nèi)執(zhí)行
ollama run deepseek-r1-custom:70B

通過(guò)Ollama API 測(cè)試模型

# 通過(guò)ollama api測(cè)試模型
curl http://10.3.6.38:11434/api/chat -d '{
  "model": "deepseek-r1-bidr:70b",
  "messages": [
    {
      "role": "user",   
      "content": "為什么天空是藍(lán)色的?"
    }
  ],
  "stream": true
}'

效果：

[root@bogon ~]# curl http://10.3.6.38:11434/api/chat -d '{
  "model": "deepseek-r1-bidr:70b",
  "messages": [
    {
      "role": "user",
      "content": "為什么天空是藍(lán)色的?"
    }
  ],
  "stream": true
}'
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:57.490403406Z","message":{"role":"assistant","content":"\u003cthink\u003e"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:57.62964314Z","message":{"role":"assistant","content":"\n\n"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:57.782469849Z","message":{"role":"assistant","content":"\u003c/think\u003e"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:57.925424223Z","message":{"role":"assistant","content":"\n\n"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:58.060309984Z","message":{"role":"assistant","content":"天"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:58.209522495Z","message":{"role":"assistant","content":"空"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:58.369515693Z","message":{"role":"assistant","content":"呈"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:58.52581505Z","message":{"role":"assistant","content":"現(xiàn)"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:58.686391838Z","message":{"role":"assistant","content":"藍(lán)"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:58.82169606Z","message":{"role":"assistant","content":"色"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:58.969790941Z","message":{"role":"assistant","content":"是"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:59.105679167Z","message":{"role":"assistant","content":"因?yàn)?},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:59.243647488Z","message":{"role":"assistant","content":"當(dāng)"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:59.380513238Z","message":{"role":"assistant","content":"太"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:59.525637676Z","message":{"role":"assistant","content":"陽(yáng)"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:59.669904633Z","message":{"role":"assistant","content":"光"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:59.813863747Z","message":{"role":"assistant","content":"照"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:25:59.970479134Z","message":{"role":"assistant","content":"射"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:26:00.109944242Z","message":{"role":"assistant","content":"到"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:26:00.242174673Z","message":{"role":"assistant","content":"地球"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:26:00.381833905Z","message":{"role":"assistant","content":"大"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:26:00.51949058Z","message":{"role":"assistant","content":"氣"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:26:00.66205966Z","message":{"role":"assistant","content":"層"},"done":false}
{"model":"deepseek-r1-bidr:70b","created_at":"2025-02-14T08:26:00.798480342Z","message":{"role":"assistant","content":"時(shí)"},"done":false}
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成，瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明：文章內(nèi)容（如有圖片或視頻亦包括在內(nèi)）由作者上傳并發(fā)布，文章內(nèi)容僅代表作者本人觀點(diǎn)，簡(jiǎn)書(shū)系信息發(fā)布平臺(tái)，僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容