SPDK (part 2, vagrant)

vagrant

一個(gè)基于ruby的部署虛擬化開發(fā)環(huán)境的工具，用virtualbox作為VIM支撐（即命令行中的provider，目前也支持libvirt，暫未測(cè)試），chef作為自動(dòng)化環(huán)境創(chuàng)建。官方的SPDK代碼中提供的Vagrantfile提供了一個(gè)掛載了NVMe設(shè)備的虛機(jī)，以及對(duì)應(yīng)實(shí)例代碼hello_world，用來幫助開發(fā)者進(jìn)行SPDK開發(fā)環(huán)境的快速搭建。

**upgrade vagrant **

默認(rèn)情況虛機(jī)分配兩個(gè)CPU，4G內(nèi)存?？梢酝ㄟ^環(huán)境變量ENV['SPDK_VAGRANT_VMCPU']和ENV['SPDK_VAGRANT_VMRAM']來進(jìn)行設(shè)置

default option: virtualbox作為provider

  • apt-get install virtualbox

  • apt-get install virtualbox-ext-pack

在spdk/scripts/vagrant下執(zhí)行

  • vagrant up

option 2: libvirt作為provider

  • apt-get install libvirt-bin

  • apt-get install libvirt-dev

  • vagrant up --provider=libvirt

虛機(jī)執(zhí)行完畢后基本的輸出如下：

Ignoring ruby-libvirt-0.7.1 because its extensions are not built. Try: gem pristine ruby-libvirt --version 0.7.1

Bringing machine 'default' up with 'virtualbox' provider...

==> default: Importing base box 'generic/fedora26'...

==> default: Matching MAC address for NAT networking...

==> default: Setting the name of the VM: vagrant_default_1534148841675_59957

==> default: Clearing any previously set network interfaces...

==> default: Preparing network interfaces based on configuration...

default: Adapter 1: nat

default: Adapter 2: hostonly

==> default: Forwarding ports...

default: 22 (guest) => 2222 (host) (adapter 1)

==> default: Running 'pre-boot' VM customizations...

==> default: Booting VM...

==> default: Waiting for machine to boot. This may take a few minutes...

default: SSH address: 127.0.0.1:2222

default: SSH username: vagrant

default: SSH auth method: private key

default:

default: Vagrant insecure key detected. Vagrant will automatically replace

default: this with a newly generated keypair for better security.

default:

default: Inserting generated public key within guest...

default: Removing insecure key from the guest if it's present...

default: Key inserted! Disconnecting and reconnecting using new SSH key...

==> default: Machine booted and ready!

==> default: Checking for guest additions in VM...

==> default: Configuring and enabling network interfaces...

==> default: Running provisioner: file...

ssh進(jìn)入該虛機(jī)，默認(rèn)目錄是/home/vagrant。理論上host的spdk目錄會(huì)共享到虛機(jī)當(dāng)中，避免在虛機(jī)上重新clone，配置和編譯spdk代碼。如果發(fā)現(xiàn)該目錄是空的，需要重新進(jìn)行一遍host的clone，配置和編譯的步驟。

  • vagrant ssh

Ignoring ruby-libvirt-0.7.1 because its extensions are not built. Try: gem pristine ruby-libvirt --version 0.7.1

執(zhí)行示例代碼（hello world）

以下是虛機(jī)內(nèi)操作，可以看到NVMe devices已經(jīng)掛載可用

[vagrant@localhost ~]$ pwd

/home/vagrant

[vagrant@localhost spdk]$ sudo HUGEMEM=2048 scripts/setup.sh

[vagrant@localhost spdk]$ ./scripts/setup.sh status

Hugepages

node hugesize free / total

node0 2048kB 1024 / 1024

NVMe devices

BDF Numa Node Driver name Device name

**0000:00:0e.0 0 uio_pci_generic **-

I/OAT DMA

BDF Numa Node Driver Name

virtio

BDF Numa Node Driver Name Device Name

[vagrant@localhost spdk]$ sudo examples/nvme/hello_world/hello_world

Starting SPDK v18.10-pre / DPDK 18.05.0 initialization...

[ DPDK EAL parameters: hello_world -c 0x1 --legacy-mem --file-prefix=spdk0 --base-virtaddr=0x200000000000 --proc-type=auto ]

EAL: Detected 2 lcore(s)

EAL: Detected 1 NUMA nodes

EAL: Auto-detected process type: PRIMARY

EAL: Multi-process socket /var/run/dpdk/spdk0/mp_socket

EAL: Probing VFIO support...

EAL: WARNING! Base virtual address hint (0x20040008f000 != 0x7f69b830d000) not respected!

EAL: This may cause issues with mapping memory into secondary processes

EAL: WARNING! Base virtual address hint (0x2008000f0000 != 0x7f69b82ac000) not respected!

EAL: This may cause issues with mapping memory into secondary processes

EAL: WARNING! Base virtual address hint (0x200c00151000 != 0x7f69b824b000) not respected!

EAL: This may cause issues with mapping memory into secondary processes

Initializing NVMe Controllers

EAL: PCI device 0000:00:0e.0 on NUMA socket 0

EAL: probe driver: 80ee:4e56 spdk_nvme

Attaching to 0000:00:0e.0

Attached to 0000:00:0e.0

Using controller ORCL-VBOX-NVME-VER12 (VB1234-56789 ) with 1 namespaces.

Namespace ID: 1 size: 1GB

Initialization complete.

INFO: using host memory buffer for IO

Hello world!

執(zhí)行示例代碼（FIO）

退出到重啟VM釋放大頁內(nèi)存

  • vagrant reload

  • vagrant ssh

部署FIO

  • git clone https://github.com/axboe/fio

  • cd fio && git checkout fio-3.3

  • make

  • cd <SPDK folder>

  • ./configure --with-fio=/home/vagrant/spdk/fio/

  • vim examples/nvme/fio_plugin/example_config.fio 并加入最后一行配置指定PCI地址

image
  • sudo LD_PRELOAD=/home/vagrant/spdk/examples/nvme/fio_plugin/fio_plugin /home/vagrant/spdk/fio/fio /home/vagrant/spdk/examples/nvme/fio_plugin/example_config.fio
image

執(zhí)行普通FIO，添加配置文件中size=10G

  • sudo /home/vagrant/spdk/fio/fio /home/vagrant/spdk/examples/nvme/fio_plugin/example_config.fio
image

更多參見https://github.com/spdk/spdk/tree/master/examples/nvme/fio_plugin

執(zhí)行示例代碼（perf****）

參見http://www.spdk.io/doc/nvme.html

注意執(zhí)行時(shí)需要加上sudo確保root權(quán)限執(zhí)行

[vagrant@localhost spdk]$ sudo ./examples/nvme/perf/perf -q 128 -s 4096 -w randread -r 'trtype:PCIe traddr:0000:00:0e.0' -t 300

Starting SPDK v18.10-pre / DPDK 18.05.0 initialization...

[ DPDK EAL parameters: perf -c 0x1 --legacy-mem --file-prefix=spdk_pid4352 ]

EAL: Detected 2 lcore(s)

EAL: Detected 1 NUMA nodes

EAL: Multi-process socket /var/run/dpdk/spdk_pid4352/mp_socket

EAL: Probing VFIO support...

Initializing NVMe Controllers

EAL: PCI device 0000:00:0e.0 on NUMA socket 0

EAL: probe driver: 80ee:4e56 spdk_nvme

Attaching to NVMe Controller at 0000:00:0e.0

Attached to NVMe Controller at 0000:00:0e.0 [80ee:4e56]

Associating ORCL-VBOX-NVME-VER12 (VB1234-56789 ) with lcore 0

Initialization complete. Launching workers.

Starting thread on core 0

========================================================

                                                                                        Latency(us)

Device Information : IOPS MB/s Average min max

ORCL-VBOX-NVME-VER12 (VB1234-56789 ) from core 0: 62271.46 243.25 2055.47 982.67 14446.51

========================================================

Total : 62271.46 243.25 2055.47 982.67 14446.51

最后編輯于 ：
?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成，瀏覽時(shí)請(qǐng)結(jié)合常識(shí)與多方信息審慎甄別。
平臺(tái)聲明：文章內(nèi)容（如有圖片或視頻亦包括在內(nèi)）由作者上傳并發(fā)布，文章內(nèi)容僅代表作者本人觀點(diǎn)，簡(jiǎn)書系信息發(fā)布平臺(tái)，僅提供信息存儲(chǔ)服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

  • 需要提前了解的概念 Linux內(nèi)核驅(qū)動(dòng)： UIO： DPDK的官方文檔http://doc.dpdk.org/gu...
    拖鞋花短褲閱讀 6,635評(píng)論 0贊 3
  • 1. 簡(jiǎn)介 本文檔包含DPDK軟件安裝和配置的相關(guān)說明。旨在幫助用戶快速啟動(dòng)和運(yùn)行軟件。文檔主要描述了在Linux...
    半天妖閱讀 18,109評(píng)論 0贊 22
  • X710-DA4有四個(gè)10GE口 兩臺(tái)DELL 730(支持PCIE3.0)各插一張X710-DA4網(wǎng)卡 一臺(tái)用于...
    嚴(yán)炎2016閱讀 3,281評(píng)論 0贊 0
  • 城市的污水 向著農(nóng)村的方向奔流 沿途寸草不生 城市的垃圾 一車車運(yùn)往鄉(xiāng)間 不會(huì)說話的生命在哭泣 城市的現(xiàn)代文明 被...
    冷冬年閱讀 2,664評(píng)論 73贊 111
  • 今日賞荷，感嘆荷花之美，美在一生一世。 從含苞待放到嬌羞初放，從豐滿綻放到蓮蓬長(zhǎng)成，直至葉腐莖朽，果實(shí)枯澀，她的每...
    同_昕閱讀 598評(píng)論 0贊 4

友情鏈接更多精彩內(nèi)容