Skip to content

Instantly share code, notes, and snippets.

@tanabarr
Created September 24, 2019 15:49
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save tanabarr/56c02b7cd7eb38841797743048274a2e to your computer and use it in GitHub Desktop.
Save tanabarr/56c02b7cd7eb38841797743048274a2e to your computer and use it in GitHub Desktop.
Probe UIO and map hugepages when running DAOS in docker container
daos@wolf-130:~/docker/daos$ ls /sys/bus/pci/drivers/uio_pci_generic
ls: cannot access /sys/bus/pci/drivers/uio_pci_generic: No such file or director
daos@wolf-130:~/docker/daos$ sudo modprobe uio_pci_generic
daos@wolf-130:~/docker/daos $ ls /sys/bus/pci/drivers/uio_pci_generic
0000:00:04.0 0000:00:04.3 0000:00:04.6 0000:5f:00.0 0000:80:04.2 0000:80:04.5 0000:81:00.0 module uevent
0000:00:04.1 0000:00:04.4 0000:00:04.7 0000:80:04.0 0000:80:04.3 0000:80:04.6 0000:da:00.0 new_id unbind
0000:00:04.2 0000:00:04.5 0000:5e:00.0 0000:80:04.1 0000:80:04.4 0000:80:04.7 bind remove_id
daos@wolf-130:~/docker/daos$ sudo docker run -it -d --privileged --name server -v /tmp/uri:/tmp/uri -v /dev/hugepages:/dev/hugepages daos
d60a2424d38bed55f8638b3a043dfc28b213edc0e4f6624e75abfcd1dd287309
daos@wolf-130:~/docker/daos$ sudo docker exec server daos_server storage prepare -n -f
Preparing locally-attached NVMe storage...
daos@wolf-130:~/docker/daos$ sudo docker exec server daos_server storage scan
Scanning locally-attached storage...
Starting SPDK v18.07-pre / DPDK 18.02.0 initialization...
[ DPDK EAL parameters: spdk -c 0x1 --file-prefix=spdk_pid1065 ]
EAL: Detected 96 lcore(s)
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Multi-process socket /var/run/.spdk_pid1065_unix
EAL: Probing VFIO support...
EAL: PCI device 0000:5e:00.0 on NUMA socket 0
EAL: probe driver: 8086:a54 spdk_nvme
EAL: PCI device 0000:5f:00.0 on NUMA socket 0
EAL: probe driver: 8086:a54 spdk_nvme
EAL: PCI device 0000:81:00.0 on NUMA socket 1
EAL: probe driver: 8086:2701 spdk_nvme
EAL: PCI device 0000:da:00.0 on NUMA socket 1
EAL: probe driver: 8086:2701 spdk_nvme
NVMe SSD controller and constituent namespaces:
PCI Addr:0000:81:00.0 Serial:PHKS75050081750BGN Model:INTEL SSDPED1K750GA Fwrev:E2010325 Socket:1
Namespace: id:1 capacity:750
PCI Addr:0000:da:00.0 Serial:PHKS7505007E750BGN Model:INTEL SSDPED1K750GA Fwrev:E2010325 Socket:1
Namespace: id:1 capacity:750
PCI Addr:0000:5f:00.0 Serial:BTLN836300141P6AGN Model:INTEL SSDPE2KE016T8 Fwrev:VDV10140 Socket:0
Namespace: id:1 capacity:1600
PCI Addr:0000:5e:00.0 Serial:BTLN836302E01P6AGN Model:INTEL SSDPE2KE016T8 Fwrev:VDV10140 Socket:0
Namespace: id:1 capacity:1600
SCM modules:
PhysicalID:40 Capacity:539661172736 Location:(socket:0 memctrlr:0 chan:0 pos:1)
PhysicalID:44 Capacity:539661172736 Location:(socket:0 memctrlr:0 chan:1 pos:1)
PhysicalID:48 Capacity:539661172736 Location:(socket:0 memctrlr:0 chan:2 pos:1)
PhysicalID:54 Capacity:539661172736 Location:(socket:0 memctrlr:1 chan:0 pos:1)
PhysicalID:58 Capacity:539661172736 Location:(socket:0 memctrlr:1 chan:1 pos:1)
PhysicalID:62 Capacity:539661172736 Location:(socket:0 memctrlr:1 chan:2 pos:1)
PhysicalID:68 Capacity:539661172736 Location:(socket:1 memctrlr:0 chan:0 pos:1)
PhysicalID:72 Capacity:539661172736 Location:(socket:1 memctrlr:0 chan:1 pos:1)
PhysicalID:76 Capacity:539661172736 Location:(socket:1 memctrlr:0 chan:2 pos:1)
PhysicalID:82 Capacity:539661172736 Location:(socket:1 memctrlr:1 chan:0 pos:1)
PhysicalID:86 Capacity:539661172736 Location:(socket:1 memctrlr:1 chan:1 pos:1)
PhysicalID:90 Capacity:539661172736 Location:(socket:1 memctrlr:1 chan:2 pos:1)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment