Skip to content

Instantly share code, notes, and snippets.

Avatar
💭
working on daos.io

Tom Nabarro tanabarr

💭
working on daos.io
  • @intel-hpdd
  • Oxfordshire, UK
View GitHub Profile
View isa-l undefined
Found ninja-1.7.2 at /usr/bin/ninja
+ ninja -C build
ninja: Entering directory `build'
[1415/1451] Linking target drivers/librte_compress_isal.so.21.0.
FAILED: drivers/librte_compress_isal.so.21.0
cc -o drivers/librte_compress_isal.so.21.0 'drivers/drivers@@rte_compress_isal@sha/meson-generated_.._rte_compress_isal.pmd.c.o' 'drivers/drivers@@tmp_rte_compress_isal@sta/compress_isal_isal_compress_pmd.c.o' 'drivers/drivers@@tmp_rte_compress_isal@sta/compress_isal_isal_compress_pmd_ops.c.o' -Wl,--no-undefined -Wl,--as-needed -Wl,-O1 -shared -fPIC -Wl,--start-group -Wl,-soname,librte_compress_isal.so.21 -Wl,--no-as-needed -pthread -lm -ldl -lnuma lib/librte_compressdev.so.21.0 lib/librte_eal.so.21.0 lib/librte_kvargs.so.21.0 lib/librte_telemetry.so.21.0 lib/librte_mbuf.so.21.0 lib/librte_mempool.so.21.0 lib/librte_ring.so.21.0 drivers/librte_bus_vdev.so.21.0 -Wl,--end-group -Wl,--version-script=/home/tanabarr/projects/daos-dpdk-20_11/_topdir/BUILD/dpdk-20.11/drivers/compress/isal/version.map '-Wl,-rpath,$ORIGIN/.
@tanabarr
tanabarr / create_pool_with_acls
Created Sep 26, 2019
Create a DAOS pool with permissions read from a supplied ACL file.
View create_pool_with_acls
[root@wolf-71 daos_m]# daos_shell pool create -a /tmp/acls -s 1G -n 10G -i
Active connections: [localhost:10001]
Creating DAOS pool with 1GB SCM and 10GB NvMe storage (0.100 ratio)
Pool-create command SUCCEEDED: UUID: 58191829-6d76-4a37-b7cd-ded5604bfbde, Service replicas: 0
[root@wolf-71 daos_m]# cat /tmp/acls
A::root@:rw
A::OWNER@:rw
A:g:readers@:r
L:f:baduser@:rw
U:f:EVERYONE@:rw
@tanabarr
tanabarr / daos_test_wolf_psm2
Last active Sep 26, 2019
Commands and config file needed to run daos_test to validate installation using psm2 provider.
View daos_test_wolf_psm2
Build from commit 2e4fcb939e83f8d65b1cc1667f2a195f8cf1131c
Verify provider is available:
daos_m/install/bin/fi_info | grep psm
(should show some output, if not then rebuild daos_m/_build.external/ofi)
need to run client and server on different hosts to be able to reconnect without restarting server (required in daos_test)
CONFIG:
@tanabarr
tanabarr / daos_test_wolf_sockets
Last active Oct 9, 2019
Commands and config file needed to run daos_test to validate installation using sockets provider.
View daos_test_wolf_sockets
Build from commit 2e4fcb939e83f8d65b1cc1667f2a195f8cf1131c
CONFIG:
--- a/utils/config/examples/daos_server_sockets.yml
+++ b/utils/config/examples/daos_server_sockets.yml
@@ -3,13 +3,15 @@
name: daos_server # map to -g daos_server
port: 10001 # mgmt port
provider: ofi+sockets # map to CRT_PHY_ADDR_STR=ofi+sockets
@tanabarr
tanabarr / daos_test_psm2
Last active Oct 31, 2019
Run DAOS pool test suite with OFI PSM2 provider, DCPM modules and NVMe SSDs.
View daos_test_psm2
CONFIG:
name: daos_server # map to -g daos_server
port: 10001 # mgmt port
provider: ofi+psm2 # map to CRT_PHY_ADDR_STR=ofi+psm2
nr_hugepages: 4096
control_log_mask: DEBUG
control_log_file: /tmp/daos_control.log
transport_config:
@tanabarr
tanabarr / run_daos_docker_troubleshooting
Created Sep 24, 2019
Probe UIO and map hugepages when running DAOS in docker container
View run_daos_docker_troubleshooting
daos@wolf-130:~/docker/daos$ ls /sys/bus/pci/drivers/uio_pci_generic
ls: cannot access /sys/bus/pci/drivers/uio_pci_generic: No such file or director
daos@wolf-130:~/docker/daos$ sudo modprobe uio_pci_generic
daos@wolf-130:~/docker/daos $ ls /sys/bus/pci/drivers/uio_pci_generic
0000:00:04.0 0000:00:04.3 0000:00:04.6 0000:5f:00.0 0000:80:04.2 0000:80:04.5 0000:81:00.0 module uevent
0000:00:04.1 0000:00:04.4 0000:00:04.7 0000:80:04.0 0000:80:04.3 0000:80:04.6 0000:da:00.0 new_id unbind
0000:00:04.2 0000:00:04.5 0000:5e:00.0 0000:80:04.1 0000:80:04.4 0000:80:04.7 bind remove_id
@tanabarr
tanabarr / pool_create_no_client_config
Created Sep 23, 2019
Creating Pool without specifying a client side config file in command
View pool_create_no_client_config
SERVER:
--- a/utils/config/examples/daos_server_sockets.yml
+++ b/utils/config/examples/daos_server_sockets.yml
@@@ -6,10 -6,8 +6,12 @@@ provider: ofi+sockets # map to CR
socket_dir: /tmp/daos_sockets
nr_hugepages: 4096
control_log_mask: DEBUG
- control_log_file: /tmp/daos_control.log
++#control_log_file: /tmp/daos_control.log
@tanabarr
tanabarr / prepare_scm
Last active Sep 12, 2019
Using daos_server to prepare local SCM/DCPM modules
View prepare_scm
[root@wolf-72 daos_m]# daos_server storage prepare --debug
bash: daos_server: command not found
[root@wolf-72 daos_m]# source scons_local/utils/setup_local.sh
/home/tanabarr/projects/daos_m
Build vars file found: ./.build_vars.sh
OLD_PATH is /usr/lib64/qt-3.3/bin /usr/local/bin /usr/bin /usr/local/sbin /usr/sbin /usr/lib/go-1.10/bin /usr/local/go/bin /usr/local/go/bin /home/tanabarr/bin /usr/lib/go-1.10/bin /usr/local/go/bin /usr/local/go/bin
[root@wolf-72 daos_m]# daos_server storage prepare --debug
DEBUG 13:54:28.030669 ctl_storage.go:98: spdk setup with _NRHUGE=1024
DEBUG 13:54:28.033110 ctl_storage.go:98: spdk setup with _TARGET_USER=root
Preparing locally-attached SCM...
@tanabarr
tanabarr / create_container
Last active Oct 31, 2019
Create DAOS container after creating DAOS pool through the control plane (dmg)
View create_container
These commands were issued on binaries built from f6a34514a45eb3fb38dc43771d11c9f61e510516
SERVER CONFIG (NVMe is disabled to simplify this example only, should work fine with it enabled):
--- a/utils/config/examples/daos_server_sockets.yml
+++ b/utils/config/examples/daos_server_sockets.yml
@@ -6,10 +6,11 @@ provider: ofi+sockets # map to CRT_PHY_ADDR_STR=ofi+sockets
socket_dir: /tmp/daos_sockets
nr_hugepages: 4096
control_log_mask: DEBUG
@tanabarr
tanabarr / basic_daos_deployment_workflow
Last active Aug 28, 2019
Deployment workflow for DAOS using the control plane tools
View basic_daos_deployment_workflow
## Prepare devices for use
[root@<hostname> tanabarr]# cd projects/daos_m/
[root@<hostname> daos_m]# source scons_local/utils/setup_local.sh
/home/tanabarr/projects/daos_m
Build vars file found: ./.build_vars.sh
OLD_PATH is /usr/lib64/qt-3.3/bin /usr/local/bin /usr/bin /usr/local/sbin /usr/sbin /usr/lib/go-1.10/bin /usr/local/go/bin /usr/local/go/bin /home/tanabarr/bin /usr/lib/go-1.10/bin /usr/local/go/bin /usr/local/go/bin
[root@<hostname> daos_m]# daos_server storage prep-scm
Memory allocation goals for SCM will be changed and namespaces modified, this will be a destructive operation. ensure namespaces are unmounted and SCM is otherwise unused.
Are you sure you want to continue? (yes/no)