tanabarr
/ verify_blob_removed
Created
July 4, 2019 15:37
Verify SPDK blobs are removed through Control Plane "storage format" command
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| -model: 'INTEL SSDPEDMD016T4 ' | |
| serial: 'CVFT6010002F1P6DGN ' | |
| pciaddr: 0000:81:00.0 | |
| fwrev: 8DV10171 | |
| namespaces: id: 1 | |
| capacity: 1600 | |
| 0000:81:00.0 == Nvme0n1 (note names do not necessarily correlate with those reported by nvme-cli tool) | |
| dev_list: ["0000:81:00.0"] # generate regular nvme.conf |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [tanabarr@wolf-71 go-spdk]$ export CGO_LDFLAGS="-L/home/tanabarr/projects/daos_m/install/lib" | |
| [tanabarr@wolf-71 go-spdk]$ export CGO_CFLAGS="-I/home/tanabarr/projects/daos_m/install/include" | |
| [tanabarr@wolf-71 go-spdk]$ bin/./task main-task | |
| rm -f *.o *.so | |
| gcc ${CGO_LDFLAGS} ${CGO_CFLAGS} -Werror -g -Wshadow -Wall -Wno-missing-braces -c -fpic -Iinclude src/*.c -lspdk | |
| gcc ${CGO_LDFLAGS} ${CGO_CFLAGS} -shared -o libnvme_control.so *.o | |
| go test -v | |
| # _/home/tanabarr/projects/daos_m/src/control/vendor/github.com/daos-stack/go-spdk/spdk | |
| /tmp/go-build011208827/b046/_x003.o: In function `_cgo_dc3fa52d99f9_Cfunc_spdk_env_init': | |
| spdk.cgo2.c:(.text+0x5e): undefined reference to `spdk_env_init' |
tanabarr
/ respawn_non_root
Last active
October 8, 2019 17:20
respawn daos_server as normal user (specified in server config) after starting as root and performing format from management tool (daos_shell)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ---- SERVER: ---- | |
| root@wolf-72:/home/tanabarr/projects/daos_m$ umount /mnt/daos; rm -rf /mnt/daos; mkdir /mnt/daos | |
| root@wolf-72:/home/tanabarr/projects/daos_m$ daos_server storage prepare -n -b | |
| Preparing locally-attached NVMe storage... | |
| 2019/07/11 20:45:30 storage_nvme.go:98: debug: spdk setup with _NRHUGE=1024 | |
| 2019/07/11 20:45:30 storage_nvme.go:102: debug: spdk setup with _TARGET_USER=root | |
| +++ b/utils/config/examples/daos_server_sockets.yml | |
| @@ -9,7 +9,7 @@ control_log_mask: DEBUG |
tanabarr
/ basic_daos_deployment_workflow
Last active
August 28, 2019 13:09
Deployment workflow for DAOS using the control plane tools
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ## Prepare devices for use | |
| [root@<hostname> tanabarr]# cd projects/daos_m/ | |
| [root@<hostname> daos_m]# source scons_local/utils/setup_local.sh | |
| /home/tanabarr/projects/daos_m | |
| Build vars file found: ./.build_vars.sh | |
| OLD_PATH is /usr/lib64/qt-3.3/bin /usr/local/bin /usr/bin /usr/local/sbin /usr/sbin /usr/lib/go-1.10/bin /usr/local/go/bin /usr/local/go/bin /home/tanabarr/bin /usr/lib/go-1.10/bin /usr/local/go/bin /usr/local/go/bin | |
| [root@<hostname> daos_m]# daos_server storage prep-scm | |
| Memory allocation goals for SCM will be changed and namespaces modified, this will be a destructive operation. ensure namespaces are unmounted and SCM is otherwise unused. | |
| Are you sure you want to continue? (yes/no) |
tanabarr
/ create_container
Last active
October 31, 2019 00:37
Create DAOS container after creating DAOS pool through the control plane (dmg)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| These commands were issued on binaries built from f6a34514a45eb3fb38dc43771d11c9f61e510516 | |
| SERVER CONFIG (NVMe is disabled to simplify this example only, should work fine with it enabled): | |
| --- a/utils/config/examples/daos_server_sockets.yml | |
| +++ b/utils/config/examples/daos_server_sockets.yml | |
| @@ -6,10 +6,11 @@ provider: ofi+sockets # map to CRT_PHY_ADDR_STR=ofi+sockets | |
| socket_dir: /tmp/daos_sockets | |
| nr_hugepages: 4096 | |
| control_log_mask: DEBUG |
tanabarr
/ prepare_scm
Last active
September 12, 2019 14:00
Using daos_server to prepare local SCM/DCPM modules
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@wolf-72 daos_m]# daos_server storage prepare --debug | |
| bash: daos_server: command not found | |
| [root@wolf-72 daos_m]# source scons_local/utils/setup_local.sh | |
| /home/tanabarr/projects/daos_m | |
| Build vars file found: ./.build_vars.sh | |
| OLD_PATH is /usr/lib64/qt-3.3/bin /usr/local/bin /usr/bin /usr/local/sbin /usr/sbin /usr/lib/go-1.10/bin /usr/local/go/bin /usr/local/go/bin /home/tanabarr/bin /usr/lib/go-1.10/bin /usr/local/go/bin /usr/local/go/bin | |
| [root@wolf-72 daos_m]# daos_server storage prepare --debug | |
| DEBUG 13:54:28.030669 ctl_storage.go:98: spdk setup with _NRHUGE=1024 | |
| DEBUG 13:54:28.033110 ctl_storage.go:98: spdk setup with _TARGET_USER=root | |
| Preparing locally-attached SCM... |
tanabarr
/ pool_create_no_client_config
Created
September 23, 2019 15:12
Creating Pool without specifying a client side config file in command
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| SERVER: | |
| --- a/utils/config/examples/daos_server_sockets.yml | |
| +++ b/utils/config/examples/daos_server_sockets.yml | |
| @@@ -6,10 -6,8 +6,12 @@@ provider: ofi+sockets # map to CR | |
| socket_dir: /tmp/daos_sockets | |
| nr_hugepages: 4096 | |
| control_log_mask: DEBUG | |
| - control_log_file: /tmp/daos_control.log | |
| ++#control_log_file: /tmp/daos_control.log |
tanabarr
/ run_daos_docker_troubleshooting
Created
September 24, 2019 15:49
Probe UIO and map hugepages when running DAOS in docker container
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| daos@wolf-130:~/docker/daos$ ls /sys/bus/pci/drivers/uio_pci_generic | |
| ls: cannot access /sys/bus/pci/drivers/uio_pci_generic: No such file or director | |
| daos@wolf-130:~/docker/daos$ sudo modprobe uio_pci_generic | |
| daos@wolf-130:~/docker/daos $ ls /sys/bus/pci/drivers/uio_pci_generic | |
| 0000:00:04.0 0000:00:04.3 0000:00:04.6 0000:5f:00.0 0000:80:04.2 0000:80:04.5 0000:81:00.0 module uevent | |
| 0000:00:04.1 0000:00:04.4 0000:00:04.7 0000:80:04.0 0000:80:04.3 0000:80:04.6 0000:da:00.0 new_id unbind | |
| 0000:00:04.2 0000:00:04.5 0000:5e:00.0 0000:80:04.1 0000:80:04.4 0000:80:04.7 bind remove_id |
tanabarr
/ daos_test_psm2
Last active
October 31, 2019 09:26
Run DAOS pool test suite with OFI PSM2 provider, DCPM modules and NVMe SSDs.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| CONFIG: | |
| name: daos_server # map to -g daos_server | |
| port: 10001 # mgmt port | |
| provider: ofi+psm2 # map to CRT_PHY_ADDR_STR=ofi+psm2 | |
| nr_hugepages: 4096 | |
| control_log_mask: DEBUG | |
| control_log_file: /tmp/daos_control.log | |
| transport_config: |
tanabarr
/ daos_test_wolf_sockets
Last active
October 9, 2019 10:01
Commands and config file needed to run daos_test to validate installation using sockets provider.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Build from commit 2e4fcb939e83f8d65b1cc1667f2a195f8cf1131c | |
| CONFIG: | |
| --- a/utils/config/examples/daos_server_sockets.yml | |
| +++ b/utils/config/examples/daos_server_sockets.yml | |
| @@ -3,13 +3,15 @@ | |
| name: daos_server # map to -g daos_server | |
| port: 10001 # mgmt port | |
| provider: ofi+sockets # map to CRT_PHY_ADDR_STR=ofi+sockets |