Skip to content

Instantly share code, notes, and snippets.

@navidcy

navidcy/dask_PBSCluster_MinimalExample.ipynb

Created May 15, 2019 06:54
Show Gist options
  • Save navidcy/81e0bed4d8485d7c4bb57a13e24299d1 to your computer and use it in GitHub Desktop.
Save navidcy/81e0bed4d8485d7c4bb57a13e24299d1 to your computer and use it in GitHub Desktop.
Download ZIP
A minimal example for dask_PBSCluster
Display the source blob
Display the rendered blob
Raw
dask_PBSCluster_MinimalExample.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@josuemtzmo
Copy link

josuemtzmo commented May 20, 2019 •
edited
Loading

I've been able to replace the lines:
!/sbin/ifconfig
cluster = PBSCluster(cores=10, memory='70 GB', ip='10.9.105.1', dashboard_address='192.43.239.21')
cluster
by:

import socket
import fcntl
import struct
def get_interface_ip(ifname):
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
return socket.inet_ntoa(fcntl.ioctl(
s.fileno(),
0x8915,
struct.pack('256s', bytes(ifname[:15], 'utf-8'))
)[20:24])
cluster = PBSCluster(cores=10, memory='70 GB', ip=get_interface_ip('ib0'), dashboard_address=get_interface_ip('vlan192'))
cluster
This allows you to avoid the copy and paste of IPs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment