Skip to content

Instantly share code, notes, and snippets.

@navidcy
Created May 15, 2019 06:54
Show Gist options
  • Save navidcy/81e0bed4d8485d7c4bb57a13e24299d1 to your computer and use it in GitHub Desktop.
Save navidcy/81e0bed4d8485d7c4bb57a13e24299d1 to your computer and use it in GitHub Desktop.
A minimal example for dask_PBSCluster
Display the source blob
Display the rendered blob
Raw
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@navidcy
Copy link
Author

navidcy commented May 15, 2019

I don't know why the cell with the PBSCluster was not rendered correctly when I downloaded the notebook. Here's how it looks like:

Screen Shot 2019-05-15 at 4 57 56 pm

@josuemtzmo
Copy link

josuemtzmo commented May 20, 2019

I've been able to replace the lines:
!/sbin/ifconfig
cluster = PBSCluster(cores=10, memory='70 GB', ip='10.9.105.1', dashboard_address='192.43.239.21')
cluster
by:

import socket
import fcntl
import struct
def get_interface_ip(ifname):
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
return socket.inet_ntoa(fcntl.ioctl(
s.fileno(),
0x8915,
struct.pack('256s', bytes(ifname[:15], 'utf-8'))
)[20:24])
cluster = PBSCluster(cores=10, memory='70 GB', ip=get_interface_ip('ib0'), dashboard_address=get_interface_ip('vlan192'))
cluster
This allows you to avoid the copy and paste of IPs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment