Skip to content

Instantly share code, notes, and snippets.

@matpalm
Last active October 10, 2023 10:31
Show Gist options
  • Save matpalm/9c0c7c6a6f3681a0d39d to your computer and use it in GitHub Desktop.
Save matpalm/9c0c7c6a6f3681a0d39d to your computer and use it in GitHub Desktop.
json formatting of nvidia-settings
#!/usr/bin/env python
# gpu_stat.py [DELAY [COUNT]]
# dump some gpu stats as a line of json
# {"util":{"PCIe":"0", "memory":"11", "video":"0", "graphics":"13"}, "used_mem":"161"}
import json, socket, subprocess, sys, time
try:
delay = int(sys.argv[1])
except:
delay = 1
try:
count = int(sys.argv[2])
except:
count = None
i = 0
while True:
GPU = socket.gethostname() + ":0[gpu:0]"
cmd = ['nvidia-settings', '-t', '-q', GPU + '/GPUUtilization']
gpu_util = subprocess.check_output(cmd).strip().split(",")
gpu_util = dict([f.strip().split("=") for f in gpu_util])
cmd[-1] = GPU + '/UsedDedicatedGPUMemory'
gpu_used_mem = subprocess.check_output(cmd).strip()
print json.dumps({"used_mem": gpu_used_mem, "util": gpu_util, "time": int(time.time())})
if count != None:
i += 1
if i == count:
exit(0)
time.sleep(delay)
@Ahwar
Copy link

Ahwar commented Apr 6, 2020

I am running it on google Colab and it is showing this error.

Unable to init server: Could not connect: Connection refused

ERROR: The control display is undefined; please run `nvidia-settings
       --help` for usage information.

@SysOverdrive
Copy link

I am running it on a local machine and the subprocess cannot find the specified path
['nvidia-settings', '-t', '-q', 'TudPC:0[gpu:0]/GPUUtilization']

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment