-
-
Save jacobtomlinson/ee5ba79228e42bcc9975faf0179c3d1a to your computer and use it in GitHub Desktop.
Yeah, an AWS issue there, had the same problem.
One of the AWS Data Scientists suggested this post to me about using the load balancer to monitor:- https://aws.amazon.com/blogs/machine-learning/machine-learning-on-distributed-dask-using-amazon-sagemaker-and-aws-fargate/
Ive tried this as well. The cloud formation template doesnt work either. I essentially need to get my python instance into a the same VPC that the dask fargate cluster is in, or conversely, to use a proxy server.
@jacobtomlinson if we already have a fargate cluster set up, how are we able to connect to it in this step
c = distributed.Client('tcp://{}:8786'.format(scheduler_public_ip))
following the above steps returns a timeout error
Hi @ks233ever and @adair-kovac
Any resolution of this Client timeout issue?
To reiterate
All current work is being done in dask-cloudprovider and things here are definitely not supported.
When I got to generate the link to the scheduler, I get network / connectivity issues, which makes me think my companies firewall is messing it up, and that my security group rules need to be updated, but I dont know how to troubleshoot this.
from IPython.core.display import display, HTML display(HTML('<a href="{url}" target="_blank">{url}</a>'.format(url='http://{}:8787/status'.format(scheduler_public_ip))))