Skip to content

Instantly share code, notes, and snippets.

@Davidnet
Created May 24, 2023 17:04
Show Gist options
  • Save Davidnet/775bc511e6a57962df1e6d3137024370 to your computer and use it in GitHub Desktop.
Save Davidnet/775bc511e6a57962df1e6d3137024370 to your computer and use it in GitHub Desktop.
Public
Hello everyone I am trying to run the following pipeline with TPUs,
In GCP I have quotas, and credits, yet in vertex it fails with the same message: "Internal Error" please contact Vertex AI, but to contact them, yet does anyone see anything wrong with the following definition?
from kfp import dsl
import google_cloud_pipeline_components.v1.custom_job as gcp_components_custom_job
@dsl.component
def print_test():
print("Hello World!")
# Using TPUs
printOp = gcp_components_custom_job.create_custom_training_job_from_component(
print_test,
display_name="Print Test",
machine_type="cloud-tpu",
accelerator_type="TPU_V3",
accelerator_count=8,
)
@dsl.pipeline
def pipeline():
print_task = printOp(
project="CENSORED",
location="us-central1",
)
if __name__ == "__main__":
import kfp.compiler as compiler
compiler.Compiler().compile(pipeline, "problem_pipeline.yaml")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment