Skip to content

Instantly share code, notes, and snippets.

@mhash1m
Created November 8, 2022 10:03
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mhash1m/dbf89266d294801fe94437628a076265 to your computer and use it in GitHub Desktop.
Save mhash1m/dbf89266d294801fe94437628a076265 to your computer and use it in GitHub Desktop.
Checking whether there is an H2O instance running at http://localhost:54321 ..... not found.
Attempting to start a local H2O server...
Java Version: openjdk version "1.8.0_342"; OpenJDK Runtime Environment (build 1.8.0_342-8u342-b07-0ubuntu1~20.04-b07); OpenJDK 64-Bit Server VM (build 25.342-b07, mixed mode)
Starting server from /home/hash1m/miniconda3/envs/zen_ml/lib/python3.7/site-packages/h2o/backend/bin/h2o.jar
Ice root: /tmp/tmpjslvnzy5
JVM stdout: /tmp/tmpjslvnzy5/h2o_hash1m_started_from_python.out
JVM stderr: /tmp/tmpjslvnzy5/h2o_hash1m_started_from_python.err
Server is running at http://127.0.0.1:54321
Connecting to H2O server at http://127.0.0.1:54321 ... successful.
-------------------------- -----------------------------
H2O_cluster_uptime: 01 secs
H2O_cluster_timezone: Asia/Karachi
H2O_data_parsing_timezone: UTC
H2O_cluster_version: 3.36.1.5
H2O_cluster_version_age: 1 month and 24 days
H2O_cluster_name: H2O_from_python_hash1m_kzwvlu
H2O_cluster_total_nodes: 1
H2O_cluster_free_memory: 3.424 Gb
H2O_cluster_total_cores: 8
H2O_cluster_allowed_cores: 8
H2O_cluster_status: locked, healthy
H2O_connection_url: http://127.0.0.1:54321
H2O_connection_proxy: {"http": null, "https": null}
H2O_internal_security: False
Python_version: 3.7.13 final
-------------------------- -----------------------------
You are running with a non-default project 'pascalle1'. Any stacks, components, pipelines and pipeline runs produced in this project will currently not be accessible through the dashboard. However, this will be possible in the near future.
You specified a plain-text value for the sensitive attribute tracking_token for a MLFlowExperimentTrackerConfig stack component. This is currently only a warning, but future versions of ZenML will require you to pass in sensitive information as secrets. Check out the documentation on how to configure your stack components with secrets here: https://docs.zenml.io/advanced-guide/practical/secrets-management
You are running with a non-default project 'pascalle1'. Any stacks, components, pipelines and pipeline runs produced in this project will currently not be accessible through the dashboard. However, this will be possible in the near future.
You are running with a non-default project 'pascalle1'. Any stacks, components, pipelines and pipeline runs produced in this project will currently not be accessible through the dashboard. However, this will be possible in the near future.
You are running with a non-default project 'pascalle1'. Any stacks, components, pipelines and pipeline runs produced in this project will currently not be accessible through the dashboard. However, this will be possible in the near future.
You are running with a non-default project 'pascalle1'. Any stacks, components, pipelines and pipeline runs produced in this project will currently not be accessible through the dashboard. However, this will be possible in the near future.
Running pipeline rekog_training_pipeline on stack mlflow_s3 (caching disabled)
You are running with a non-default project 'pascalle1'. Any stacks, components, pipelines and pipeline runs produced in this project will currently not be accessible through the dashboard. However, this will be possible in the near future.
You are running with a non-default project 'pascalle1'. Any stacks, components, pipelines and pipeline runs produced in this project will currently not be accessible through the dashboard. However, this will be possible in the near future.
You specified a plain-text value for the sensitive attribute tracking_token for a MLFlowExperimentTrackerConfig stack component. This is currently only a warning, but future versions of ZenML will require you to pass in sensitive information as secrets. Check out the documentation on how to configure your stack components with secrets here: https://docs.zenml.io/advanced-guide/practical/secrets-management
You are running with a non-default project 'pascalle1'. Any stacks, components, pipelines and pipeline runs produced in this project will currently not be accessible through the dashboard. However, this will be possible in the near future.
You are running with a non-default project 'pascalle1'. Any stacks, components, pipelines and pipeline runs produced in this project will currently not be accessible through the dashboard. However, this will be possible in the near future.
You are running with a non-default project 'pascalle1'. Any stacks, components, pipelines and pipeline runs produced in this project will currently not be accessible through the dashboard. However, this will be possible in the near future.
You are running with a non-default project 'pascalle1'. Any stacks, components, pipelines and pipeline runs produced in this project will currently not be accessible through the dashboard. However, this will be possible in the near future.
Step ingest_data has started.
You are running with a non-default project 'pascalle1'. Any stacks, components, pipelines and pipeline runs produced in this project will currently not be accessible through the dashboard. However, this will be possible in the near future.
You specified a plain-text value for the sensitive attribute tracking_token for a MLFlowExperimentTrackerConfig stack component. This is currently only a warning, but future versions of ZenML will require you to pass in sensitive information as secrets. Check out the documentation on how to configure your stack components with secrets here: https://docs.zenml.io/advanced-guide/practical/secrets-management
You are running with a non-default project 'pascalle1'. Any stacks, components, pipelines and pipeline runs produced in this project will currently not be accessible through the dashboard. However, this will be possible in the near future.
You are running with a non-default project 'pascalle1'. Any stacks, components, pipelines and pipeline runs produced in this project will currently not be accessible through the dashboard. However, this will be possible in the near future.
You are running with a non-default project 'pascalle1'. Any stacks, components, pipelines and pipeline runs produced in this project will currently not be accessible through the dashboard. However, this will be possible in the near future.
INFO:botocore.credentials:Found credentials in shared credentials file: ~/.aws/credentials
Step ingest_data has finished in 22.338s.
You are running with a non-default project 'pascalle1'. Any stacks, components, pipelines and pipeline runs produced in this project will currently not be accessible through the dashboard. However, this will be possible in the near future.
You specified a plain-text value for the sensitive attribute tracking_token for a MLFlowExperimentTrackerConfig stack component. This is currently only a warning, but future versions of ZenML will require you to pass in sensitive information as secrets. Check out the documentation on how to configure your stack components with secrets here: https://docs.zenml.io/advanced-guide/practical/secrets-management
You are running with a non-default project 'pascalle1'. Any stacks, components, pipelines and pipeline runs produced in this project will currently not be accessible through the dashboard. However, this will be possible in the near future.
You are running with a non-default project 'pascalle1'. Any stacks, components, pipelines and pipeline runs produced in this project will currently not be accessible through the dashboard. However, this will be possible in the near future.
You are running with a non-default project 'pascalle1'. Any stacks, components, pipelines and pipeline runs produced in this project will currently not be accessible through the dashboard. However, this will be possible in the near future.
You are running with a non-default project 'pascalle1'. Any stacks, components, pipelines and pipeline runs produced in this project will currently not be accessible through the dashboard. However, this will be possible in the near future.
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /home/hash1m/zenml_SM/AutoML-User-Interface-master/pipeline_run.py:86 in │
│ <module> │
│ │
│ 83 │ # rekog_pipeline_run('./wiz_rekog.csv', 'likes_count', │
│ 84 │ # aug_pipeline_run('./wiz_rekog.csv', 'likes_count', ta │
│ 85 │ # time_pipeline_run('./frank.csv', 'likes_count', "regression", "3" │
│ ❱ 86 │ rekog_pipeline_run('./frank.csv', 'likes_count', "regression", "3", │
│ 87 │ # aug_pipeline_run('./sns312.csv', 'likes_count', "classification", │
│ 88 │
│ │
│ /home/hash1m/zenml_SM/AutoML-User-Interface-master/pipeline_run.py:73 in │
│ rekog_pipeline_run │
│ │
│ 70 │ │ feature_selection = feature_selection(), │
│ 71 │ │ run_h2o = run_h2o() │
│ 72 │ ) │
│ ❱ 73 │ train_pipeline.run() │
│ 74 │
│ 75 if __name__ == "__main__": │
│ 76 │ # bin_opts = ["0", "1"] │
│ │
│ /home/hash1m/miniconda3/envs/zen_ml/lib/python3.7/site-packages/zenml/pipeli │
│ nes/base_pipeline.py:538 in run │
│ │
│ 535 │ │ # behavior │
│ 536 │ │ constants.SHOULD_PREVENT_PIPELINE_EXECUTION = True │
│ 537 │ │ try: │
│ ❱ 538 │ │ │ return_value = stack.deploy_pipeline(pipeline_deployment) │
│ 539 │ │ finally: │
│ 540 │ │ │ constants.SHOULD_PREVENT_PIPELINE_EXECUTION = False │
│ 541 │
│ │
│ /home/hash1m/miniconda3/envs/zen_ml/lib/python3.7/site-packages/zenml/stack/ │
│ stack.py:698 in deploy_pipeline │
│ │
│ 695 │ │ Returns: │
│ 696 │ │ │ The return value of the call to `orchestrator.run_pipeline │
│ 697 │ │ """ │
│ ❱ 698 │ │ return self.orchestrator.run(deployment=deployment, stack=self │
│ 699 │ │
│ 700 │ def _get_active_components_for_step( │
│ 701 │ │ self, step_config: "StepConfiguration" │
│ │
│ /home/hash1m/miniconda3/envs/zen_ml/lib/python3.7/site-packages/zenml/orches │
│ trators/base_orchestrator.py:283 in run │
│ │
│ 280 │ │ self._prepare_run(deployment=deployment) │
│ 281 │ │ │
│ 282 │ │ result = self.prepare_or_run_pipeline( │
│ ❱ 283 │ │ │ deployment=deployment, stack=stack │
│ 284 │ │ ) │
│ 285 │ │ │
│ 286 │ │ self._cleanup_run() │
│ │
│ /home/hash1m/miniconda3/envs/zen_ml/lib/python3.7/site-packages/zenml/orches │
│ trators/local/local_orchestrator.py:77 in prepare_or_run_pipeline │
│ │
│ 74 │ │ │ │ ) │
│ 75 │ │ │ │
│ 76 │ │ │ self.run_step( │
│ ❱ 77 │ │ │ │ step=step, │
│ 78 │ │ │ ) │
│ 79 │ │ │
│ 80 │ │ run_duration = time.time() - start_time │
│ │
│ /home/hash1m/miniconda3/envs/zen_ml/lib/python3.7/site-packages/zenml/orches │
│ trators/base_orchestrator.py:370 in run_step │
│ │
│ 367 │ │ if step.config.step_operator: │
│ 368 │ │ │ execution_info = self._execute_step(component_launcher) │
│ 369 │ │ else: │
│ ❱ 370 │ │ │ stack.prepare_step_run(info=step_run_info) │
│ 371 │ │ │ try: │
│ 372 │ │ │ │ execution_info = self._execute_step(component_launcher │
│ 373 │ │ │ except: # noqa: E722 │
│ │
│ /home/hash1m/miniconda3/envs/zen_ml/lib/python3.7/site-packages/zenml/stack/ │
│ stack.py:745 in prepare_step_run │
│ │
│ 742 │ │ for component in self._get_active_components_for_step( │
│ 743 │ │ │ info.config │
│ 744 │ │ ).values(): │
│ ❱ 745 │ │ │ component.prepare_step_run(info=info) │
│ 746 │ │
│ 747 │ def cleanup_step_run(self, info: "StepRunInfo") -> None: │
│ 748 │ │ """Cleans up resources after the step run is finished. │
│ │
│ /home/hash1m/miniconda3/envs/zen_ml/lib/python3.7/site-packages/zenml/integr │
│ ations/mlflow/experiment_trackers/mlflow_experiment_tracker.py:192 in │
│ prepare_step_run │
│ │
│ 189 │ │ │ run_id=run_id, │
│ 190 │ │ │ run_name=info.run_name, │
│ 191 │ │ │ experiment_id=experiment.experiment_id, │
│ ❱ 192 │ │ │ tags=tags, │
│ 193 │ │ ) │
│ 194 │ │ │
│ 195 │ │ if settings.nested: │
│ │
│ /home/hash1m/miniconda3/envs/zen_ml/lib/python3.7/site-packages/mlflow/track │
│ ing/fluent.py:318 in start_run │
│ │
│ 315 │ │ │
│ 316 │ │ resolved_tags = context_registry.resolve_tags(user_specified_ │
│ 317 │ │ │
│ ❱ 318 │ │ active_run_obj = client.create_run(experiment_id=exp_id_for_r │
│ 319 │ │
│ 320 │ _active_run_stack.append(ActiveRun(active_run_obj)) │
│ 321 │ return _active_run_stack[-1] │
│ │
│ /home/hash1m/miniconda3/envs/zen_ml/lib/python3.7/site-packages/mlflow/track │
│ ing/client.py:265 in create_run │
│ │
│ 262 │ │ │ lifecycle_stage: active │
│ 263 │ │ │ status: RUNNING │
│ 264 │ │ """ │
│ ❱ 265 │ │ return self._tracking_client.create_run(experiment_id, start_ │
│ 266 │ │
│ 267 │ def list_run_infos( │
│ 268 │ │ self, │
│ │
│ /home/hash1m/miniconda3/envs/zen_ml/lib/python3.7/site-packages/mlflow/track │
│ ing/_tracking_service/client.py:103 in create_run │
│ │
│ 100 │ │ │ experiment_id=experiment_id, │
│ 101 │ │ │ user_id=user_id, │
│ 102 │ │ │ start_time=start_time or int(time.time() * 1000), │
│ ❱ 103 │ │ │ tags=[RunTag(key, value) for (key, value) in tags.items()] │
│ 104 │ │ ) │
│ 105 │ │
│ 106 │ def list_run_infos( │
│ │
│ /home/hash1m/miniconda3/envs/zen_ml/lib/python3.7/site-packages/mlflow/store │
│ /tracking/rest_store.py:170 in create_run │
│ │
│ 167 │ │ │ │ tags=tag_protos, │
│ 168 │ │ │ ) │
│ 169 │ │ ) │
│ ❱ 170 │ │ response_proto = self._call_endpoint(CreateRun, req_body) │
│ 171 │ │ run = Run.from_proto(response_proto.run) │
│ 172 │ │ return run │
│ 173 │
│ │
│ /home/hash1m/miniconda3/envs/zen_ml/lib/python3.7/site-packages/mlflow/store │
│ /tracking/rest_store.py:56 in _call_endpoint │
│ │
│ 53 │ def _call_endpoint(self, api, json_body): │
│ 54 │ │ endpoint, method = _METHOD_TO_INFO[api] │
│ 55 │ │ response_proto = api.Response() │
│ ❱ 56 │ │ return call_endpoint(self.get_host_creds(), endpoint, method, │
│ 57 │ │
│ 58 │ def list_experiments( │
│ 59 │ │ self, │
│ │
│ /home/hash1m/miniconda3/envs/zen_ml/lib/python3.7/site-packages/mlflow/utils │
│ /rest_utils.py:256 in call_endpoint │
│ │
│ 253 │ │ response = http_request( │
│ 254 │ │ │ host_creds=host_creds, endpoint=endpoint, method=method, j │
│ 255 │ │ ) │
│ ❱ 256 │ response = verify_rest_response(response, endpoint) │
│ 257 │ js_dict = json.loads(response.text) │
│ 258 │ parse_dict(js_dict=js_dict, message=response_proto) │
│ 259 │ return response_proto │
│ │
│ /home/hash1m/miniconda3/envs/zen_ml/lib/python3.7/site-packages/mlflow/utils │
│ /rest_utils.py:185 in verify_rest_response │
│ │
│ 182 │ """Verify the return code and format, raise exception if the reque │
│ 183 │ if response.status_code != 200: │
│ 184 │ │ if _can_parse_as_json_object(response.text): │
│ ❱ 185 │ │ │ raise RestException(json.loads(response.text)) │
│ 186 │ │ else: │
│ 187 │ │ │ base_msg = "API request to endpoint %s failed with error c │
│ 188 │ │ │ │ endpoint, │
╰──────────────────────────────────────────────────────────────────────────────╯
RestException: BAD_REQUEST: (psycopg2.errors.UniqueViolation) duplicate key
value violates unique constraint "tag_pk"
DETAIL: Key (key, run_uuid)=(mlflow.runName, 8bb915bab15842bc989a9868c978b5e0)
already exists.
[SQL: INSERT INTO tags (key, value, run_uuid) VALUES (%(key)s, %(value)s,
%(run_uuid)s)]
[parameters: ({'key': 'mlflow.user', 'value': 'hash1m', 'run_uuid':
'8bb915bab15842bc989a9868c978b5e0'}, {'key': 'mlflow.source.name', 'value':
'pipeline_run.py', 'run_uuid': '8bb915bab15842bc989a9868c978b5e0'}, {'key':
'mlflow.source.type', 'value': 'LOCAL', 'run_uuid':
'8bb915bab15842bc989a9868c978b5e0'}, {'key': 'mlflow.source.git.commit',
'value': '48ba36873316a75f4a1e1489459db2debdc74085', 'run_uuid':
'8bb915bab15842bc989a9868c978b5e0'}, {'key': 'zenml', 'value': '0.21.1',
'run_uuid': '8bb915bab15842bc989a9868c978b5e0'}, {'key': 'mlflow.runName',
'value': 'rekog_training_pipeline-2022_11_08-14_48_30_122312', 'run_uuid':
'8bb915bab15842bc989a9868c978b5e0'}, {'key': 'mlflow.runName', 'value':
'legendary-shrike-198', 'run_uuid': '8bb915bab15842bc989a9868c978b5e0'})]
(Background on this error at: https://sqlalche.me/e/14/gkpj)
Closing connection _sid_abb7 at exit
H2O session _sid_abb7 closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment