Skip to content

Instantly share code, notes, and snippets.

@BasPH
Created May 23, 2020 12:09
Show Gist options
  • Save BasPH/31f192546d2728efc89b8ef9151e9652 to your computer and use it in GitHub Desktop.
Save BasPH/31f192546d2728efc89b8ef9151e9652 to your computer and use it in GitHub Desktop.
Very inefficient test of Airflow providers backport packages
import docker
# fmt: off
backport_package_class_mapping = {
"apache-airflow-backport-providers-amazon==2020.5.20rc2": ["airflow.providers.amazon.aws.hooks.athena.AWSAthenaHook", "airflow.providers.amazon.aws.hooks.aws_dynamodb.AwsDynamoDBHook", "airflow.providers.amazon.aws.hooks.base_aws.AwsBaseHook", "airflow.providers.amazon.aws.hooks.cloud_formation.AWSCloudFormationHook", "airflow.providers.amazon.aws.hooks.datasync.AWSDataSyncHook", "airflow.providers.amazon.aws.hooks.ec2.EC2Hook", "airflow.providers.amazon.aws.hooks.emr.EmrHook", "airflow.providers.amazon.aws.hooks.glue.AwsGlueJobHook", "airflow.providers.amazon.aws.hooks.glue_catalog.AwsGlueCatalogHook", "airflow.providers.amazon.aws.hooks.kinesis.AwsFirehoseHook", "airflow.providers.amazon.aws.hooks.lambda_function.AwsLambdaHook", "airflow.providers.amazon.aws.hooks.logs.AwsLogsHook", "airflow.providers.amazon.aws.hooks.redshift.RedshiftHook", "airflow.providers.amazon.aws.hooks.s3.S3Hook", "airflow.providers.amazon.aws.hooks.sagemaker.SageMakerHook", "airflow.providers.amazon.aws.hooks.sns.AwsSnsHook", "airflow.providers.amazon.aws.hooks.sqs.SQSHook", "airflow.providers.amazon.aws.operators.athena.AWSAthenaOperator", "airflow.providers.amazon.aws.operators.batch.AwsBatchOperator", "airflow.providers.amazon.aws.operators.cloud_formation.CloudFormationCreateStackOperator", "airflow.providers.amazon.aws.operators.cloud_formation.CloudFormationDeleteStackOperator", "airflow.providers.amazon.aws.operators.datasync.AWSDataSyncOperator", "airflow.providers.amazon.aws.operators.dynamodb_to_s3.DynamoDBToS3Operator", "airflow.providers.amazon.aws.operators.ec2_start_instance.EC2StartInstanceOperator", "airflow.providers.amazon.aws.operators.ec2_stop_instance.EC2StopInstanceOperator", "airflow.providers.amazon.aws.operators.ecs.ECSOperator", "airflow.providers.amazon.aws.operators.emr_add_steps.EmrAddStepsOperator", "airflow.providers.amazon.aws.operators.emr_create_job_flow.EmrCreateJobFlowOperator", "airflow.providers.amazon.aws.operators.emr_modify_cluster.EmrModifyClusterOperator", "airflow.providers.amazon.aws.operators.emr_terminate_job_flow.EmrTerminateJobFlowOperator", "airflow.providers.amazon.aws.operators.gcs_to_s3.GCSToS3Operator", "airflow.providers.amazon.aws.operators.glue.AwsGlueJobOperator", "airflow.providers.amazon.aws.operators.google_api_to_s3_transfer.GoogleApiToS3Transfer", "airflow.providers.amazon.aws.operators.hive_to_dynamodb.HiveToDynamoDBTransferOperator", "airflow.providers.amazon.aws.operators.imap_attachment_to_s3.ImapAttachmentToS3Operator", "airflow.providers.amazon.aws.operators.redshift_to_s3.RedshiftToS3Transfer", "airflow.providers.amazon.aws.operators.s3_copy_object.S3CopyObjectOperator", "airflow.providers.amazon.aws.operators.s3_delete_objects.S3DeleteObjectsOperator", "airflow.providers.amazon.aws.operators.s3_file_transform.S3FileTransformOperator", "airflow.providers.amazon.aws.operators.s3_list.S3ListOperator", "airflow.providers.amazon.aws.operators.s3_to_redshift.S3ToRedshiftTransfer", "airflow.providers.amazon.aws.operators.s3_to_sftp.S3ToSFTPOperator", "airflow.providers.amazon.aws.operators.sagemaker_base.SageMakerBaseOperator", "airflow.providers.amazon.aws.operators.sagemaker_endpoint.SageMakerEndpointOperator", "airflow.providers.amazon.aws.operators.sagemaker_endpoint_config.SageMakerEndpointConfigOperator", "airflow.providers.amazon.aws.operators.sagemaker_model.SageMakerModelOperator", "airflow.providers.amazon.aws.operators.sagemaker_training.SageMakerTrainingOperator", "airflow.providers.amazon.aws.operators.sagemaker_transform.SageMakerTransformOperator", "airflow.providers.amazon.aws.operators.sagemaker_tuning.SageMakerTuningOperator", "airflow.providers.amazon.aws.operators.sftp_to_s3.SFTPToS3Operator", "airflow.providers.amazon.aws.operators.sns.SnsPublishOperator", "airflow.providers.amazon.aws.operators.sqs.SQSPublishOperator", "airflow.providers.amazon.aws.secrets.secrets_manager.SecretsManagerBackend", "airflow.providers.amazon.aws.secrets.systems_manager.SystemsManagerParameterStoreBackend", "airflow.providers.amazon.aws.sensors.athena.AthenaSensor", "airflow.providers.amazon.aws.sensors.cloud_formation.CloudFormationCreateStackSensor", "airflow.providers.amazon.aws.sensors.cloud_formation.CloudFormationDeleteStackSensor", "airflow.providers.amazon.aws.sensors.ec2_instance_state.EC2InstanceStateSensor", "airflow.providers.amazon.aws.sensors.emr_base.EmrBaseSensor", "airflow.providers.amazon.aws.sensors.emr_job_flow.EmrJobFlowSensor", "airflow.providers.amazon.aws.sensors.emr_step.EmrStepSensor", "airflow.providers.amazon.aws.sensors.glue.AwsGlueJobSensor", "airflow.providers.amazon.aws.sensors.glue_catalog_partition.AwsGlueCatalogPartitionSensor", "airflow.providers.amazon.aws.sensors.redshift.AwsRedshiftClusterSensor", "airflow.providers.amazon.aws.sensors.s3_key.S3KeySensor", "airflow.providers.amazon.aws.sensors.s3_prefix.S3PrefixSensor", "airflow.providers.amazon.aws.sensors.sagemaker_base.SageMakerBaseSensor", "airflow.providers.amazon.aws.sensors.sagemaker_endpoint.SageMakerEndpointSensor", "airflow.providers.amazon.aws.sensors.sagemaker_training.SageMakerTrainingSensor", "airflow.providers.amazon.aws.sensors.sagemaker_transform.SageMakerTransformSensor", "airflow.providers.amazon.aws.sensors.sagemaker_tuning.SageMakerTuningSensor", "airflow.providers.amazon.aws.sensors.sqs.SQSSensor"],
"apache-airflow-backport-providers-apache-cassandra==2020.5.20rc2": ["airflow.providers.apache.cassandra.hooks.cassandra.CassandraHook", "airflow.providers.apache.cassandra.sensors.record.CassandraRecordSensor", "airflow.providers.apache.cassandra.sensors.table.CassandraTableSensor"],
"apache-airflow-backport-providers-apache-druid==2020.5.20rc2": ["airflow.providers.apache.druid.hooks.druid.DruidDbApiHook", "airflow.providers.apache.druid.hooks.druid.DruidHook", "airflow.providers.apache.druid.operators.druid.DruidOperator", "airflow.providers.apache.druid.operators.druid_check.DruidCheckOperator", "airflow.providers.apache.druid.operators.hive_to_druid.HiveToDruidTransfer"],
"apache-airflow-backport-providers-apache-hdfs==2020.5.20rc2": ["airflow.providers.apache.hdfs.hooks.hdfs.HDFSHook", "airflow.providers.apache.hdfs.hooks.webhdfs.WebHDFSHook", "airflow.providers.apache.hdfs.sensors.hdfs.HdfsSensor", "airflow.providers.apache.hdfs.sensors.web_hdfs.WebHdfsSensor"],
"apache-airflow-backport-providers-apache-hive==2020.5.20rc2": ["airflow.providers.apache.hive.hooks.hive.HiveCliHook", "airflow.providers.apache.hive.hooks.hive.HiveMetastoreHook", "airflow.providers.apache.hive.hooks.hive.HiveServer2Hook", "airflow.providers.apache.hive.operators.hive.HiveOperator", "airflow.providers.apache.hive.operators.hive_stats.HiveStatsCollectionOperator", "airflow.providers.apache.hive.operators.hive_to_mysql.HiveToMySqlTransfer", "airflow.providers.apache.hive.operators.hive_to_samba.Hive2SambaOperator", "airflow.providers.apache.hive.operators.mssql_to_hive.MsSqlToHiveTransfer", "airflow.providers.apache.hive.operators.mysql_to_hive.MySqlToHiveTransfer", "airflow.providers.apache.hive.operators.s3_to_hive.S3ToHiveTransfer", "airflow.providers.apache.hive.operators.vertica_to_hive.VerticaToHiveTransfer", "airflow.providers.apache.hive.sensors.hive_partition.HivePartitionSensor", "airflow.providers.apache.hive.sensors.metastore_partition.MetastorePartitionSensor", "airflow.providers.apache.hive.sensors.named_hive_partition.NamedHivePartitionSensor"],
"apache-airflow-backport-providers-apache-livy==2020.5.20rc2": ["airflow.providers.apache.livy.hooks.livy.LivyHook", "airflow.providers.apache.livy.operators.livy.LivyOperator", "airflow.providers.apache.livy.sensors.livy.LivySensor"],
"apache-airflow-backport-providers-apache-pig==2020.5.20rc2": ["airflow.providers.apache.pig.hooks.pig.PigCliHook", "airflow.providers.apache.pig.operators.pig.PigOperator"],
"apache-airflow-backport-providers-apache-pinot==2020.5.20rc2": ["airflow.providers.apache.pinot.hooks.pinot.PinotAdminHook", "airflow.providers.apache.pinot.hooks.pinot.PinotDbApiHook"],
"apache-airflow-backport-providers-apache-spark==2020.5.20rc2": ["airflow.providers.apache.spark.hooks.spark_jdbc.SparkJDBCHook", "airflow.providers.apache.spark.hooks.spark_sql.SparkSqlHook", "airflow.providers.apache.spark.hooks.spark_submit.SparkSubmitHook", "airflow.providers.apache.spark.operators.spark_jdbc.SparkJDBCOperator", "airflow.providers.apache.spark.operators.spark_sql.SparkSqlOperator", "airflow.providers.apache.spark.operators.spark_submit.SparkSubmitOperator"],
"apache-airflow-backport-providers-apache-sqoop==2020.5.20rc2": ["airflow.providers.apache.sqoop.hooks.sqoop.SqoopHook", "airflow.providers.apache.sqoop.operators.sqoop.SqoopOperator"],
"apache-airflow-backport-providers-celery==2020.5.20rc2": ["airflow.providers.celery.sensors.celery_queue.CeleryQueueSensor"],
"apache-airflow-backport-providers-cloudant==2020.5.20rc2": ["airflow.providers.cloudant.hooks.cloudant.CloudantHook"],
"apache-airflow-backport-providers-databricks==2020.5.20rc2": ["airflow.providers.databricks.hooks.databricks.DatabricksHook", "airflow.providers.databricks.operators.databricks.DatabricksRunNowOperator", "airflow.providers.databricks.operators.databricks.DatabricksSubmitRunOperator"],
"apache-airflow-backport-providers-datadog==2020.5.20rc2": ["airflow.providers.datadog.hooks.datadog.DatadogHook", "airflow.providers.datadog.sensors.datadog.DatadogSensor"],
"apache-airflow-backport-providers-dingding==2020.5.20rc2": ["airflow.providers.dingding.hooks.dingding.DingdingHook", "airflow.providers.dingding.operators.dingding.DingdingOperator"],
"apache-airflow-backport-providers-discord==2020.5.20rc2": ["airflow.providers.discord.hooks.discord_webhook.DiscordWebhookHook", "airflow.providers.discord.operators.discord_webhook.DiscordWebhookOperator"],
"apache-airflow-backport-providers-docker==2020.5.20rc2": ["airflow.providers.docker.hooks.docker.DockerHook", "airflow.providers.docker.operators.docker.DockerOperator", "airflow.providers.docker.operators.docker_swarm.DockerSwarmOperator"],
"apache-airflow-backport-providers-elasticsearch==2020.5.20rc2": ["airflow.providers.elasticsearch.hooks.elasticsearch.ElasticsearchHook"],
"apache-airflow-backport-providers-email==2020.5.20rc2": ["airflow.providers.email.operators.email.EmailOperator"],
"apache-airflow-backport-providers-exasol==2020.5.20rc2": ["airflow.providers.exasol.hooks.exasol.ExasolHook", "airflow.providers.exasol.operators.exasol.ExasolOperator"],
"apache-airflow-backport-providers-facebook==2020.5.20rc2": ["airflow.providers.facebook.ads.hooks.ads.FacebookAdsReportingHook"],
"apache-airflow-backport-providers-ftp==2020.5.20rc2": ["airflow.providers.ftp.sensors.ftp.FTPSensor", "airflow.providers.ftp.sensors.ftp.FTPSSensor", "airflow.providers.ftp.hooks.ftp.FTPHook", "airflow.providers.ftp.hooks.ftp.FTPSHook"],
"apache-airflow-backport-providers-google==2020.5.20rc2": ["airflow.providers.google.ads.hooks.ads.GoogleAdsHook", "airflow.providers.google.ads.operators.ads.GoogleAdsListAccountsOperator", "airflow.providers.google.ads.operators.ads.GoogleAdsToGcsOperator", "airflow.providers.google.cloud.hooks.automl.CloudAutoMLHook", "airflow.providers.google.cloud.hooks.bigquery.BigQueryHook", "airflow.providers.google.cloud.hooks.bigquery_dts.BiqQueryDataTransferServiceHook", "airflow.providers.google.cloud.hooks.bigtable.BigtableHook", "airflow.providers.google.cloud.hooks.cloud_build.CloudBuildHook", "airflow.providers.google.cloud.hooks.cloud_memorystore.CloudMemorystoreHook", "airflow.providers.google.cloud.hooks.cloud_sql.CloudSQLDatabaseHook", "airflow.providers.google.cloud.hooks.cloud_sql.CloudSQLHook", "airflow.providers.google.cloud.hooks.cloud_storage_transfer_service.CloudDataTransferServiceHook", "airflow.providers.google.cloud.hooks.compute.ComputeEngineHook", "airflow.providers.google.cloud.hooks.datacatalog.CloudDataCatalogHook", "airflow.providers.google.cloud.hooks.dataflow.DataflowHook", "airflow.providers.google.cloud.hooks.datafusion.DataFusionHook", "airflow.providers.google.cloud.hooks.dataproc.DataprocHook", "airflow.providers.google.cloud.hooks.datastore.DatastoreHook", "airflow.providers.google.cloud.hooks.dlp.CloudDLPHook", "airflow.providers.google.cloud.hooks.functions.CloudFunctionsHook", "airflow.providers.google.cloud.hooks.gcs.GCSHook", "airflow.providers.google.cloud.hooks.kms.CloudKMSHook", "airflow.providers.google.cloud.hooks.kubernetes_engine.GKEHook", "airflow.providers.google.cloud.hooks.life_sciences.LifeSciencesHook", "airflow.providers.google.cloud.hooks.mlengine.MLEngineHook", "airflow.providers.google.cloud.hooks.natural_language.CloudNaturalLanguageHook", "airflow.providers.google.cloud.hooks.pubsub.PubSubHook", "airflow.providers.google.cloud.hooks.spanner.SpannerHook", "airflow.providers.google.cloud.hooks.speech_to_text.CloudSpeechToTextHook", "airflow.providers.google.cloud.hooks.stackdriver.StackdriverHook", "airflow.providers.google.cloud.hooks.tasks.CloudTasksHook", "airflow.providers.google.cloud.hooks.text_to_speech.CloudTextToSpeechHook", "airflow.providers.google.cloud.hooks.translate.CloudTranslateHook", "airflow.providers.google.cloud.hooks.video_intelligence.CloudVideoIntelligenceHook", "airflow.providers.google.cloud.hooks.vision.CloudVisionHook", "airflow.providers.google.cloud.operators.adls_to_gcs.ADLSToGCSOperator", "airflow.providers.google.cloud.operators.automl.AutoMLBatchPredictOperator", "airflow.providers.google.cloud.operators.automl.AutoMLCreateDatasetOperator", "airflow.providers.google.cloud.operators.automl.AutoMLDeleteDatasetOperator", "airflow.providers.google.cloud.operators.automl.AutoMLDeleteModelOperator", "airflow.providers.google.cloud.operators.automl.AutoMLDeployModelOperator", "airflow.providers.google.cloud.operators.automl.AutoMLGetModelOperator", "airflow.providers.google.cloud.operators.automl.AutoMLImportDataOperator", "airflow.providers.google.cloud.operators.automl.AutoMLListDatasetOperator", "airflow.providers.google.cloud.operators.automl.AutoMLPredictOperator", "airflow.providers.google.cloud.operators.automl.AutoMLTablesListColumnSpecsOperator", "airflow.providers.google.cloud.operators.automl.AutoMLTablesListTableSpecsOperator", "airflow.providers.google.cloud.operators.automl.AutoMLTablesUpdateDatasetOperator", "airflow.providers.google.cloud.operators.automl.AutoMLTrainModelOperator", "airflow.providers.google.cloud.operators.bigquery.BigQueryCheckOperator", "airflow.providers.google.cloud.operators.bigquery.BigQueryCreateEmptyDatasetOperator", "airflow.providers.google.cloud.operators.bigquery.BigQueryCreateEmptyTableOperator", "airflow.providers.google.cloud.operators.bigquery.BigQueryCreateExternalTableOperator", "airflow.providers.google.cloud.operators.bigquery.BigQueryDeleteDatasetOperator", "airflow.providers.google.cloud.operators.bigquery.BigQueryDeleteTableOperator", "airflow.providers.google.cloud.operators.bigquery.BigQueryExecuteQueryOperator", "airflow.providers.google.cloud.operators.bigquery.BigQueryGetDataOperator", "airflow.providers.google.cloud.operators.bigquery.BigQueryGetDatasetOperator", "airflow.providers.google.cloud.operators.bigquery.BigQueryGetDatasetTablesOperator", "airflow.providers.google.cloud.operators.bigquery.BigQueryIntervalCheckOperator", "airflow.providers.google.cloud.operators.bigquery.BigQueryPatchDatasetOperator", "airflow.providers.google.cloud.operators.bigquery.BigQueryUpdateDatasetOperator", "airflow.providers.google.cloud.operators.bigquery.BigQueryUpsertTableOperator", "airflow.providers.google.cloud.operators.bigquery.BigQueryValueCheckOperator", "airflow.providers.google.cloud.operators.bigquery_dts.BigQueryCreateDataTransferOperator", "airflow.providers.google.cloud.operators.bigquery_dts.BigQueryDataTransferServiceStartTransferRunsOperator", "airflow.providers.google.cloud.operators.bigquery_dts.BigQueryDeleteDataTransferConfigOperator", "airflow.providers.google.cloud.operators.bigquery_to_bigquery.BigQueryToBigQueryOperator", "airflow.providers.google.cloud.operators.bigquery_to_gcs.BigQueryToGCSOperator", "airflow.providers.google.cloud.operators.bigquery_to_mysql.BigQueryToMySqlOperator", "airflow.providers.google.cloud.operators.bigtable.BigtableCreateInstanceOperator", "airflow.providers.google.cloud.operators.bigtable.BigtableCreateTableOperator", "airflow.providers.google.cloud.operators.bigtable.BigtableDeleteInstanceOperator", "airflow.providers.google.cloud.operators.bigtable.BigtableDeleteTableOperator", "airflow.providers.google.cloud.operators.bigtable.BigtableUpdateClusterOperator", "airflow.providers.google.cloud.operators.cassandra_to_gcs.CassandraToGCSOperator", "airflow.providers.google.cloud.operators.cloud_build.CloudBuildCreateOperator", "airflow.providers.google.cloud.operators.cloud_memorystore.CloudMemorystoreCreateInstanceAndImportOperator", "airflow.providers.google.cloud.operators.cloud_memorystore.CloudMemorystoreCreateInstanceOperator", "airflow.providers.google.cloud.operators.cloud_memorystore.CloudMemorystoreDeleteInstanceOperator", "airflow.providers.google.cloud.operators.cloud_memorystore.CloudMemorystoreExportAndDeleteInstanceOperator", "airflow.providers.google.cloud.operators.cloud_memorystore.CloudMemorystoreExportInstanceOperator", "airflow.providers.google.cloud.operators.cloud_memorystore.CloudMemorystoreFailoverInstanceOperator", "airflow.providers.google.cloud.operators.cloud_memorystore.CloudMemorystoreGetInstanceOperator", "airflow.providers.google.cloud.operators.cloud_memorystore.CloudMemorystoreImportOperator", "airflow.providers.google.cloud.operators.cloud_memorystore.CloudMemorystoreListInstancesOperator", "airflow.providers.google.cloud.operators.cloud_memorystore.CloudMemorystoreScaleInstanceOperator", "airflow.providers.google.cloud.operators.cloud_memorystore.CloudMemorystoreUpdateInstanceOperator", "airflow.providers.google.cloud.operators.cloud_sql.CloudSQLBaseOperator", "airflow.providers.google.cloud.operators.cloud_sql.CloudSQLCreateInstanceDatabaseOperator", "airflow.providers.google.cloud.operators.cloud_sql.CloudSQLCreateInstanceOperator", "airflow.providers.google.cloud.operators.cloud_sql.CloudSQLDeleteInstanceDatabaseOperator", "airflow.providers.google.cloud.operators.cloud_sql.CloudSQLDeleteInstanceOperator", "airflow.providers.google.cloud.operators.cloud_sql.CloudSQLExecuteQueryOperator", "airflow.providers.google.cloud.operators.cloud_sql.CloudSQLExportInstanceOperator", "airflow.providers.google.cloud.operators.cloud_sql.CloudSQLImportInstanceOperator", "airflow.providers.google.cloud.operators.cloud_sql.CloudSQLInstancePatchOperator", "airflow.providers.google.cloud.operators.cloud_sql.CloudSQLPatchInstanceDatabaseOperator", "airflow.providers.google.cloud.operators.cloud_storage_transfer_service.CloudDataTransferServiceCancelOperationOperator", "airflow.providers.google.cloud.operators.cloud_storage_transfer_service.CloudDataTransferServiceCreateJobOperator", "airflow.providers.google.cloud.operators.cloud_storage_transfer_service.CloudDataTransferServiceDeleteJobOperator", "airflow.providers.google.cloud.operators.cloud_storage_transfer_service.CloudDataTransferServiceGCSToGCSOperator", "airflow.providers.google.cloud.operators.cloud_storage_transfer_service.CloudDataTransferServiceGetOperationOperator", "airflow.providers.google.cloud.operators.cloud_storage_transfer_service.CloudDataTransferServiceListOperationsOperator", "airflow.providers.google.cloud.operators.cloud_storage_transfer_service.CloudDataTransferServicePauseOperationOperator", "airflow.providers.google.cloud.operators.cloud_storage_transfer_service.CloudDataTransferServiceResumeOperationOperator", "airflow.providers.google.cloud.operators.cloud_storage_transfer_service.CloudDataTransferServiceS3ToGCSOperator", "airflow.providers.google.cloud.operators.cloud_storage_transfer_service.CloudDataTransferServiceUpdateJobOperator", "airflow.providers.google.cloud.operators.compute.ComputeEngineBaseOperator", "airflow.providers.google.cloud.operators.compute.ComputeEngineCopyInstanceTemplateOperator", "airflow.providers.google.cloud.operators.compute.ComputeEngineInstanceGroupUpdateManagerTemplateOperator", "airflow.providers.google.cloud.operators.compute.ComputeEngineSetMachineTypeOperator", "airflow.providers.google.cloud.operators.compute.ComputeEngineStartInstanceOperator", "airflow.providers.google.cloud.operators.compute.ComputeEngineStopInstanceOperator", "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogCreateEntryGroupOperator", "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogCreateEntryOperator", "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogCreateTagOperator", "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogCreateTagTemplateFieldOperator", "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogCreateTagTemplateOperator", "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogDeleteEntryGroupOperator", "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogDeleteEntryOperator", "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogDeleteTagOperator", "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogDeleteTagTemplateFieldOperator", "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogDeleteTagTemplateOperator", "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogGetEntryGroupOperator", "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogGetEntryOperator", "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogGetTagTemplateOperator", "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogListTagsOperator", "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogLookupEntryOperator", "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogRenameTagTemplateFieldOperator", "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogSearchCatalogOperator", "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogUpdateEntryOperator", "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogUpdateTagOperator", "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogUpdateTagTemplateFieldOperator", "airflow.providers.google.cloud.operators.datacatalog.CloudDataCatalogUpdateTagTemplateOperator", "airflow.providers.google.cloud.operators.dataflow.DataflowCreateJavaJobOperator", "airflow.providers.google.cloud.operators.dataflow.DataflowCreatePythonJobOperator", "airflow.providers.google.cloud.operators.dataflow.DataflowTemplatedJobStartOperator", "airflow.providers.google.cloud.operators.datafusion.CloudDataFusionCreateInstanceOperator", "airflow.providers.google.cloud.operators.datafusion.CloudDataFusionCreatePipelineOperator", "airflow.providers.google.cloud.operators.datafusion.CloudDataFusionDeleteInstanceOperator", "airflow.providers.google.cloud.operators.datafusion.CloudDataFusionDeletePipelineOperator", "airflow.providers.google.cloud.operators.datafusion.CloudDataFusionGetInstanceOperator", "airflow.providers.google.cloud.operators.datafusion.CloudDataFusionListPipelinesOperator", "airflow.providers.google.cloud.operators.datafusion.CloudDataFusionRestartInstanceOperator", "airflow.providers.google.cloud.operators.datafusion.CloudDataFusionStartPipelineOperator", "airflow.providers.google.cloud.operators.datafusion.CloudDataFusionStopPipelineOperator", "airflow.providers.google.cloud.operators.datafusion.CloudDataFusionUpdateInstanceOperator", "airflow.providers.google.cloud.operators.dataproc.DataprocCreateClusterOperator", "airflow.providers.google.cloud.operators.dataproc.DataprocDeleteClusterOperator", "airflow.providers.google.cloud.operators.dataproc.DataprocInstantiateInlineWorkflowTemplateOperator", "airflow.providers.google.cloud.operators.dataproc.DataprocInstantiateWorkflowTemplateOperator", "airflow.providers.google.cloud.operators.dataproc.DataprocJobBaseOperator", "airflow.providers.google.cloud.operators.dataproc.DataprocScaleClusterOperator", "airflow.providers.google.cloud.operators.dataproc.DataprocSubmitHadoopJobOperator", "airflow.providers.google.cloud.operators.dataproc.DataprocSubmitHiveJobOperator", "airflow.providers.google.cloud.operators.dataproc.DataprocSubmitJobOperator", "airflow.providers.google.cloud.operators.dataproc.DataprocSubmitPigJobOperator", "airflow.providers.google.cloud.operators.dataproc.DataprocSubmitPySparkJobOperator", "airflow.providers.google.cloud.operators.dataproc.DataprocSubmitSparkJobOperator", "airflow.providers.google.cloud.operators.dataproc.DataprocSubmitSparkSqlJobOperator", "airflow.providers.google.cloud.operators.dataproc.DataprocUpdateClusterOperator", "airflow.providers.google.cloud.operators.datastore.CloudDatastoreExportEntitiesOperator", "airflow.providers.google.cloud.operators.datastore.CloudDatastoreImportEntitiesOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPCancelDLPJobOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPCreateDeidentifyTemplateOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPCreateDLPJobOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPCreateInspectTemplateOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPCreateJobTriggerOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPCreateStoredInfoTypeOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPDeidentifyContentOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPDeleteDeidentifyTemplateOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPDeleteDLPJobOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPDeleteInspectTemplateOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPDeleteJobTriggerOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPDeleteStoredInfoTypeOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPGetDeidentifyTemplateOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPGetDLPJobOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPGetDLPJobTriggerOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPGetInspectTemplateOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPGetStoredInfoTypeOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPInspectContentOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPListDeidentifyTemplatesOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPListDLPJobsOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPListInfoTypesOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPListInspectTemplatesOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPListJobTriggersOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPListStoredInfoTypesOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPRedactImageOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPReidentifyContentOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPUpdateDeidentifyTemplateOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPUpdateInspectTemplateOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPUpdateJobTriggerOperator", "airflow.providers.google.cloud.operators.dlp.CloudDLPUpdateStoredInfoTypeOperator", "airflow.providers.google.cloud.operators.facebook_ads_to_gcs.FacebookAdsReportToGcsOperator", "airflow.providers.google.cloud.operators.functions.CloudFunctionDeleteFunctionOperator", "airflow.providers.google.cloud.operators.functions.CloudFunctionDeployFunctionOperator", "airflow.providers.google.cloud.operators.functions.CloudFunctionInvokeFunctionOperator", "airflow.providers.google.cloud.operators.gcs.GCSBucketCreateAclEntryOperator", "airflow.providers.google.cloud.operators.gcs.GCSCreateBucketOperator", "airflow.providers.google.cloud.operators.gcs.GCSDeleteBucketOperator", "airflow.providers.google.cloud.operators.gcs.GCSDeleteObjectsOperator", "airflow.providers.google.cloud.operators.gcs.GCSFileTransformOperator", "airflow.providers.google.cloud.operators.gcs.GCSListObjectsOperator", "airflow.providers.google.cloud.operators.gcs.GCSObjectCreateAclEntryOperator", "airflow.providers.google.cloud.operators.gcs.GCSToLocalOperator", "airflow.providers.google.cloud.operators.gcs_to_bigquery.GCSToBigQueryOperator", "airflow.providers.google.cloud.operators.gcs_to_gcs.GCSToGCSOperator", "airflow.providers.google.cloud.operators.gcs_to_sftp.GCSToSFTPOperator", "airflow.providers.google.cloud.operators.kubernetes_engine.GKECreateClusterOperator", "airflow.providers.google.cloud.operators.kubernetes_engine.GKEDeleteClusterOperator", "airflow.providers.google.cloud.operators.kubernetes_engine.GKEStartPodOperator", "airflow.providers.google.cloud.operators.life_sciences.LifeSciencesRunPipelineOperator", "airflow.providers.google.cloud.operators.local_to_gcs.LocalFilesystemToGCSOperator", "airflow.providers.google.cloud.operators.mlengine.MLEngineCreateModelOperator", "airflow.providers.google.cloud.operators.mlengine.MLEngineCreateVersionOperator", "airflow.providers.google.cloud.operators.mlengine.MLEngineDeleteModelOperator", "airflow.providers.google.cloud.operators.mlengine.MLEngineDeleteVersionOperator", "airflow.providers.google.cloud.operators.mlengine.MLEngineGetModelOperator", "airflow.providers.google.cloud.operators.mlengine.MLEngineListVersionsOperator", "airflow.providers.google.cloud.operators.mlengine.MLEngineManageModelOperator", "airflow.providers.google.cloud.operators.mlengine.MLEngineManageVersionOperator", "airflow.providers.google.cloud.operators.mlengine.MLEngineSetDefaultVersionOperator", "airflow.providers.google.cloud.operators.mlengine.MLEngineStartBatchPredictionJobOperator", "airflow.providers.google.cloud.operators.mlengine.MLEngineStartTrainingJobOperator", "airflow.providers.google.cloud.operators.mlengine.MLEngineTrainingCancelJobOperator", "airflow.providers.google.cloud.operators.mssql_to_gcs.MSSQLToGCSOperator", "airflow.providers.google.cloud.operators.mysql_to_gcs.MySQLToGCSOperator", "airflow.providers.google.cloud.operators.natural_language.CloudNaturalLanguageAnalyzeEntitiesOperator", "airflow.providers.google.cloud.operators.natural_language.CloudNaturalLanguageAnalyzeEntitySentimentOperator", "airflow.providers.google.cloud.operators.natural_language.CloudNaturalLanguageAnalyzeSentimentOperator", "airflow.providers.google.cloud.operators.natural_language.CloudNaturalLanguageClassifyTextOperator", "airflow.providers.google.cloud.operators.postgres_to_gcs.PostgresToGCSOperator", "airflow.providers.google.cloud.operators.presto_to_gcs.PrestoToGCSOperator", "airflow.providers.google.cloud.operators.pubsub.PubSubCreateSubscriptionOperator", "airflow.providers.google.cloud.operators.pubsub.PubSubCreateTopicOperator", "airflow.providers.google.cloud.operators.pubsub.PubSubDeleteSubscriptionOperator", "airflow.providers.google.cloud.operators.pubsub.PubSubDeleteTopicOperator", "airflow.providers.google.cloud.operators.pubsub.PubSubPublishMessageOperator", "airflow.providers.google.cloud.operators.pubsub.PubSubPullOperator", "airflow.providers.google.cloud.operators.s3_to_gcs.S3ToGCSOperator", "airflow.providers.google.cloud.operators.sftp_to_gcs.SFTPToGCSOperator", "airflow.providers.google.cloud.operators.sheets_to_gcs.GoogleSheetsToGCSOperator", "airflow.providers.google.cloud.operators.spanner.SpannerDeleteDatabaseInstanceOperator", "airflow.providers.google.cloud.operators.spanner.SpannerDeleteInstanceOperator", "airflow.providers.google.cloud.operators.spanner.SpannerDeployDatabaseInstanceOperator", "airflow.providers.google.cloud.operators.spanner.SpannerDeployInstanceOperator", "airflow.providers.google.cloud.operators.spanner.SpannerQueryDatabaseInstanceOperator", "airflow.providers.google.cloud.operators.spanner.SpannerUpdateDatabaseInstanceOperator", "airflow.providers.google.cloud.operators.speech_to_text.CloudSpeechToTextRecognizeSpeechOperator", "airflow.providers.google.cloud.operators.sql_to_gcs.BaseSQLToGCSOperator", "airflow.providers.google.cloud.operators.stackdriver.StackdriverDeleteAlertOperator", "airflow.providers.google.cloud.operators.stackdriver.StackdriverDeleteNotificationChannelOperator", "airflow.providers.google.cloud.operators.stackdriver.StackdriverDisableAlertPoliciesOperator", "airflow.providers.google.cloud.operators.stackdriver.StackdriverDisableNotificationChannelsOperator", "airflow.providers.google.cloud.operators.stackdriver.StackdriverEnableAlertPoliciesOperator", "airflow.providers.google.cloud.operators.stackdriver.StackdriverEnableNotificationChannelsOperator", "airflow.providers.google.cloud.operators.stackdriver.StackdriverListAlertPoliciesOperator", "airflow.providers.google.cloud.operators.stackdriver.StackdriverListNotificationChannelsOperator", "airflow.providers.google.cloud.operators.stackdriver.StackdriverUpsertAlertOperator", "airflow.providers.google.cloud.operators.stackdriver.StackdriverUpsertNotificationChannelOperator", "airflow.providers.google.cloud.operators.tasks.CloudTasksQueueCreateOperator", "airflow.providers.google.cloud.operators.tasks.CloudTasksQueueDeleteOperator", "airflow.providers.google.cloud.operators.tasks.CloudTasksQueueGetOperator", "airflow.providers.google.cloud.operators.tasks.CloudTasksQueuePauseOperator", "airflow.providers.google.cloud.operators.tasks.CloudTasksQueuePurgeOperator", "airflow.providers.google.cloud.operators.tasks.CloudTasksQueueResumeOperator", "airflow.providers.google.cloud.operators.tasks.CloudTasksQueuesListOperator", "airflow.providers.google.cloud.operators.tasks.CloudTasksQueueUpdateOperator", "airflow.providers.google.cloud.operators.tasks.CloudTasksTaskCreateOperator", "airflow.providers.google.cloud.operators.tasks.CloudTasksTaskDeleteOperator", "airflow.providers.google.cloud.operators.tasks.CloudTasksTaskGetOperator", "airflow.providers.google.cloud.operators.tasks.CloudTasksTaskRunOperator", "airflow.providers.google.cloud.operators.tasks.CloudTasksTasksListOperator", "airflow.providers.google.cloud.operators.text_to_speech.CloudTextToSpeechSynthesizeOperator", "airflow.providers.google.cloud.operators.translate.CloudTranslateTextOperator", "airflow.providers.google.cloud.operators.translate_speech.CloudTranslateSpeechOperator", "airflow.providers.google.cloud.operators.video_intelligence.CloudVideoIntelligenceDetectVideoExplicitContentOperator", "airflow.providers.google.cloud.operators.video_intelligence.CloudVideoIntelligenceDetectVideoLabelsOperator", "airflow.providers.google.cloud.operators.video_intelligence.CloudVideoIntelligenceDetectVideoShotsOperator", "airflow.providers.google.cloud.operators.vision.CloudVisionAddProductToProductSetOperator", "airflow.providers.google.cloud.operators.vision.CloudVisionCreateProductOperator", "airflow.providers.google.cloud.operators.vision.CloudVisionCreateProductSetOperator", "airflow.providers.google.cloud.operators.vision.CloudVisionCreateReferenceImageOperator", "airflow.providers.google.cloud.operators.vision.CloudVisionDeleteProductOperator", "airflow.providers.google.cloud.operators.vision.CloudVisionDeleteProductSetOperator", "airflow.providers.google.cloud.operators.vision.CloudVisionDetectImageLabelsOperator", "airflow.providers.google.cloud.operators.vision.CloudVisionDetectImageSafeSearchOperator", "airflow.providers.google.cloud.operators.vision.CloudVisionDetectTextOperator", "airflow.providers.google.cloud.operators.vision.CloudVisionGetProductOperator", "airflow.providers.google.cloud.operators.vision.CloudVisionGetProductSetOperator", "airflow.providers.google.cloud.operators.vision.CloudVisionImageAnnotateOperator", "airflow.providers.google.cloud.operators.vision.CloudVisionRemoveProductFromProductSetOperator", "airflow.providers.google.cloud.operators.vision.CloudVisionTextDetectOperator", "airflow.providers.google.cloud.operators.vision.CloudVisionUpdateProductOperator", "airflow.providers.google.cloud.operators.vision.CloudVisionUpdateProductSetOperator", "airflow.providers.google.cloud.secrets.secrets_manager.CloudSecretsManagerBackend", "airflow.providers.google.cloud.sensors.bigquery.BigQueryTableExistenceSensor", "airflow.providers.google.cloud.sensors.bigquery_dts.BigQueryDataTransferServiceTransferRunSensor", "airflow.providers.google.cloud.sensors.bigtable.BigtableTableReplicationCompletedSensor", "airflow.providers.google.cloud.sensors.cloud_storage_transfer_service.CloudDataTransferServiceJobStatusSensor", "airflow.providers.google.cloud.sensors.gcs.GCSObjectExistenceSensor", "airflow.providers.google.cloud.sensors.gcs.GCSObjectsWtihPrefixExistenceSensor", "airflow.providers.google.cloud.sensors.gcs.GCSObjectUpdateSensor", "airflow.providers.google.cloud.sensors.gcs.GCSUploadSessionCompleteSensor", "airflow.providers.google.cloud.sensors.pubsub.PubSubPullSensor", "airflow.providers.google.common.hooks.base_google.GoogleBaseHook", "airflow.providers.google.common.hooks.discovery_api.GoogleDiscoveryApiHook", "airflow.providers.google.firebase.hooks.firestore.CloudFirestoreHook", "airflow.providers.google.firebase.operators.firestore.CloudFirestoreExportDatabaseOperator", "airflow.providers.google.marketing_platform.hooks.analytics.GoogleAnalyticsHook", "airflow.providers.google.marketing_platform.hooks.campaign_manager.GoogleCampaignManagerHook", "airflow.providers.google.marketing_platform.hooks.display_video.GoogleDisplayVideo360Hook", "airflow.providers.google.marketing_platform.hooks.search_ads.GoogleSearchAdsHook", "airflow.providers.google.marketing_platform.operators.analytics.GoogleAnalyticsDataImportUploadOperator", "airflow.providers.google.marketing_platform.operators.analytics.GoogleAnalyticsDeletePreviousDataUploadsOperator", "airflow.providers.google.marketing_platform.operators.analytics.GoogleAnalyticsGetAdsLinkOperator", "airflow.providers.google.marketing_platform.operators.analytics.GoogleAnalyticsListAccountsOperator", "airflow.providers.google.marketing_platform.operators.analytics.GoogleAnalyticsModifyFileHeadersDataImportOperator", "airflow.providers.google.marketing_platform.operators.analytics.GoogleAnalyticsRetrieveAdsLinksListOperator", "airflow.providers.google.marketing_platform.operators.campaign_manager.GoogleCampaignManagerBatchInsertConversionsOperator", "airflow.providers.google.marketing_platform.operators.campaign_manager.GoogleCampaignManagerBatchUpdateConversionsOperator", "airflow.providers.google.marketing_platform.operators.campaign_manager.GoogleCampaignManagerDeleteReportOperator", "airflow.providers.google.marketing_platform.operators.campaign_manager.GoogleCampaignManagerDownloadReportOperator", "airflow.providers.google.marketing_platform.operators.campaign_manager.GoogleCampaignManagerInsertReportOperator", "airflow.providers.google.marketing_platform.operators.campaign_manager.GoogleCampaignManagerRunReportOperator", "airflow.providers.google.marketing_platform.operators.display_video.GoogleDisplayVideo360CreateReportOperator", "airflow.providers.google.marketing_platform.operators.display_video.GoogleDisplayVideo360CreateSDFDownloadTaskOperator", "airflow.providers.google.marketing_platform.operators.display_video.GoogleDisplayVideo360DeleteReportOperator", "airflow.providers.google.marketing_platform.operators.display_video.GoogleDisplayVideo360DownloadLineItemsOperator", "airflow.providers.google.marketing_platform.operators.display_video.GoogleDisplayVideo360DownloadReportOperator", "airflow.providers.google.marketing_platform.operators.display_video.GoogleDisplayVideo360RunReportOperator", "airflow.providers.google.marketing_platform.operators.display_video.GoogleDisplayVideo360SDFtoGCSOperator", "airflow.providers.google.marketing_platform.operators.display_video.GoogleDisplayVideo360UploadLineItemsOperator", "airflow.providers.google.marketing_platform.operators.search_ads.GoogleSearchAdsDownloadReportOperator", "airflow.providers.google.marketing_platform.operators.search_ads.GoogleSearchAdsInsertReportOperator", "airflow.providers.google.marketing_platform.sensors.campaign_manager.GoogleCampaignManagerReportSensor", "airflow.providers.google.marketing_platform.sensors.display_video.GoogleDisplayVideo360GetSDFDownloadOperationSensor", "airflow.providers.google.marketing_platform.sensors.display_video.GoogleDisplayVideo360ReportSensor", "airflow.providers.google.marketing_platform.sensors.search_ads.GoogleSearchAdsReportSensor", "airflow.providers.google.suite.hooks.drive.GoogleDriveHook", "airflow.providers.google.suite.hooks.sheets.GSheetsHook", "airflow.providers.google.suite.operators.gcs_to_gdrive.GCSToGoogleDriveOperator", "airflow.providers.google.suite.operators.gcs_to_sheets.GCSToGoogleSheetsOperator", "airflow.providers.google.suite.operators.sheets.GoogleSheetsCreateSpreadsheet"],
"apache-airflow-backport-providers-grpc==2020.5.20rc2": ["airflow.providers.grpc.hooks.grpc.GrpcHook", "airflow.providers.grpc.operators.grpc.GrpcOperator"],
"apache-airflow-backport-providers-hashicorp==2020.5.20rc2": ["airflow.providers.hashicorp.secrets.vault.VaultBackend"],
"apache-airflow-backport-providers-http==2020.5.20rc2": ["airflow.providers.http.hooks.http.HttpHook", "airflow.providers.http.operators.http.SimpleHttpOperator", "airflow.providers.http.sensors.http.HttpSensor"],
"apache-airflow-backport-providers-imap==2020.5.20rc2": ["airflow.providers.imap.hooks.imap.ImapHook", "airflow.providers.imap.sensors.imap_attachment.ImapAttachmentSensor"],
"apache-airflow-backport-providers-jdbc==2020.5.20rc2": ["airflow.providers.jdbc.hooks.jdbc.JdbcHook", "airflow.providers.jdbc.operators.jdbc.JdbcOperator"],
"apache-airflow-backport-providers-jenkins==2020.5.20rc2": ["airflow.providers.jenkins.hooks.jenkins.JenkinsHook", "airflow.providers.jenkins.operators.jenkins_job_trigger.JenkinsJobTriggerOperator"],
"apache-airflow-backport-providers-jira==2020.5.20rc2": ["airflow.providers.jira.hooks.jira.JiraHook", "airflow.providers.jira.operators.jira.JiraOperator", "airflow.providers.jira.sensors.jira.JiraSensor", "airflow.providers.jira.sensors.jira.JiraTicketSensor"],
"apache-airflow-backport-providers-microsoft-azure==2020.5.20rc2": ["airflow.providers.microsoft.azure.hooks.adx.AzureDataExplorerHook", "airflow.providers.microsoft.azure.hooks.azure_batch.AzureBatchHook", "airflow.providers.microsoft.azure.hooks.azure_container_instance.AzureContainerInstanceHook", "airflow.providers.microsoft.azure.hooks.azure_container_registry.AzureContainerRegistryHook", "airflow.providers.microsoft.azure.hooks.azure_container_volume.AzureContainerVolumeHook", "airflow.providers.microsoft.azure.hooks.azure_cosmos.AzureCosmosDBHook", "airflow.providers.microsoft.azure.hooks.azure_data_lake.AzureDataLakeHook", "airflow.providers.microsoft.azure.hooks.azure_fileshare.AzureFileShareHook", "airflow.providers.microsoft.azure.hooks.wasb.WasbHook", "airflow.providers.microsoft.azure.operators.adls_list.AzureDataLakeStorageListOperator", "airflow.providers.microsoft.azure.operators.adx.AzureDataExplorerQueryOperator", "airflow.providers.microsoft.azure.operators.azure_batch.AzureBatchOperator", "airflow.providers.microsoft.azure.operators.azure_container_instances.AzureContainerInstancesOperator", "airflow.providers.microsoft.azure.operators.azure_cosmos.AzureCosmosInsertDocumentOperator", "airflow.providers.microsoft.azure.operators.file_to_wasb.FileToWasbOperator", "airflow.providers.microsoft.azure.operators.oracle_to_azure_data_lake_transfer.OracleToAzureDataLakeTransfer", "airflow.providers.microsoft.azure.operators.wasb_delete_blob.WasbDeleteBlobOperator", "airflow.providers.microsoft.azure.sensors.azure_cosmos.AzureCosmosDocumentSensor", "airflow.providers.microsoft.azure.sensors.wasb.WasbBlobSensor", "airflow.providers.microsoft.azure.sensors.wasb.WasbPrefixSensor"],
"apache-airflow-backport-providers-microsoft-mssql==2020.5.20rc2": ["airflow.providers.microsoft.mssql.hooks.mssql.MsSqlHook", "airflow.providers.microsoft.mssql.operators.mssql.MsSqlOperator"],
"apache-airflow-backport-providers-microsoft-winrm==2020.5.20rc2": ["airflow.providers.microsoft.winrm.hooks.winrm.WinRMHook", "airflow.providers.microsoft.winrm.operators.winrm.WinRMOperator"],
"apache-airflow-backport-providers-mongo==2020.5.20rc2": ["airflow.providers.mongo.hooks.mongo.MongoHook", "airflow.providers.mongo.sensors.mongo.MongoSensor"],
"apache-airflow-backport-providers-mysql==2020.5.20rc2": ["airflow.providers.mysql.hooks.mysql.MySqlHook", "airflow.providers.mysql.operators.mysql.MySqlOperator", "airflow.providers.mysql.operators.presto_to_mysql.PrestoToMySqlTransfer", "airflow.providers.mysql.operators.s3_to_mysql.S3ToMySqlTransfer", "airflow.providers.mysql.operators.vertica_to_mysql.VerticaToMySqlTransfer"],
"apache-airflow-backport-providers-odbc==2020.5.20rc2": ["airflow.providers.odbc.hooks.odbc.OdbcHook"],
"apache-airflow-backport-providers-openfaas==2020.5.20rc2": ["airflow.providers.openfaas.hooks.openfaas.OpenFaasHook"],
"apache-airflow-backport-providers-opsgenie==2020.5.20rc2": ["airflow.providers.opsgenie.hooks.opsgenie_alert.OpsgenieAlertHook", "airflow.providers.opsgenie.operators.opsgenie_alert.OpsgenieAlertOperator"],
"apache-airflow-backport-providers-oracle==2020.5.20rc2": ["airflow.providers.oracle.hooks.oracle.OracleHook", "airflow.providers.oracle.operators.oracle.OracleOperator", "airflow.providers.oracle.operators.oracle_to_oracle_transfer.OracleToOracleTransfer"],
"apache-airflow-backport-providers-pagerduty==2020.5.20rc2": ["airflow.providers.pagerduty.hooks.pagerduty.PagerdutyHook"],
"apache-airflow-backport-providers-papermill==2020.5.20rc2": ["airflow.providers.papermill.operators.papermill.PapermillOperator"],
"apache-airflow-backport-providers-postgres==2020.5.20rc2": ["airflow.providers.postgres.hooks.postgres.PostgresHook", "airflow.providers.postgres.operators.postgres.PostgresOperator"],
"apache-airflow-backport-providers-presto==2020.5.20rc2": ["airflow.providers.presto.hooks.presto.PrestoHook"],
"apache-airflow-backport-providers-qubole==2020.5.20rc2": ["airflow.providers.qubole.hooks.qubole.QuboleHook", "airflow.providers.qubole.hooks.qubole_check.QuboleCheckHook", "airflow.providers.qubole.operators.qubole.QuboleOperator", "airflow.providers.qubole.operators.qubole_check.QuboleCheckOperator", "airflow.providers.qubole.operators.qubole_check.QuboleValueCheckOperator", "airflow.providers.qubole.sensors.qubole.QuboleFileSensor", "airflow.providers.qubole.sensors.qubole.QubolePartitionSensor", "airflow.providers.qubole.sensors.qubole.QuboleSensor"],
"apache-airflow-backport-providers-redis==2020.5.20rc2": ["airflow.providers.redis.hooks.redis.RedisHook", "airflow.providers.redis.operators.redis_publish.RedisPublishOperator", "airflow.providers.redis.sensors.redis_key.RedisKeySensor", "airflow.providers.redis.sensors.redis_pub_sub.RedisPubSubSensor"],
"apache-airflow-backport-providers-salesforce==2020.5.20rc2": ["airflow.providers.salesforce.hooks.salesforce.SalesforceHook", "airflow.providers.salesforce.hooks.tableau.TableauHook","airflow.providers.salesforce.operators.tableau_refresh_workbook.TableauRefreshWorkbookOperator", "airflow.providers.salesforce.sensors.tableau_job_status.TableauJobStatusSensor"],
"apache-airflow-backport-providers-samba==2020.5.20rc2": ["airflow.providers.samba.hooks.samba.SambaHook"],
"apache-airflow-backport-providers-segment==2020.5.20rc2": ["airflow.providers.segment.hooks.segment.SegmentHook", "airflow.providers.segment.operators.segment_track_event.SegmentTrackEventOperator"],
"apache-airflow-backport-providers-sftp==2020.5.20rc2": ["airflow.providers.sftp.hooks.sftp.SFTPHook", "airflow.providers.sftp.operators.sftp.SFTPOperator", "airflow.providers.sftp.sensors.sftp.SFTPSensor"],
"apache-airflow-backport-providers-singularity==2020.5.20rc2": ["airflow.providers.singularity.operators.singularity.SingularityOperator"],
"apache-airflow-backport-providers-slack==2020.5.20rc2": ["airflow.providers.slack.hooks.slack.SlackHook", "airflow.providers.slack.hooks.slack_webhook.SlackWebhookHook", "airflow.providers.slack.operators.slack.SlackAPIOperator", "airflow.providers.slack.operators.slack.SlackAPIPostOperator", "airflow.providers.slack.operators.slack_webhook.SlackWebhookOperator"],
"apache-airflow-backport-providers-snowflake==2020.5.20rc2": ["airflow.providers.snowflake.hooks.snowflake.SnowflakeHook", "airflow.providers.snowflake.operators.s3_to_snowflake.S3ToSnowflakeTransfer", "airflow.providers.snowflake.operators.snowflake.SnowflakeOperator"],
"apache-airflow-backport-providers-sqlite==2020.5.20rc2": ["airflow.providers.sqlite.hooks.sqlite.SqliteHook", "airflow.providers.sqlite.operators.sqlite.SqliteOperator"],
"apache-airflow-backport-providers-ssh==2020.5.20rc2": ["airflow.providers.ssh.hooks.ssh.SSHHook", "airflow.providers.ssh.operators.ssh.SSHOperator"],
"apache-airflow-backport-providers-vertica==2020.5.20rc2": ["airflow.providers.vertica.hooks.vertica.VerticaHook", "airflow.providers.vertica.operators.vertica.VerticaOperator"],
"apache-airflow-backport-providers-yandex==2020.5.20rc2": ["airflow.providers.yandex.hooks.yandex.YandexCloudBaseHook", "airflow.providers.yandex.hooks.yandexcloud_dataproc.DataprocHook", "airflow.providers.yandex.operators.yandexcloud_dataproc.DataprocCreateClusterOperator", "airflow.providers.yandex.operators.yandexcloud_dataproc.DataprocCreateHiveJobOperator", "airflow.providers.yandex.operators.yandexcloud_dataproc.DataprocCreateMapReduceJobOperator", "airflow.providers.yandex.operators.yandexcloud_dataproc.DataprocCreatePysparkJobOperator", "airflow.providers.yandex.operators.yandexcloud_dataproc.DataprocCreateSparkJobOperator", "airflow.providers.yandex.operators.yandexcloud_dataproc.DataprocDeleteClusterOperator"],
"apache-airflow-backport-providers-zendesk==2020.5.20rc2": ["airflow.providers.zendesk.hooks.zendesk.ZendeskHook"],
}
# fmt: on
client = docker.from_env()
for backport_package, classes in backport_package_class_mapping.items():
print(f"Testing backport package {backport_package}")
container = client.containers.run("python:3.7", "tail -f /dev/null", detach=True) # Keep container running
container.exec_run("pip install apache-airflow==1.10.10")
exit_code, output = container.exec_run(f"pip install {backport_package}")
if exit_code != 0:
print(f"ERROR INSTALLING {backport_package}: {str(output)}")
for import_class in classes:
import_class_parts = import_class.split(".")
import_class_cmd = f"from {'.'.join(import_class_parts[:-1])} import {import_class_parts[-1]}"
exit_code, output = container.exec_run(f'python -c "{import_class_cmd}"')
if exit_code != 0 and "ModuleNotFoundError: No module named \\'airflow.providers." not in str(output):
print(f"{import_class} -> {str(output)}")
container.remove(force=True)
print()
@BasPH
Copy link
Author

BasPH commented May 23, 2020

  • airflow.providers.amazon.aws.operators.batch.AwsBatchOperator -> b'Traceback (most recent call last):\n File "", line 1, in \n File "/usr/local/lib/python3.7/site-packages/airflow/providers/amazon/aws/operators/batch.py", line 33, in \n from airflow.providers.amazon.aws.hooks.batch_client import AwsBatchClient\n File "/usr/local/lib/python3.7/site-packages/airflow/providers/amazon/aws/hooks/batch_client.py", line 39, in \n from airflow.typing_compat import Protocol, runtime_checkable\nModuleNotFoundError: No module named 'airflow.typing_compat'\n'
  • airflow.providers.amazon.aws.operators.dynamodb_to_s3.DynamoDBToS3Operator -> b'Traceback (most recent call last):\n File "", line 1, in \n File "/usr/local/lib/python3.7/site-packages/airflow/providers/amazon/aws/operators/dynamodb_to_s3.py", line 31, in \n from boto.compat import json # type: ignore\nModuleNotFoundError: No module named 'boto'\n'
  • airflow.providers.amazon.aws.operators.ecs.ECSOperator -> b'Traceback (most recent call last):\n File "", line 1, in \n File "/usr/local/lib/python3.7/site-packages/airflow/providers/amazon/aws/operators/ecs.py", line 27, in \n from airflow.typing_compat import Protocol, runtime_checkable\nModuleNotFoundError: No module named 'airflow.typing_compat'\n'
  • airflow.providers.apache.hdfs.hooks.hdfs.HDFSHook -> b'Traceback (most recent call last):\n File "", line 1, in \n File "/usr/local/lib/python3.7/site-packages/airflow/providers/apache/hdfs/hooks/hdfs.py", line 24, in \n from snakebite.client import Client, HAClient, Namenode, AutoConfigClient # pylint: disable=syntax-error\n File "/usr/local/lib/python3.7/site-packages/snakebite/client.py", line 1473\n baseTime = min(time * (1L << retries), cap);\n ^\nSyntaxError: invalid syntax\n'
  • airflow.providers.apache.hdfs.hooks.webhdfs.WebHDFSHook -> b'Traceback (most recent call last):\n File "", line 1, in \n File "/usr/local/lib/python3.7/site-packages/airflow/providers/apache/hdfs/hooks/webhdfs.py", line 22, in \n from hdfs import HdfsError, InsecureClient\nModuleNotFoundError: No module named 'hdfs'\n'
  • airflow.providers.apache.hdfs.sensors.hdfs.HdfsSensor -> b'Traceback (most recent call last):\n File "", line 1, in \n File "/usr/local/lib/python3.7/site-packages/airflow/providers/apache/hdfs/sensors/hdfs.py", line 23, in \n from airflow.providers.apache.hdfs.hooks.hdfs import HDFSHook\n File "/usr/local/lib/python3.7/site-packages/airflow/providers/apache/hdfs/hooks/hdfs.py", line 24, in \n from snakebite.client import Client, HAClient, Namenode, AutoConfigClient # pylint: disable=syntax-error\n File "/usr/local/lib/python3.7/site-packages/snakebite/client.py", line 1473\n baseTime = min(time * (1L << retries), cap);\n ^\nSyntaxError: invalid syntax\n'
  • airflow.providers.elasticsearch.hooks.elasticsearch.ElasticsearchHook -> b'Traceback (most recent call last):\n File "", line 1, in \n File "/usr/local/lib/python3.7/site-packages/airflow/providers/elasticsearch/hooks/elasticsearch.py", line 19, in \n from es.elastic.api import connect\nModuleNotFoundError: No module named 'es'\n'
  • airflow.providers.grpc.hooks.grpc.GrpcHook -> b'Traceback (most recent call last):\n File "", line 1, in \n File "/usr/local/lib/python3.7/site-packages/airflow/providers/grpc/hooks/grpc.py", line 21, in \n from google import auth as google_auth\nModuleNotFoundError: No module named 'google'\n'
  • airflow.providers.grpc.operators.grpc.GrpcOperator -> b'Traceback (most recent call last):\n File "", line 1, in \n File "/usr/local/lib/python3.7/site-packages/airflow/providers/grpc/operators/grpc.py", line 20, in \n from airflow.providers.grpc.hooks.grpc import GrpcHook\n File "/usr/local/lib/python3.7/site-packages/airflow/providers/grpc/hooks/grpc.py", line 21, in \n from google import auth as google_auth\nModuleNotFoundError: No module named 'google'\n'
  • airflow.providers.microsoft.azure.hooks.azure_cosmos.AzureCosmosDBHook -> b'Traceback (most recent call last):\n File "", line 1, in \n File "/usr/local/lib/python3.7/site-packages/airflow/providers/microsoft/azure/hooks/azure_cosmos.py", line 29, in \n from azure.cosmos.errors import HTTPFailure\nImportError: cannot import name 'HTTPFailure' from 'azure.cosmos.errors' (/usr/local/lib/python3.7/site-packages/azure/cosmos/errors.py)\n'
  • airflow.providers.microsoft.azure.operators.azure_cosmos.AzureCosmosInsertDocumentOperator -> b'Traceback (most recent call last):\n File "", line 1, in \n File "/usr/local/lib/python3.7/site-packages/airflow/providers/microsoft/azure/operators/azure_cosmos.py", line 20, in \n from airflow.providers.microsoft.azure.hooks.azure_cosmos import AzureCosmosDBHook\n File "/usr/local/lib/python3.7/site-packages/airflow/providers/microsoft/azure/hooks/azure_cosmos.py", line 29, in \n from azure.cosmos.errors import HTTPFailure\nImportError: cannot import name 'HTTPFailure' from 'azure.cosmos.errors' (/usr/local/lib/python3.7/site-packages/azure/cosmos/errors.py)\n'
  • airflow.providers.microsoft.azure.sensors.azure_cosmos.AzureCosmosDocumentSensor -> b'Traceback (most recent call last):\n File "", line 1, in \n File "/usr/local/lib/python3.7/site-packages/airflow/providers/microsoft/azure/sensors/azure_cosmos.py", line 18, in \n from airflow.providers.microsoft.azure.hooks.azure_cosmos import AzureCosmosDBHook\n File "/usr/local/lib/python3.7/site-packages/airflow/providers/microsoft/azure/hooks/azure_cosmos.py", line 29, in \n from azure.cosmos.errors import HTTPFailure\nImportError: cannot import name 'HTTPFailure' from 'azure.cosmos.errors' (/usr/local/lib/python3.7/site-packages/azure/cosmos/errors.py)\n'
  • airflow.providers.papermill.operators.papermill.PapermillOperator -> b'Traceback (most recent call last):\n File "", line 1, in \n File "/usr/local/lib/python3.7/site-packages/airflow/providers/papermill/operators/papermill.py", line 23, in \n from airflow.lineage.entities import File\nModuleNotFoundError: No module named 'airflow.lineage.entities'\n'
  • airflow.providers.salesforce.hooks.tableau.TableauHook -> b'Traceback (most recent call last):\n File "", line 1, in \n File "/usr/local/lib/python3.7/site-packages/airflow/providers/salesforce/hooks/tableau.py", line 20, in \n from tableauserverclient import Pager, PersonalAccessTokenAuth, Server, TableauAuth\nModuleNotFoundError: No module named 'tableauserverclient'\n'
  • airflow.providers.salesforce.operators.tableau_refresh_workbook.TableauRefreshWorkbookOperator -> b'Traceback (most recent call last):\n File "", line 1, in \n File "/usr/local/lib/python3.7/site-packages/airflow/providers/salesforce/operators/tableau_refresh_workbook.py", line 19, in \n from tableauserverclient import WorkbookItem\nModuleNotFoundError: No module named 'tableauserverclient'\n'
  • airflow.providers.salesforce.sensors.tableau_job_status.TableauJobStatusSensor -> b'Traceback (most recent call last):\n File "", line 1, in \n File "/usr/local/lib/python3.7/site-packages/airflow/providers/salesforce/sensors/tableau_job_status.py", line 20, in \n from airflow.providers.salesforce.hooks.tableau import TableauHook, TableauJobFinishCode\n File "/usr/local/lib/python3.7/site-packages/airflow/providers/salesforce/hooks/tableau.py", line 20, in \n from tableauserverclient import Pager, PersonalAccessTokenAuth, Server, TableauAuth\nModuleNotFoundError: No module named 'tableauserverclient'\n'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment