This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
https://openmetadata.slack.com/archives/C02B6955S4S/p1709558609380039 | |
Hey, guys. | |
Question about general description metadata ingestion. | |
So, let's use as an example the PostgreSQL source. We may want to add/change descriptions of tables/columns on PostgreSQL, | |
which will reflect on OMD after enabling the metadata ingestion. | |
If we change the description later on OMD, | |
it seems that new metadata ingestions won't overwrite the changed description to the former/PostgreSQL value. | |
If we update descriptions on PostgreSQL that were changed on OMD, it will overwrite the changed on in OMD? | |
I've read that we should use OMD as the Source of Truth, | |
so it's not advisable to actually interact with the source to update the metadata which will further reflect on OMD via metadata ingestion? |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import traceback | |
from typing import TYPE_CHECKING, Any, List, Optional | |
from airflow.exceptions import AirflowException | |
from airflow.models import BaseOperator | |
if TYPE_CHECKING: | |
from airflow.utils.context import Context | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from typing import ( # noqa: F401 | |
Any, Dict, Iterator, Union, | |
) | |
from pyhocon import ConfigFactory, ConfigTree # noqa: F401 | |
from databuilder.extractor.base_postgres_metadata_extractor import BasePostgresMetadataExtractor | |
class PostgresViewMetadataExtractor(BasePostgresMetadataExtractor): |