| """ | |
| Exports metadata information for all columns in a BigQuery table to a CSV file. | |
| This function retrieves the schema of the specified BigQuery table and computes the following | |
| details for each column: | |
| - Column name | |
| - Data type | |
| - Count of NOT NULL values | |
| - Count of NULL values | |
| - Sample of up to 10 distinct values |
| """ | |
| Airflow Hook for Notion API: https://developers.notion.com/reference/intro | |
| author: Aman Ranjan Verma(https://www.linkedin.com/in/ar-verma/) | |
| """ | |
| import json | |
| from airflow.providers.http.hooks.http import HttpHook | |
| from airflow.exceptions import AirflowException | |
| import logging |
| """ | |
| Airflow Hook for Amplitude API: https://www.docs.developers.amplitude.com/analytics/apis/taxonomy-api/ | |
| author: Aman Ranjan Verma(https://www.linkedin.com/in/ar-verma/) | |
| """ | |
| from airflow.providers.http.hooks.http import HttpHook | |
| from airflow.exceptions import AirflowException | |
| from urllib.parse import quote | |
| AMPLITUDE_EVENT_ENDPOINT = "amplitude.com/api/2/taxonomy/event" |
| Customer Schema Names | Constraints | |
|---|---|---|
| customer_id | PRIMARY KEY | |
| customer_name | NOT NULL | |
| mob_num | CHECK | |
| email_id | CHECK | |
| active_from | DEFAULT CURRENT_DATE_TIME | |
| active_to |
| """ | |
| Suppose you have a big text log file and there are three columns: | |
| ------------------------------------ | |
| Name | City | Timestamp | |
| ------------------------------------- | |
| Jamie city1 ts4 | |
| Dana city1 ts3 | |
| Jamie city2 ts1 | |
| Jamie city3 ts3 |
Question Description
During the Covid outbreak, a system was implemented to allow hospitals and test centers to report the daily number of confirmed cases to a central system.Your team is responsible to answer certain questions based on the stream of events.
You may only use the following given operators of a hypothetical distributed stream processing framework:
As part of a job submission to a server, that is responsible for orchestrating the creation of tables for consumption by reports, a piece of metadata is submitted that declares the dependencies of the various tables for the given job.It is the responsibility of the server to make sure that a correct build order is chosen and executed.
An example of such a dependency declaration is:
{
"TableA": ["TableB", "TableC"],
| class Node: | |
| def __init__(self, data): | |
| self.data = data | |
| self.next = None | |
| def create_linked_list(a): | |
| head = Node(a[0]) | |
| start = head | |
| for element in a[1::]: |