| import os | |
| import unittest | |
| from airflow.models import DagBag | |
| class TestDags(unittest.TestCase): | |
| """ | |
| Generic tests that all DAGs in the repository should be able to pass. | |
| """ |
| #!/bin/bash | |
| # Runs airflow-dags tests. | |
| # Set Nose defaults if no arguments are passed from CLI. | |
| CWD="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" | |
| NOSE_ARGS=$@ | |
| if [ -z "$NOSE_ARGS" ]; then | |
| NOSE_ARGS=" \ | |
| --with-coverage \ |
| # A redacted version of airflow.cfg | |
| [core] | |
| airflow_home = /etc/airflow | |
| dags_folder = /etc/airflow/dags | |
| base_log_folder = /var/log/airflow | |
| s3_log_folder = None | |
| remote_base_log_folder = gs://some-bucket | |
| remote_log_conn_id = gcp_di | |
| executor = LocalExecutor |
| ; Configuration for Airflow webserver and scheduler in Supervisor | |
| [program:airflow] | |
| command=/bin/airflow webserver | |
| stopsignal=QUIT | |
| stopasgroup=true | |
| user=airflow | |
| stdout_logfile=/var/log/airflow/airflow-stdout.log | |
| stderr_logfile=/var/log/airflow/airflow-stderr.log | |
| environment=HOME="/home/airflow",AIRFLOW_HOME="/etc/airflow",TMPDIR="/storage/airflow_tmp" |
| From MySQL's docs: | |
| * [Switching Masters During Failover](https://dev.mysql.com/doc/refman/5.5/en/replication-solutions-switch.html) | |
| * [mysqlfailover — Automatic replication health monitoring and failover](https://dev.mysql.com/doc/mysql-utilities/1.5/en/mysqlfailover.html) | |
| * [Replication with Global Transaction Identifiers](https://dev.mysql.com/doc/refman/5.6/en/replication-gtids.html) | |
| * [MySQL Multi-Source Replication](https://dev.mysql.com/doc/refman/5.7/en/replication-multi-source.html) | |
| * [Improving Replication Performance](https://dev.mysql.com/doc/refman/5.7/en/replication-solutions-performance.html) | |
| * [Using Replication for Backups](https://dev.mysql.com/doc/refman/5.7/en/replication-solutions-backups.html) | |
| From Facebook: |
I hereby claim:
To claim this, I am signing this object:
| <p>I am very excited to announce that Apache Incubator Samza 0.7.0 has been released. In all, 156 JIRAs were <a href="https://issues.apache.org/jira/browse/SAMZA-309?jql=project%20%3D%20SAMZA%20AND%20fixVersion%20%3D%200.7.0%20AND%20status%20in%20(Resolved%2C%20Closed)">resolved</a> in this release. Notable work done includes:</p> | |
| <ul> | |
| <li>Initial import of code into Apache. (<a href="https://issues.apache.org/jira/browse/SAMZA-1">SAMZA-1</a>)</li> | |
| <li>Upgraded to YARN 2.2 from YARN 2.05-alpha. (<a href="https://issues.apache.org/jira/browse/SAMZA-9">SAMZA-9</a>)</li> | |
| <li><a href="https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20fixVersion%20%3D%200.7.0%20AND%20status%20in%20(Resolved%2C%20Closed)%20and%20component%20%3D%20kv">Numerous</a> state management bug fixes.</li> | |
| <li>Java 7 support. (<a href="https://issues.apache.org/jira/browse/SAMZA-16">SAMZA-16</a>)</li> | |
| <li>A <a href="https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20fixVersion%20%3D%200.7.0%20AND% |
| $ yum install cmake | |
| $ mvn clean package -Dcontainer-executor.conf.dir=/etc/hadoop/ -DskipTests -Pnative |
| $ cat <path to your container-executor.cfg file> | |
| yarn.nodemanager.linux-container-executor.group=<your hadoop user group> | |
| min.user.id=1000 | |
| $ chown root:root bin/container-executor | |
| $ chmod 6050 bin/container-executor |