| #http://docs.confluent.io/3.0.0/connect/connect-hdfs/docs/configuration_options.html | |
| flush.size=50 | |
| hdfs.url=hdfs:/// | |
| hadoop.conf.dir= | |
| hive.conf.dir=hive.home | |
| hive.database=kafka | |
| hive.home=hdfs:///user/hive/warehouse | |
| hive.metastore.uris= |
This is an example docker-compose.yml for the Confluent Platform.
In order to properly use this docker-compose make sure that your containers are resolvable by their container name. Take a look at the hosts file. Each container must be resolvable from the host machine if you are connecting from the host to the docker environment.
Download the AWS Java SDK Copy aws-java-sdk-1.10.11.jar to $CONFLUENT_HOME/share/java/kafka-connect-hdfs/ Download hadoop-aws-2.6.0.jar to $CONFLUENT_HOME/share/java/kafka-connect-hdfs/ Download
| [Unit] | |
| Description = Apache Kafka by Confluent | |
| Wants = basic.target | |
| After = basic.target network.target | |
| [Service] | |
| EnvironmentFile = /etc/sysconfig/kafka | |
| ExecStart = /usr/bin/kafka-server-start /etc/kafka/server.properties | |
| LimitNOFILE = 131072 | |
| KillMode = process |