| Spark Executor Command: "/usr/lib/jvm/java-11-openjdk/bin/java" "-cp" "/opt/spark3/conf/:/opt/spark3/jars/*" "-Xmx1024M" "-Dspark.driver.port=35601" "org.apache.spark.executor.CoarseGrainedExecutorBackend" "--driver-url" "spark://CoarseGrainedScheduler@192.168.11.2:35601" "--executor-id" "0" "--hostname" "192.168.11.2" "--cores" "1" "--app-id" "app-20210816011025-0000" "--worker-url" "spark://Worker@192.168.11.2:39677" | |
| ======================================== | |
| Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties | |
| 21/08/16 01:10:26 INFO CoarseGrainedExecutorBackend: Started daemon with process name: 134769@laptop-archlinux | |
| 21/08/16 01:10:26 INFO SignalUtils: Registering signal handler for TERM | |
| 21/08/16 01:10:26 INFO SignalUtils: Registering signal handler for HUP | |
| 21/08/16 01:10:26 INFO SignalUtils: Registering signal handler for INT | |
| 21/08/16 01:10:26 WARN Utils: Your hostname, laptop-archlinux resolves to a loopback address: 127.0.0.1; using 192.168.11.2 instead (on interface enp0s20f |
| export SDK_HARNESS_ENDPOINT=localhost:50000 | |
| export JOB_SERVICE_ENDPOINT=localhost:8099 |
| [ywatanabe@laptop-archlinux spark3.2.1]$ bin/spark-shell | |
| 22/02/13 17:06:55 WARN Utils: Your hostname, laptop-archlinux resolves to a loopback address: 127.0.0.1; using 10.23.109.137 instead (on interface wlp0s20f3) | |
| 22/02/13 17:06:55 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address | |
| WARNING: An illegal reflective access operation has occurred | |
| WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform (file:/opt/spark3.2.1/jars/spark-unsafe_2.12-3.2.1.jar) to constructor java.nio.DirectByteBuffer(long,int) | |
| WARNING: Please consider reporting this to the maintainers of org.apache.spark.unsafe.Platform | |
| WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations | |
| WARNING: All illegal access operations will be denied in a future release | |
| Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties | |
| Setting default log level to "WARN". |
| /* | |
| * Licensed to the Apache Software Foundation (ASF) under one | |
| * or more contributor license agreements. See the NOTICE file | |
| * distributed with this work for additional information | |
| * regarding copyright ownership. The ASF licenses this file | |
| * to you under the Apache License, Version 2.0 (the | |
| * License); you may not use this file except in compliance | |
| * with the License. You may obtain a copy of the License at | |
| * | |
| * http://www.apache.org/licenses/LICENSE-2.0 |
| GET invoices/_search | |
| { | |
| "size": 0, | |
| "query": {"match_all": {} }, | |
| "aggs": { | |
| "amount_by_month": { | |
| "date_histogram": { | |
| "field": "date", | |
| "interval": "month", | |
| "order": { |
This tutorial will guide through consuming message using jdbc source connector.
For consumer I used kafka python.
| ###################### Filebeat Configuration Example ######################### | |
| # This file is an example configuration file highlighting only the most common | |
| # options. The filebeat.reference.yml file from the same directory contains all the | |
| # supported options with more comments. You can use it as a reference. | |
| # | |
| # You can find the full configuration reference here: | |
| # https://www.elastic.co/guide/en/beats/filebeat/index.html | |
| # For more available modules and options, please see the filebeat.reference.yml sample |
See go.result for full log.
(venv) y-watanabe@LAPTOP-IG41EBJ5:~/repos/github/google-api-tutorials/sdks/go$ go run examples/quickstarts/admin/report/main.go &> go.result
(venv) y-watanabe@LAPTOP-IG41EBJ5:~/repos/github/google-api-tutorials/sdks/go$ grep Page go.result
2022/04/08 18:35:22 Page: 1, Received 1000 items
| # ILM | |
| # https://www.elastic.co/guide/en/elasticsearch/reference/7.17/ilm-rollover.html#ilm-rollover-primar-shardsize-ex | |
| # rollover in 5 min | |
| PUT _ilm/policy/my_policy | |
| { | |
| "policy": { | |
| "phases": { | |
| "hot": { | |
| "actions": { | |
| "rollover" : { |
TBC