Skip to content

Instantly share code, notes, and snippets.

@reizist
Created October 19, 2017 04:19
Show Gist options
  • Save reizist/d943cd54dee635213ad7d791aedf31cc to your computer and use it in GitHub Desktop.
Save reizist/d943cd54dee635213ad7d791aedf31cc to your computer and use it in GitHub Desktop.
presto launching log on emr
INFO main io.airlift.log.Logging Disabling stderr output
INFO main Bootstrap PROPERTY DEFAULT RUNTIME DESCRIPTION
INFO main Bootstrap discovery-server.enabled false true
INFO main Bootstrap event.max-output-stage-size 16MB 16MB
INFO main Bootstrap query.client.timeout 5.00m 5.00m
INFO main Bootstrap query.initial-hash-partitions 100 100
INFO main Bootstrap query.max-concurrent-queries 1000 1000
INFO main Bootstrap query.max-history 100 40
INFO main Bootstrap query.max-length 1000000 1000000
INFO main Bootstrap query.max-queued-queries 5000 5000
INFO main Bootstrap query.min-expire-age 15.00m 30.00m
INFO main Bootstrap query.min-schedule-split-batch-size 100 100
INFO main Bootstrap query.execution-policy all-at-once all-at-once
INFO main Bootstrap query.manager-executor-pool-size 5 5
INFO main Bootstrap query.max-cpu-time 1000000000.00d 1000000000.00d
INFO main Bootstrap query.max-run-time 100.00d 100.00d
INFO main Bootstrap query.queue-config-file null /home/hadoop/emr_configs/queue.json
INFO main Bootstrap query.remote-task.max-callback-threads 1000 1000
INFO main Bootstrap query.remote-task.max-error-duration 5.00m 5.00m
INFO main Bootstrap query.remote-task.min-error-duration 2.00m 2.00m
INFO main Bootstrap query.schedule-split-batch-size 1000 1000
INFO main Bootstrap task.client.timeout 2.00m 2.00m
INFO main Bootstrap task.http-response-threads 100 100
INFO main Bootstrap task.http-timeout-threads 3 3
INFO main Bootstrap task.info.max-age 15.00m 15.00m
INFO main Bootstrap task.info-update-interval 3.00s 3.00s Interval between updating task data
INFO main Bootstrap task.initial-splits-per-node 8 8
INFO main Bootstrap task.max-index-memory 64MB 64MB
INFO main Bootstrap driver.max-page-partitioning-buffer-size 32MB 32MB
INFO main Bootstrap task.max-partial-aggregation-memory 16MB 16MB
INFO main Bootstrap task.max-worker-threads 8 8
INFO main Bootstrap task.min-drivers 16 16
INFO main Bootstrap task.share-index-loading false false
INFO main Bootstrap sink.max-buffer-size 32MB 1GB
INFO main Bootstrap task.split-concurrency-adjustment-interval 100.00ms 100.00ms
INFO main Bootstrap task.status-refresh-max-wait 1.00s 1.00s
INFO main Bootstrap task.concurrency 16 16 Default number of local parallel jobs per worker
INFO main Bootstrap task.cpu-timer-enabled true true
INFO main Bootstrap task.task-notification-threads 5 5 Number of threads used for internal task event notifications
INFO main Bootstrap task.verbose-stats false false
INFO main Bootstrap task.writer-count 1 1 Number of writers per task
INFO main Bootstrap node-scheduler.include-coordinator true false
INFO main Bootstrap node-scheduler.max-pending-splits-per-task 10 10
INFO main Bootstrap node-scheduler.max-splits-per-node 100 100
INFO main Bootstrap node-scheduler.min-candidates 10 10
INFO main Bootstrap node-scheduler.network-topology legacy legacy
INFO main Bootstrap failure-detector.enabled true true
INFO main Bootstrap failure-detector.expiration-grace-interval 10.00m 10.00m How long to wait before 'forgetting' a service after it disappears from discovery
INFO main Bootstrap failure-detector.threshold 0.1 0.1
INFO main Bootstrap failure-detector.heartbeat-interval 500.00ms 500.00ms
INFO main Bootstrap failure-detector.warmup-interval 5.00s 5.00s How long to wait after transitioning to success before considering a service alive
INFO main Bootstrap query.low-memory-killer.enabled false false Enable low memory killer
INFO main Bootstrap query.low-memory-killer.delay 5.00m 5.00m Delay between cluster running low on memory and invoking killer
INFO main Bootstrap query.max-memory 20GB 100GB
INFO main Bootstrap query.max-memory-per-node 2567418675.20B 12863427052B
INFO main Bootstrap resources.reserved-system-memory 10269674700.80B 10269674700.80B
INFO main Bootstrap catalog.config-dir etc/catalog /etc/presto/conf/catalog
INFO main Bootstrap catalog.disabled-catalogs null null
INFO main Bootstrap exchange.client-threads 25 25
INFO main Bootstrap exchange.concurrent-request-multiplier 3 3
INFO main Bootstrap exchange.max-buffer-size 32MB 32MB
INFO main Bootstrap exchange.max-error-duration 5.00m 5.00m
INFO main Bootstrap exchange.max-response-size 16MB 16MB
INFO main Bootstrap exchange.min-error-duration 1.00m 1.00m
INFO main Bootstrap code-cache-check-interval 20.00s 20.00s How often to check if the code cache need to be collected
INFO main Bootstrap code-cache-collection-threshold 40 40 Maximum code cache usage (percentage) before attempting to force a collection
INFO main Bootstrap internal-communication.https.required false false
INFO main Bootstrap internal-communication.https.keystore.key null null
INFO main Bootstrap internal-communication.https.keystore.path null null
INFO main Bootstrap internal-communication.authentication.ldap.password null null
INFO main Bootstrap internal-communication.authentication.ldap.user null null
INFO main Bootstrap plugin.dir plugin /usr/lib/presto/plugin
INFO main Bootstrap maven.repo.local /var/lib/presto/.m2/repository /var/lib/presto/.m2/repository
INFO main Bootstrap maven.repo.remote [http://repo1.maven.org/maven2/] [http://repo1.maven.org/maven2/]
INFO main Bootstrap plugin.config-dir etc etc
INFO main Bootstrap plugin.bundles null null
INFO main Bootstrap coordinator true true
INFO main Bootstrap datasources null null
INFO main Bootstrap shutdown.grace-period 2.00m 2.00m
INFO main Bootstrap http.include-exception-in-response true true
INFO main Bootstrap presto.version null null
INFO main Bootstrap http-server.authentication.type NONE NONE Authentication type (supported types: NONE, KERBEROS, LDAP)
INFO main Bootstrap colocated-joins-enabled false false Experimental: Use a colocated join when possible
INFO main Bootstrap optimizer.dictionary-aggregation false false
INFO main Bootstrap distributed-index-joins-enabled false false
INFO main Bootstrap distributed-joins-enabled true true
INFO main Bootstrap exchange.compression-enabled false false
INFO main Bootstrap experimental.iterative-optimizer-enabled false false
INFO main Bootstrap experimental.iterative-optimizer-timeout 3.00m 3.00m
INFO main Bootstrap reorder-joins false false Experimental: Reorder joins to optimize plan
INFO main Bootstrap deprecated.legacy-array-agg false false
INFO main Bootstrap deprecated.legacy-map-subscript false false
INFO main Bootstrap deprecated.legacy-order-by false false
INFO main Bootstrap experimental.operator-memory-limit-before-spill 4MB 4MB
INFO main Bootstrap optimizer.optimize-hash-generation true true
INFO main Bootstrap optimizer.optimize-metadata-queries false false
INFO main Bootstrap optimizer.optimize-mixed-distinct-aggregations false false
INFO main Bootstrap optimizer.optimize-single-distinct true true
INFO main Bootstrap optimizer.processing-optimization disabled disabled
INFO main Bootstrap optimizer.push-table-write-through-union true true
INFO main Bootstrap re2j.dfa-retries 5 5
INFO main Bootstrap re2j.dfa-states-limit 2147483647 2147483647
INFO main Bootstrap redistribute-writes true true
INFO main Bootstrap regex-library JONI JONI
INFO main Bootstrap optimizer.reorder-windows false false
INFO main Bootstrap experimental.resource-groups-enabled false false
INFO main Bootstrap experimental.spill-enabled false false
INFO main Bootstrap experimental.spiller-spill-path /tmp/presto/spills /tmp/presto/spills
INFO main Bootstrap experimental.spiller-threads 4 4
INFO main Bootstrap compiler.interpreter-enabled false false
INFO main Bootstrap transaction.idle-check-interval 1.00m 1.00m Time interval between idle transactions checks
INFO main Bootstrap transaction.idle-timeout 5.00m 5.00m Amount of time before an inactive transaction is considered expired
INFO main Bootstrap transaction.max-finishing-concurrency 1 1 Maximum parallelism for committing or aborting a transaction
INFO main Bootstrap discovery.uri null http://ip-172-31-29-54.ap-northeast-1.compute.internal:8889 Discovery service base URI
INFO main Bootstrap service-inventory.uri null null Service inventory base URI
INFO main Bootstrap service-inventory.update-interval 10.00s 10.00s Service inventory update interval
INFO main Bootstrap discovery.collector.pool general general
INFO main Bootstrap discovery.presto.pool general general
INFO main Bootstrap discovery.max-age 30.00s 30.00s
INFO main Bootstrap discovery.store-cache-ttl 1.00s 1.00s
INFO main Bootstrap dynamic.store.gc-interval 1.00h 1.00h
INFO main Bootstrap dynamic.store.remote.max-batch-size 1000 1000
INFO main Bootstrap dynamic.store.remote.queue-size 1000 1000
INFO main Bootstrap dynamic.store.remote.update-interval 5.00s 5.00s
INFO main Bootstrap dynamic.store.remote.replication-interval 1.00m 1.00m
INFO main Bootstrap dynamic.store.tombstone-max-age 1.00d 1.00d
INFO main Bootstrap failure-detector.http-client.authentication.enabled false false Enable client authentication
INFO main Bootstrap failure-detector.http-client.connect-timeout 1.00s 1.00s
INFO main Bootstrap failure-detector.http-client.http2.enabled false false Enable the HTTP/2 transport
INFO main Bootstrap failure-detector.http-client.idle-timeout 1.00m 1.00m
INFO main Bootstrap failure-detector.http-client.keep-alive-interval null null
INFO main Bootstrap failure-detector.http-client.authentication.krb5.principal null null Set kerberos client principal
INFO main Bootstrap failure-detector.http-client.authentication.krb5.remote-service-name null null Set kerberos service principal name
INFO main Bootstrap failure-detector.http-client.key-store-password null null
INFO main Bootstrap failure-detector.http-client.key-store-path null null
INFO main Bootstrap failure-detector.http-client.max-connections 200 200
INFO main Bootstrap failure-detector.http-client.max-connections-per-server 20 20
INFO main Bootstrap failure-detector.http-client.max-content-length 16MB 16MB
INFO main Bootstrap failure-detector.http-client.max-requests-queued-per-destination 1024 1024
INFO main Bootstrap failure-detector.http-client.request-timeout 5.00m 5.00m
INFO main Bootstrap failure-detector.http-client.secure-random-algorithm null null
INFO main Bootstrap failure-detector.http-client.socks-proxy null null
INFO main Bootstrap failure-detector.http-client.trust-store-password null null
INFO main Bootstrap failure-detector.http-client.trust-store-path null null
INFO main Bootstrap memoryManager.http-client.authentication.enabled false false Enable client authentication
INFO main Bootstrap memoryManager.http-client.connect-timeout 1.00s 1.00s
INFO main Bootstrap memoryManager.http-client.http2.enabled false false Enable the HTTP/2 transport
INFO main Bootstrap memoryManager.http-client.idle-timeout 30.00s 30.00s
INFO main Bootstrap memoryManager.http-client.keep-alive-interval null null
INFO main Bootstrap memoryManager.http-client.authentication.krb5.principal null null Set kerberos client principal
INFO main Bootstrap memoryManager.http-client.authentication.krb5.remote-service-name null null Set kerberos service principal name
INFO main Bootstrap memoryManager.http-client.key-store-password null null
INFO main Bootstrap memoryManager.http-client.key-store-path null null
INFO main Bootstrap memoryManager.http-client.max-connections 200 200
INFO main Bootstrap memoryManager.http-client.max-connections-per-server 20 20
INFO main Bootstrap memoryManager.http-client.max-content-length 16MB 16MB
INFO main Bootstrap memoryManager.http-client.max-requests-queued-per-destination 1024 1024
INFO main Bootstrap memoryManager.http-client.request-timeout 10.00s 10.00s
INFO main Bootstrap memoryManager.http-client.secure-random-algorithm null null
INFO main Bootstrap memoryManager.http-client.socks-proxy null null
INFO main Bootstrap memoryManager.http-client.trust-store-password null null
INFO main Bootstrap memoryManager.http-client.trust-store-path null null
INFO main Bootstrap node-manager.http-client.authentication.enabled false false Enable client authentication
INFO main Bootstrap node-manager.http-client.connect-timeout 1.00s 1.00s
INFO main Bootstrap node-manager.http-client.http2.enabled false false Enable the HTTP/2 transport
INFO main Bootstrap node-manager.http-client.idle-timeout 30.00s 30.00s
INFO main Bootstrap node-manager.http-client.keep-alive-interval null null
INFO main Bootstrap node-manager.http-client.authentication.krb5.principal null null Set kerberos client principal
INFO main Bootstrap node-manager.http-client.authentication.krb5.remote-service-name null null Set kerberos service principal name
INFO main Bootstrap node-manager.http-client.key-store-password null null
INFO main Bootstrap node-manager.http-client.key-store-path null null
INFO main Bootstrap node-manager.http-client.max-connections 200 200
INFO main Bootstrap node-manager.http-client.max-connections-per-server 20 20
INFO main Bootstrap node-manager.http-client.max-content-length 16MB 16MB
INFO main Bootstrap node-manager.http-client.max-requests-queued-per-destination 1024 1024
INFO main Bootstrap node-manager.http-client.request-timeout 10.00s 10.00s
INFO main Bootstrap node-manager.http-client.secure-random-algorithm null null
INFO main Bootstrap node-manager.http-client.socks-proxy null null
INFO main Bootstrap node-manager.http-client.trust-store-password null null
INFO main Bootstrap node-manager.http-client.trust-store-path null null
INFO main Bootstrap exchange.http-client.authentication.enabled false false Enable client authentication
INFO main Bootstrap exchange.http-client.connect-timeout 1.00s 1.00s
INFO main Bootstrap exchange.http-client.http2.enabled false false Enable the HTTP/2 transport
INFO main Bootstrap exchange.http-client.idle-timeout 30.00s 30.00s
INFO main Bootstrap exchange.http-client.keep-alive-interval null null
INFO main Bootstrap exchange.http-client.authentication.krb5.principal null null Set kerberos client principal
INFO main Bootstrap exchange.http-client.authentication.krb5.remote-service-name null null Set kerberos service principal name
INFO main Bootstrap exchange.http-client.key-store-password null null
INFO main Bootstrap exchange.http-client.key-store-path null null
INFO main Bootstrap exchange.http-client.max-connections 200 200
INFO main Bootstrap exchange.http-client.max-connections-per-server 250 250
INFO main Bootstrap exchange.http-client.max-content-length 32MB 32MB
INFO main Bootstrap exchange.http-client.max-requests-queued-per-destination 1024 1024
INFO main Bootstrap exchange.http-client.request-timeout 10.00s 10.00s
INFO main Bootstrap exchange.http-client.secure-random-algorithm null null
INFO main Bootstrap exchange.http-client.socks-proxy null null
INFO main Bootstrap exchange.http-client.trust-store-password null null
INFO main Bootstrap exchange.http-client.trust-store-path null null
INFO main Bootstrap scheduler.http-client.authentication.enabled false false Enable client authentication
INFO main Bootstrap scheduler.http-client.connect-timeout 1.00s 1.00s
INFO main Bootstrap scheduler.http-client.http2.enabled false false Enable the HTTP/2 transport
INFO main Bootstrap scheduler.http-client.idle-timeout 30.00s 30.00s
INFO main Bootstrap scheduler.http-client.keep-alive-interval null null
INFO main Bootstrap scheduler.http-client.authentication.krb5.principal null null Set kerberos client principal
INFO main Bootstrap scheduler.http-client.authentication.krb5.remote-service-name null null Set kerberos service principal name
INFO main Bootstrap scheduler.http-client.key-store-password null null
INFO main Bootstrap scheduler.http-client.key-store-path null null
INFO main Bootstrap scheduler.http-client.max-connections 200 200
INFO main Bootstrap scheduler.http-client.max-connections-per-server 250 250
INFO main Bootstrap scheduler.http-client.max-content-length 16MB 16MB
INFO main Bootstrap scheduler.http-client.max-requests-queued-per-destination 1024 1024
INFO main Bootstrap scheduler.http-client.request-timeout 10.00s 10.00s
INFO main Bootstrap scheduler.http-client.secure-random-algorithm null null
INFO main Bootstrap scheduler.http-client.socks-proxy null null
INFO main Bootstrap scheduler.http-client.trust-store-password null null
INFO main Bootstrap scheduler.http-client.trust-store-path null null
INFO main Bootstrap discovery.http-client.authentication.enabled false false Enable client authentication
INFO main Bootstrap discovery.http-client.connect-timeout 1.00s 1.00s
INFO main Bootstrap discovery.http-client.http2.enabled false false Enable the HTTP/2 transport
INFO main Bootstrap discovery.http-client.idle-timeout 1.00m 1.00m
INFO main Bootstrap discovery.http-client.keep-alive-interval null null
INFO main Bootstrap discovery.http-client.authentication.krb5.principal null null Set kerberos client principal
INFO main Bootstrap discovery.http-client.authentication.krb5.remote-service-name null null Set kerberos service principal name
INFO main Bootstrap discovery.http-client.key-store-password null null
INFO main Bootstrap discovery.http-client.key-store-path null null
INFO main Bootstrap discovery.http-client.max-connections 200 200
INFO main Bootstrap discovery.http-client.max-connections-per-server 20 20
INFO main Bootstrap discovery.http-client.max-content-length 16MB 16MB
INFO main Bootstrap discovery.http-client.max-requests-queued-per-destination 1024 1024
INFO main Bootstrap discovery.http-client.request-timeout 5.00m 5.00m
INFO main Bootstrap discovery.http-client.secure-random-algorithm null null
INFO main Bootstrap discovery.http-client.socks-proxy null null
INFO main Bootstrap discovery.http-client.trust-store-password null null
INFO main Bootstrap discovery.http-client.trust-store-path null null
INFO main Bootstrap dynamic.http-client.authentication.enabled false false Enable client authentication
INFO main Bootstrap dynamic.http-client.connect-timeout 1.00s 1.00s
INFO main Bootstrap dynamic.http-client.http2.enabled false false Enable the HTTP/2 transport
INFO main Bootstrap dynamic.http-client.idle-timeout 1.00m 1.00m
INFO main Bootstrap dynamic.http-client.keep-alive-interval null null
INFO main Bootstrap dynamic.http-client.authentication.krb5.principal null null Set kerberos client principal
INFO main Bootstrap dynamic.http-client.authentication.krb5.remote-service-name null null Set kerberos service principal name
INFO main Bootstrap dynamic.http-client.key-store-password null null
INFO main Bootstrap dynamic.http-client.key-store-path null null
INFO main Bootstrap dynamic.http-client.max-connections 200 200
INFO main Bootstrap dynamic.http-client.max-connections-per-server 20 20
INFO main Bootstrap dynamic.http-client.max-content-length 16MB 16MB
INFO main Bootstrap dynamic.http-client.max-requests-queued-per-destination 1024 1024
INFO main Bootstrap dynamic.http-client.request-timeout 5.00m 5.00m
INFO main Bootstrap dynamic.http-client.secure-random-algorithm null null
INFO main Bootstrap dynamic.http-client.socks-proxy null null
INFO main Bootstrap dynamic.http-client.trust-store-password null null
INFO main Bootstrap dynamic.http-client.trust-store-path null null
INFO main Bootstrap event.http-client.authentication.enabled false false Enable client authentication
INFO main Bootstrap event.http-client.connect-timeout 1.00s 1.00s
INFO main Bootstrap event.http-client.http2.enabled false false Enable the HTTP/2 transport
INFO main Bootstrap event.http-client.idle-timeout 1.00m 1.00m
INFO main Bootstrap event.http-client.keep-alive-interval null null
INFO main Bootstrap event.http-client.authentication.krb5.principal null null Set kerberos client principal
INFO main Bootstrap event.http-client.authentication.krb5.remote-service-name null null Set kerberos service principal name
INFO main Bootstrap event.http-client.key-store-password null null
INFO main Bootstrap event.http-client.key-store-path null null
INFO main Bootstrap event.http-client.max-connections 200 200
INFO main Bootstrap event.http-client.max-connections-per-server 20 20
INFO main Bootstrap event.http-client.max-content-length 16MB 16MB
INFO main Bootstrap event.http-client.max-requests-queued-per-destination 1024 1024
INFO main Bootstrap event.http-client.request-timeout 5.00m 5.00m
INFO main Bootstrap event.http-client.secure-random-algorithm null null
INFO main Bootstrap event.http-client.socks-proxy null null
INFO main Bootstrap event.http-client.trust-store-password null null
INFO main Bootstrap event.http-client.trust-store-path null null
INFO main Bootstrap failure-detector.http-client.max-threads 200 200
INFO main Bootstrap failure-detector.http-client.min-threads 8 8
INFO main Bootstrap http-client.max-threads 200 200
INFO main Bootstrap http-client.min-threads 8 8
INFO main Bootstrap http.authentication.krb5.config null null Set kerberos configuration path
INFO main Bootstrap http.authentication.krb5.credential-cache null null Set kerberos credential cache path
INFO main Bootstrap http.authentication.krb5.keytab null null Set kerberos key table path
INFO main Bootstrap http.authentication.krb5.use-canonical-hostname true true Canonicalize service hostname using the DNS reverse lookup
INFO main Bootstrap http-server.admin.enabled true true
INFO main Bootstrap http-server.admin.threads.max 200 200
INFO main Bootstrap http-server.admin.threads.min 2 2
INFO main Bootstrap http-server.admin.port 0 0
INFO main Bootstrap http-server.http2.max-concurrent-streams 16384 16384 Maximum concurrent streams per connection for HTTP/2
INFO main Bootstrap http-server.accept-queue-size 8000 8000
INFO main Bootstrap http-server.http.acceptor-threads null null
INFO main Bootstrap http-server.http.enabled true true
INFO main Bootstrap http-server.http.port 8080 8889
INFO main Bootstrap http-server.http.selector-threads null null
INFO main Bootstrap http-server.https.acceptor-threads null null
INFO main Bootstrap http-server.https.enabled false false
INFO main Bootstrap http-server.https.excluded-cipher [] []
INFO main Bootstrap http-server.https.included-cipher [] []
INFO main Bootstrap http-server.https.port 8443 8443
INFO main Bootstrap http-server.https.selector-threads null null
INFO main Bootstrap http-server.https.keymanager.password null null
INFO main Bootstrap http-server.https.keystore.key null null
INFO main Bootstrap http-server.https.keystore.path null null
INFO main Bootstrap http-server.log.enabled true true
INFO main Bootstrap http-server.log.max-history 15 5
INFO main Bootstrap http-server.log.max-size 9223372036854775807B 67108864B
INFO main Bootstrap http-server.log.path var/log/http-request.log /var/log/presto/http-request.log
INFO main Bootstrap http-server.max-request-header-size null null
INFO main Bootstrap http-server.threads.max 200 500
INFO main Bootstrap http-server.threads.min 2 2
INFO main Bootstrap http-server.net.max-idle-time 200.00s 200.00s
INFO main Bootstrap http-server.https.secure-random-algorithm null null
INFO main Bootstrap http-server.show-stack-trace true true Show the stack trace when generating an error response
INFO main Bootstrap http-server.threads.max-idle-time 1.00m 1.00m
INFO main Bootstrap http-server.auth.users-file null null
INFO main Bootstrap jmx.rmiregistry.port null null
INFO main Bootstrap jmx.rmiserver.port null null
INFO main Bootstrap node.binary-spec null null
INFO main Bootstrap node.config-spec null null
INFO main Bootstrap node.environment null production
INFO main Bootstrap node.internal-address-source IP IP
INFO main Bootstrap node.location null null
INFO main Bootstrap node.bind-ip null null
INFO main Bootstrap node.external-address null null
INFO main Bootstrap node.id null i-06ae3b9bda98776aa
INFO main Bootstrap node.internal-address null null
INFO main Bootstrap node.pool general general
INFO main Bootstrap
WARN main io.airlift.jmx.JmxAgent Cannot determine if JMX agent is already running (not an Oracle JVM?). Will try to start it manually.
INFO main io.airlift.jmx.JmxAgent JMX agent started and listening on ip-172-31-29-54:44168
ERROR Discovery-0 io.airlift.discovery.client.CachingServiceSelector Cannot connect to discovery server for refresh (presto/general): Lookup of presto failed for http://ip-172-31-29-54.ap-northeast-1.compute.internal:8889/v1/service/presto/general
WARN http-client-shared-17 com.facebook.presto.metadata.RemoteNodeState Error fetching node state from http://172.31.29.54:8889/v1/info/state: Server refused connection: http://172.31.29.54:8889/v1/info/state
ERROR Discovery-3 io.airlift.discovery.client.CachingServiceSelector Cannot connect to discovery server for refresh (collector/general): Lookup of collector failed for http://ip-172-31-29-54.ap-northeast-1.compute.internal:8889/v1/service/collector/general
INFO main org.eclipse.jetty.server.Server jetty-9.3.13.M0
WARN main org.eclipse.jetty.server.Server THIS IS NOT A STABLE RELEASE! DO NOT USE IN PRODUCTION!
WARN main org.eclipse.jetty.server.Server Download a stable release from http://download.eclipse.org/jetty/
WARN main org.eclipse.jetty.server.handler.AbstractHandler No Server set for org.eclipse.jetty.server.handler.ErrorHandler@5099c59b
WARN query-management-0 com.facebook.presto.memory.RemoteNodeMemory Error fetching memory info from http://172.31.29.54:8889/v1/memory: Server refused connection: http://172.31.29.54:8889/v1/memory
INFO main org.eclipse.jetty.server.handler.ContextHandler Started o.e.j.s.ServletContextHandler@13ed066e{/,null,AVAILABLE,@http}
INFO main org.eclipse.jetty.server.AbstractConnector Started http@5b202ff{HTTP/1.1,[http/1.1, h2c, h2c-17, h2c-16, h2c-15, h2c-14]}{0.0.0.0:8889}
INFO main org.eclipse.jetty.server.Server Started @6951ms
INFO main io.airlift.bootstrap.LifeCycleManager Life cycle starting...
INFO main io.airlift.bootstrap.LifeCycleManager Life cycle startup complete. System ready.
INFO main com.facebook.presto.server.PluginManager -- Loading plugin /usr/lib/presto/plugin/accumulo --
INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.accumulo.AccumuloPlugin
INFO main com.facebook.presto.server.PluginManager Registering connector accumulo
INFO main com.facebook.presto.server.PluginManager Registering functions from com.facebook.presto.accumulo.udf.AccumuloStringFunctions
INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /usr/lib/presto/plugin/accumulo --
INFO main com.facebook.presto.server.PluginManager -- Loading plugin /usr/lib/presto/plugin/atop --
INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.atop.AtopPlugin
INFO main com.facebook.presto.server.PluginManager Registering connector atop
INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /usr/lib/presto/plugin/atop --
INFO main com.facebook.presto.server.PluginManager -- Loading plugin /usr/lib/presto/plugin/blackhole --
INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.plugin.blackhole.BlackHolePlugin
INFO main com.facebook.presto.server.PluginManager Registering connector blackhole
INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /usr/lib/presto/plugin/blackhole --
INFO main com.facebook.presto.server.PluginManager -- Loading plugin /usr/lib/presto/plugin/cassandra --
INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.cassandra.CassandraPlugin
INFO main com.facebook.presto.server.PluginManager Registering connector cassandra
INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /usr/lib/presto/plugin/cassandra --
INFO main com.facebook.presto.server.PluginManager -- Loading plugin /usr/lib/presto/plugin/example-http --
INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.example.ExamplePlugin
INFO main com.facebook.presto.server.PluginManager Registering connector example-http
INFO Discovery-4 io.airlift.discovery.client.CachingServiceSelector Discovery server connect succeeded for refresh (presto/general)
INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /usr/lib/presto/plugin/example-http --
INFO main com.facebook.presto.server.PluginManager -- Loading plugin /usr/lib/presto/plugin/hive-hadoop2 --
INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.hive.HiveHadoop2Plugin
INFO main com.facebook.presto.server.PluginManager Registering connector hive-hadoop2
INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /usr/lib/presto/plugin/hive-hadoop2 --
INFO main com.facebook.presto.server.PluginManager -- Loading plugin /usr/lib/presto/plugin/jmx --
INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.connector.jmx.JmxPlugin
INFO main com.facebook.presto.server.PluginManager Registering connector jmx
INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /usr/lib/presto/plugin/jmx --
INFO main com.facebook.presto.server.PluginManager -- Loading plugin /usr/lib/presto/plugin/kafka --
INFO Discovery-2 io.airlift.discovery.client.CachingServiceSelector Discovery server connect succeeded for refresh (collector/general)
INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.kafka.KafkaPlugin
INFO main com.facebook.presto.server.PluginManager Registering connector kafka
INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /usr/lib/presto/plugin/kafka --
INFO main com.facebook.presto.server.PluginManager -- Loading plugin /usr/lib/presto/plugin/localfile --
INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.localfile.LocalFilePlugin
INFO main com.facebook.presto.server.PluginManager Registering connector localfile
INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /usr/lib/presto/plugin/localfile --
INFO main com.facebook.presto.server.PluginManager -- Loading plugin /usr/lib/presto/plugin/memory --
INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.plugin.memory.MemoryPlugin
INFO main com.facebook.presto.server.PluginManager Registering connector memory
INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /usr/lib/presto/plugin/memory --
INFO main com.facebook.presto.server.PluginManager -- Loading plugin /usr/lib/presto/plugin/ml --
INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.ml.MLPlugin
INFO main com.facebook.presto.server.PluginManager Registering type Model
INFO main com.facebook.presto.server.PluginManager Registering type Regressor
INFO main com.facebook.presto.server.PluginManager Registering parametric type Classifier
INFO main com.facebook.presto.server.PluginManager Registering functions from com.facebook.presto.ml.LearnClassifierAggregation
INFO main com.facebook.presto.server.PluginManager Registering functions from com.facebook.presto.ml.LearnVarcharClassifierAggregation
INFO main com.facebook.presto.server.PluginManager Registering functions from com.facebook.presto.ml.LearnRegressorAggregation
INFO main com.facebook.presto.server.PluginManager Registering functions from com.facebook.presto.ml.LearnLibSvmClassifierAggregation
INFO main com.facebook.presto.server.PluginManager Registering functions from com.facebook.presto.ml.LearnLibSvmVarcharClassifierAggregation
INFO main com.facebook.presto.server.PluginManager Registering functions from com.facebook.presto.ml.LearnLibSvmRegressorAggregation
INFO main com.facebook.presto.server.PluginManager Registering functions from com.facebook.presto.ml.EvaluateClassifierPredictionsAggregation
INFO main com.facebook.presto.server.PluginManager Registering functions from com.facebook.presto.ml.MLFunctions
INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /usr/lib/presto/plugin/ml --
INFO main com.facebook.presto.server.PluginManager -- Loading plugin /usr/lib/presto/plugin/mongodb --
INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.mongodb.MongoPlugin
INFO main com.facebook.presto.server.PluginManager Registering type ObjectId
INFO main com.facebook.presto.server.PluginManager Registering connector mongodb
INFO main com.facebook.presto.server.PluginManager Registering functions from com.facebook.presto.mongodb.ObjectIdFunctions
INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /usr/lib/presto/plugin/mongodb --
INFO main com.facebook.presto.server.PluginManager -- Loading plugin /usr/lib/presto/plugin/mysql --
INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.plugin.mysql.MySqlPlugin
INFO main com.facebook.presto.server.PluginManager Registering connector mysql
INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /usr/lib/presto/plugin/mysql --
INFO main com.facebook.presto.server.PluginManager -- Loading plugin /usr/lib/presto/plugin/postgresql --
INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.plugin.postgresql.PostgreSqlPlugin
INFO main com.facebook.presto.server.PluginManager Registering connector postgresql
INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /usr/lib/presto/plugin/postgresql --
INFO main com.facebook.presto.server.PluginManager -- Loading plugin /usr/lib/presto/plugin/presto-fluentd --
INFO main com.facebook.presto.server.PluginManager Installing fluentd.FluentdPlugin
INFO main com.facebook.presto.server.PluginManager Registering event listener presto-fluentd
INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /usr/lib/presto/plugin/presto-fluentd --
INFO main com.facebook.presto.server.PluginManager -- Loading plugin /usr/lib/presto/plugin/redis --
INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.redis.RedisPlugin
INFO main com.facebook.presto.server.PluginManager Registering connector redis
INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /usr/lib/presto/plugin/redis --
INFO main com.facebook.presto.server.PluginManager -- Loading plugin /usr/lib/presto/plugin/resource-group-managers --
INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.resourceGroups.ResourceGroupManagerPlugin
INFO main com.facebook.presto.server.PluginManager Registering resource group configuration manager file
INFO main com.facebook.presto.server.PluginManager Registering resource group configuration manager db
INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /usr/lib/presto/plugin/resource-group-managers --
INFO main com.facebook.presto.server.PluginManager -- Loading plugin /usr/lib/presto/plugin/teradata-functions --
INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.teradata.functions.TeradataFunctionsPlugin
INFO main com.facebook.presto.server.PluginManager Registering functions from com.facebook.presto.teradata.functions.TeradataDateFunctions
INFO main com.facebook.presto.server.PluginManager Registering functions from com.facebook.presto.teradata.functions.TeradataStringFunctions
INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /usr/lib/presto/plugin/teradata-functions --
INFO main com.facebook.presto.server.PluginManager -- Loading plugin /usr/lib/presto/plugin/tpch --
INFO main com.facebook.presto.server.PluginManager Installing com.facebook.presto.tpch.TpchPlugin
INFO main com.facebook.presto.server.PluginManager Registering connector tpch
INFO main com.facebook.presto.server.PluginManager -- Finished loading plugin /usr/lib/presto/plugin/tpch --
INFO main com.facebook.presto.metadata.StaticCatalogStore -- Loading catalog /etc/presto/conf/catalog/mysql.properties --
INFO main Bootstrap PROPERTY DEFAULT RUNTIME DESCRIPTION
INFO main Bootstrap connection-password null hogehoge
INFO main Bootstrap connection-url null hogehoge
INFO main Bootstrap connection-user null hogehoge
INFO main Bootstrap allow-drop-table false false Allow connector to drop tables
INFO main Bootstrap mysql.auto-reconnect true true
INFO main Bootstrap mysql.connection-timeout 10.00s 10.00s
INFO main Bootstrap mysql.max-reconnects 3 3
INFO main Bootstrap
INFO main com.facebook.presto.metadata.StaticCatalogStore -- Added catalog mysql using connector mysql --
INFO main com.facebook.presto.metadata.StaticCatalogStore -- Loading catalog /etc/presto/conf/catalog/hive.properties --
INFO main Bootstrap PROPERTY DEFAULT RUNTIME DESCRIPTION
INFO main Bootstrap hive.allow-corrupt-writes-for-testing false false Allow Hive connector to write data even when data will likely be corrupt
INFO main Bootstrap hive.assume-canonical-partition-keys false false
INFO main Bootstrap hive.bucket-execution true true Enable bucket-aware execution: only use a single worker per bucket
INFO main Bootstrap hive.bucket-writing true true Enable writing to bucketed tables
INFO main Bootstrap hive.dfs.connect.max-retries 5 5
INFO main Bootstrap hive.dfs.connect.timeout 500.00ms 500.00ms
INFO main Bootstrap hive.dfs-timeout 60.00s 60.00s
INFO main Bootstrap hive.domain-compaction-threshold 100 100 Maximum ranges to allow in a tuple domain without compacting it
INFO main Bootstrap hive.dfs.domain-socket-path null null
INFO main Bootstrap hive.fs.cache.max-size 1000 1000 Hadoop FileSystem cache size
INFO main Bootstrap hive.force-local-scheduling false false
INFO main Bootstrap hive.hdfs.authentication.type NONE NONE HDFS authentication type
INFO main Bootstrap hive.hdfs.impersonation.enabled false false Should Presto user be impersonated when communicating with HDFS
INFO main Bootstrap hive.hdfs.presto.keytab null null Presto keytab used to access HDFS
INFO main Bootstrap hive.hdfs.presto.principal null null Presto principal used to access HDFS
INFO main Bootstrap hive.compression-codec GZIP GZIP
INFO main Bootstrap hive.external-table-writable false true
INFO main Bootstrap hive.metastore.authentication.type NONE NONE Hive Metastore authentication type
INFO main Bootstrap hive.metastore.client.keytab null null Hive Metastore client keytab location
INFO main Bootstrap hive.metastore.client.principal null null Hive Metastore client principal
INFO main Bootstrap hive.metastore.service.principal null null Hive Metastore service principal
INFO main Bootstrap hive.storage-format RCBINARY RCBINARY
INFO main Bootstrap hive.immutable-partitions false false Can new data be inserted into existing partitions or existing unpartitioned tables
INFO main Bootstrap hive.dfs.ipc-ping-interval 10.00s 10.00s
INFO main Bootstrap hive.max-concurrent-file-renames 20 20
INFO main Bootstrap hive.max-initial-split-size 32MB 32MB
INFO main Bootstrap hive.max-initial-splits 200 200
INFO main Bootstrap hive.metastore-refresh-max-threads 100 100
INFO main Bootstrap hive.max-outstanding-splits 1000 1000
INFO main Bootstrap hive.metastore.partition-batch-size.max 100 100
INFO main Bootstrap hive.max-partitions-per-scan 100000 100000 Maximum allowed partitions for a single table scan
INFO main Bootstrap hive.max-partitions-per-writers 100 100 Maximum number of partitions per writer
INFO main Bootstrap hive.max-split-iterator-threads 1000 1000
INFO main Bootstrap hive.max-split-size 64MB 64MB
INFO main Bootstrap hive.metastore-cache-maximum-size 10000 10000
INFO main Bootstrap hive.metastore-cache-ttl 1.00h 20.00m
INFO main Bootstrap hive.metastore-refresh-interval 1.00s 1.00m
INFO main Bootstrap hive.metastore.thrift.client.socks-proxy null null
INFO main Bootstrap hive.metastore-timeout 10.00s 10.00s
INFO main Bootstrap hive.metastore.partition-batch-size.min 10 10
INFO main Bootstrap hive.orc.bloom-filters.enabled false false
INFO main Bootstrap hive.orc.max-buffer-size 8MB 8MB
INFO main Bootstrap hive.orc.max-merge-distance 1MB 1MB
INFO main Bootstrap hive.orc.stream-buffer-size 8MB 8MB
INFO main Bootstrap hive.parquet-optimized-reader.enabled false false
INFO main Bootstrap hive.parquet-predicate-pushdown.enabled false false
INFO main Bootstrap hive.per-transaction-metastore-cache-maximum-size 1000 1000
INFO main Bootstrap hive.rcfile-optimized-reader.enabled true true
INFO main Bootstrap hive.rcfile-optimized-writer.enabled false false
INFO main Bootstrap hive.recursive-directories false false
INFO main Bootstrap hive.config.resources null [/etc/hadoop/conf/core-site.xml, /etc/hadoop/conf/hdfs-site.xml]
INFO main Bootstrap hive.respect-table-format true true Should new partitions be written using the existing table format or the default Presto format
INFO main Bootstrap hive.skip-deletion-for-alter false false Skip deletion of old partition data when a partition is deleted and then inserted in the same transaction
INFO main Bootstrap hive.time-zone Universal Universal
INFO main Bootstrap hive.orc.use-column-names false false Access ORC columns using names from the file
INFO main Bootstrap hive.parquet.use-column-names false false Access Parquet columns using names from the file
INFO main Bootstrap hive.dfs.verify-checksum true true
INFO main Bootstrap hive.write-validation-threads 16 16 Number of threads used for verifying data after a write
INFO main Bootstrap hive.s3.pin-client-to-current-region false false Should the S3 client be pinned to the current EC2 region
INFO main Bootstrap hive.s3.aws-access-key null null
INFO main Bootstrap hive.s3.aws-secret-key null null
INFO main Bootstrap hive.s3.connect-timeout 5.00s 2.00m
INFO main Bootstrap hive.s3.encryption-materials-provider null null Use a custom encryption materials provider for S3 data encryption
INFO main Bootstrap hive.s3.endpoint null null
INFO main Bootstrap hive.s3.kms-key-id null null Use an AWS KMS key for S3 data encryption
INFO main Bootstrap hive.s3.max-backoff-time 10.00m 10.00m
INFO main Bootstrap hive.s3.max-client-retries 3 50
INFO main Bootstrap hive.s3.max-connections 500 500
INFO main Bootstrap hive.s3.max-error-retries 10 50
INFO main Bootstrap hive.s3.max-retry-time 10.00m 10.00m
INFO main Bootstrap hive.s3.multipart.min-file-size 16MB 16MB Minimum file size for an S3 multipart upload
INFO main Bootstrap hive.s3.multipart.min-part-size 5MB 5MB Minimum part size for an S3 multipart upload
INFO main Bootstrap hive.s3.signer-type null null
INFO main Bootstrap hive.s3.socket-timeout 5.00s 2.00m
INFO main Bootstrap hive.s3.sse.enabled false false Enable S3 server side encryption
INFO main Bootstrap hive.s3.sse.kms-key-id null null KMS Key ID to use for S3 server-side encryption with KMS-managed key
INFO main Bootstrap hive.s3.sse.type S3 S3 Key management type for S3 server-side encryption (S3 or KMS)
INFO main Bootstrap hive.s3.ssl.enabled true true
INFO main Bootstrap hive.s3.staging-directory /tmp /mnt/tmp Temporary directory for staging files before uploading to S3
INFO main Bootstrap hive.s3.use-instance-credentials true true
INFO main Bootstrap hive.s3.user-agent-prefix The user agent prefix to use for S3 calls
INFO main Bootstrap hive.metastore.uri null [thrift://ip-172-31-29-54.ap-northeast-1.compute.internal:9083] Hive metastore URIs (comma separated)
INFO main Bootstrap hive.metastore thrift thrift
INFO main Bootstrap hive.allow-add-column false false Allow Hive connector to add column
INFO main Bootstrap hive.allow-drop-table false false Allow Hive connector to drop table
INFO main Bootstrap hive.allow-rename-column false false Allow Hive connector to rename column
INFO main Bootstrap hive.allow-rename-table false false Allow Hive connector to rename table
INFO main Bootstrap hive.security legacy legacy
INFO main Bootstrap
INFO main com.facebook.presto.metadata.StaticCatalogStore -- Added catalog hive using connector hive-hadoop2 --
INFO main com.facebook.presto.execution.resourceGroups.InternalResourceGroupManager -- Loading resource group configuration manager --
INFO main com.facebook.presto.execution.resourceGroups.InternalResourceGroupManager -- Loaded resource group configuration manager legacy --
INFO main com.facebook.presto.security.AccessControlManager -- Loading system access control --
INFO main com.facebook.presto.security.AccessControlManager -- Loaded system access control allow-all --
INFO main com.facebook.presto.server.PrestoServer ======== SERVER STARTED ========
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment