Skip to content

Instantly share code, notes, and snippets.

View steveloughran's full-sized avatar
🌴
offline until 2025

Steve Loughran steveloughran

🌴
offline until 2025
View GitHub Profile
@steveloughran
steveloughran / faslane-synchrotron.gpx
Created January 29, 2018 03:47
Faslane Synchrotron raw GPX trace
<?xml version="1.0" ?>
<gpx xmlns="http://www.topografix.com/GPX/1/1">
<trk>
<name>
faslane synchrotron
</name>
<trkseg>
<trkpt lat="56.0666066942" lon="-4.8184543848"/>
<trkpt lat="56.0664389978" lon="-4.8188728094"/>
<trkpt lat="56.0664389978" lon="-4.8188728094"/>
- listFiles
2017-06-27 15:47:02,289 [ScalaTest-main-running-S3ACSVReadSuite] INFO s3.S3ACSVReadSuite (Logging.scala:logInfo(54)) - ReadBytesReturned
Read in blocks and assess their size and duration.
This is to identify buffering quirks.
-------------------------------------------
2017-06-27 15:47:03,003 [ScalaTest-main-running-S3ACSVReadSuite] INFO s3a.S3AFileSystem (S3AFileSystem.java:initialize(276)) - Magic committer is enabled
2017-06-27 15:47:05,121 [ScalaTest-main-running-S3ACSVReadSuite] WARN internal.S3AbortableInputStream (S3AbortableInputStream.java:close(163)) - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use.
2017-06-27 15:47:06,730 [ScalaTest-main-running-S3ACSVReadSuite] WARN internal.S3AbortableInputStream (S3AbortableInputStream.java:close(163)) - Not all bytes were read from the S3ObjectInputStream, aborting
@steveloughran
steveloughran / gist:c8b39a7b87a9bd63d7a383bda8687e7e
Created August 23, 2016 15:20
log of a streaming against s3 test
2016-08-23 16:17:51,877 INFO server.Server (Server.java:doStart(379)) - Started @4407ms
2016-08-23 16:17:51,878 INFO util.Utils (Logging.scala:logInfo(54)) - Successfully started service 'SparkUI' on port 4040.
2016-08-23 16:17:51,881 INFO ui.SparkUI (Logging.scala:logInfo(54)) - Bound SparkUI to 0.0.0.0, and started at http://192.168.1.7:4040
2016-08-23 16:17:51,953 INFO executor.Executor (Logging.scala:logInfo(54)) - Starting executor ID driver on host localhost
2016-08-23 16:17:51,984 INFO util.Utils (Logging.scala:logInfo(54)) - Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 61294.
2016-08-23 16:17:51,985 INFO netty.NettyBlockTransferService (Logging.scala:logInfo(54)) - Server created on 192.168.1.7:61294
2016-08-23 16:17:51,986 INFO storage.BlockManagerMaster (Logging.scala:logInfo(54)) - Registering BlockManager BlockManagerId(driver, 192.168.1.7, 61294)
2016-08-23 16:17:51,991 INFO storage.BlockManagerMasterEndpoint (Logging.scala:logInfo(54)) -