Skip to content

Instantly share code, notes, and snippets.

@dakatsuka
Created May 31, 2017 15:45
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save dakatsuka/8c73c8232b95e70c7baf1a5fee74b2bc to your computer and use it in GitHub Desktop.
Save dakatsuka/8c73c8232b95e70c7baf1a5fee74b2bc to your computer and use it in GitHub Desktop.
List s3 keys recursively
import scala.collection.JavaConverters._
val request = new ListObjectsV2Request()
request.setBucketName(bucket)
request.setPrefix(prefix)
@tailrec
def fetch(req: ListObjectsV2Request, xs: Iterable[String]): Iterable[String] = {
val result = s3Client.listObjectsV2(req)
val keys = xs ++ result.getObjectSummaries.asScala.map(_.getKey)
if (result.isTruncated) {
req.setContinuationToken(result.getNextContinuationToken)
fetch(req, keys)
} else keys
}
Source(fetch(request, Iterable.empty[String]).toList)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment