Skip to content

Instantly share code, notes, and snippets.

@elarib
elarib / hdfs.scala
Created March 20, 2019 16:14
List recusively directories
def listSubdirectoriesRecursively(
directory: Path,
list: mutable.ListBuffer[Path] = new mutable.ListBuffer[Path]
): List[Path] = {
val iterator = fs.listLocatedStatus(directory)
while (iterator.hasNext) {
val pathStatus = iterator.next()
val path = pathStatus.getPath
@elarib
elarib / get_original_filename_of_gzip.py
Created January 14, 2019 13:05
Get original filename of a gzip (python)
# Try to get the original filename (present in the header) of the gz file.
# Check http://www.onicos.com/staff/iz/formats/gzip.html
def get_original_fileName(path):
with open(path, 'rb') as f:
magic = f.read(2)
if magic != '\037\213':
raise IOError, 'Not a gzipped file'
@elarib
elarib / do & grep
Created September 18, 2017 13:20
Tips for linux command
ls -l -ad dev-*