Skip to content

Instantly share code, notes, and snippets.

@avar
Last active August 29, 2015 14:19
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save avar/6f0dce9c91bf6afa4e30 to your computer and use it in GitHub Desktop.
Save avar/6f0dce9c91bf6afa4e30 to your computer and use it in GitHub Desktop.
An example of writing long JSON lines to a file atomically with Perl while warning about writes in excess of PIPE_BUF
use strict;
use warnings;
use JSON::XS ();
use POSIX qw(PIPE_BUF);
my $PATH = ...;
my $DATA = ...;
my $TIME = time;
my $JSON = JSON::XS->new->ascii(1)->indent(0);
open my $fh, ">>", $PATH or die "PANIC: Unable to open(<$PATH>): <$!>";
for my $item (@$DATA) {
my $json = $JSON->encode($item);
my $output = sprintf "%d\t%s\n", $TIME, $json;
my $length = bytes::length($output);
warn "PANIC: We shouldn't write items of size >PIPE_BUF. We're relying on atomic writes! Potentially writing corrupt data!"
if $length > PIPE_BUF;
syswrite $fh, $output or die "PANIC: Unable to syswrite() <$length> bytes of output: <$!>";
}
close $fh or die "PANIC: Unable to close(<$PATH>): <$!>";
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment