p120ph37/README.md

## README.md

      
    Raw
  

              README.md
            
          
    teesocket.pl

Like tee, but with dynamically-attached socket clients
Usage

Server mode

./teesocket.pl -s /tmp/my.sock
Client mode

./teesocket.pl -c /tmp/my.sock
Examples

Observable Output Log

$ (while true; do sleep 1; echo $((n++)); done) | ./teesocket.pl -s /tmp/my.log.sock >>/tmp/my.log &
[1] 18012
$ tail -5 /tmp/my.log
54
55
56
57
58
$ ./teesocket.pl -c /tmp/my.log.sock
61
62
63
64
65
^C
$

This also works when output is piped to some other process such as rotatelogs:
$ (while true; do sleep 1; echo $((n++)); done) | ./teesocket.pl -s /tmp/my.log.sock | rotatelogs /tmp/my.log 60 &
[1] 16405
$ # ...wait a few minutes here so we can see that rotatelogs is working...
$ ls -l /tmp/my.log.*
-rw-r--r-- 1 root root  38 Jan 25 02:33 /tmp/my.log.1611538380
-rw-r--r-- 1 root root 180 Jan 25 02:34 /tmp/my.log.1611538440
-rw-r--r-- 1 root root  82 Jan 25 02:35 /tmp/my.log.1611538500
srwxr-xr-x 1 root root   0 Jan 25 02:33 /tmp/my.log.sock
$ ./teesocket.pl -c /tmp/my.log.sock
./teesocket.pl -c /tmp/my.log.sock
157
158
159
160
161
^C
$

Popcorn

Create a "popcorn" service
by writing the output of the date command to teesocket once a second.  The primary output
of teesocket.pl is directed to /dev/null, and client connections can be made to the unix
socket to begin observing the stream at any time.
$ setsid sh -c '(while true; do sleep 1; date; done) | ./teesocket.pl -s /tmp/popcorn.sock >/dev/null'
$ ./teesocket.pl -c /tmp/popcorn.sock
Mon Jan 25 02:15:35 CET 2021
Mon Jan 25 02:15:36 CET 2021
Mon Jan 25 02:15:37 CET 2021
Mon Jan 25 02:15:38 CET 2021
Mon Jan 25 02:15:39 CET 2021
^C
$

Using a FIFO to look like a file

Sometimes you may need to attach this utility to the output of a process which normally writes to a file.
By creating a FIFO at the location where the process will write, you can capture the stream and handle it.
In this example, we create a FIFO, attach teesocket to read from it and write to /dev/null, then we start
a process which logs to the filename of the FIFO.  The output is discarded, until we use teesocket to
begin observing. (note: the tail -f layer is used in order to insulate teesocket from the EOF event
which would otherwise occur each time the FIFO gets closed by one of the wget processes)
$ mkfifo /tmp/my.fifo
$ tail -f /tmp/my.fifo | ./teesocket.pl -s /tmp/my.sock >/dev/null &
$ while true; do sleep 1; wget -a /tmp/my.fifo http://example.com; done &
[1] 4640
$ tail -f /tmp/my.fifo | ./teesocket.pl -s /tmp/my.sock
Resolving example.com (example.com)... 2606:2800:220:1:248:1893:25c8:1946, 93.184.216.34
Connecting to example.com (example.com)|2606:2800:220:1:248:1893:25c8:1946|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1256 (1.2K) [text/html]
Saving to: ‘index.html.30’

     0K .                                                     100%  105M=0s

2021-01-25 03:46:02 (105 MB/s) - ‘index.html.30’ saved [1256/1256]

--2021-01-25 03:46:03--  http://example.com/
Resolving example.com (example.com)... 2606:2800:220:1:248:1893:25c8:1946, 93.184.216.34
Connecting to example.com (example.com)|2606:2800:220:1:248:1893:25c8:1946|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1256 (1.2K) [text/html]
Saving to: ‘index.html.31’

     0K .                                                     100%  101M=0s

2021-01-25 03:46:03 (101 MB/s) - ‘index.html.31’ saved [1256/1256]

^C
$

Sampling the STDOUT stream within Docker

When using STDOUT as the log output of a Docker container, such that the Docker Engine is
responsible for storing or forwarding the log stream, it is sometimes necessary for processes
within the container to sample this stream.  Since the container itself may not know where
the stream is stored or sent by the Engine, in order to maintain a separation of concerns,
it is useful to sample this stream before it leaves the container.
Within a Docker container, the STDOUT which is passed to the Docker Engine logdriver is the
STDOUT of the primary process, i.e. /proc/1/fd/1  Knowing this, we can set up teesocket
with a FIFO for input as in the above example, and with this specific file descriptor as
output (or just the default output if it is started at a point in the container when its
output is still connected), and use the FIFO as the log target of all other processes within
the contianer.
$ docker build -t teesocket - <<'EOF'
FROM debian:10-slim
ADD https://gist.githubusercontent.com/p120ph37/9c4f566869f0ced3cca3c7b54a435e6c/raw/teesocket.pl /teesocket.pl
RUN chmod 755 /teesocket.pl
RUN echo "#!/bin/sh\n\
mkfifo /tmp/container-output.fifo\n\
setsid sh -c 'tail -f /tmp/container-output.fifo | ./teesocket.pl -s /tmp/container-output.sock' &\n\
while true; do sleep 1; date >/tmp/container-output.fifo; done" >/startup.sh \
&& chmod 755 /startup.sh
CMD ["/startup.sh"]
EOF
...
Successfully built b52ecdcbd1c7
Successfully tagged teesocket:latest
$ docker run --name teesocket -d teesocket
1b39a4639ae2d74d04d3bdcdcf5ad999fc5fbcd19748051be8041705139aba43
$ docker logs teesocket
Mon Jan 25 18:05:11 UTC 2021
Mon Jan 25 18:05:12 UTC 2021
$ docker exec -ti teesocket /bin/bash
root@1b39a4639ae2:/# ./teesocket.pl -c /tmp/container-output.sock
Mon Jan 25 18:05:35 UTC 2021
Mon Jan 25 18:05:36 UTC 2021
Mon Jan 25 18:05:37 UTC 2021
^C
root@1b39a4639ae2:/# exit
$

Compatability with socat

socat is a general-purpose socket connection tool.  Because the socket produced by teesocket
is a generic UNIX socket, it can be observed not only by teesocket itself, but you can
also use socat to observe it:
$ (while true; do sleep 1; echo $((n++)); done) | ./teesocket.pl -s /tmp/my.log.sock >>/tmp/my.log &
[1] 8709
$ tail -5 /tmp/my.log
4
5
6
7
8
$ socat -u UNIX-CONNECT:/tmp/my.log.sock STDOUT
32
33
34
35
^C
$


## teesocket.pl
#!/usr/bin/perl
use warnings;
use strict;
use Errno qw(EAGAIN EADDRINUSE ECONNREFUSED);
use IO::Socket::UNIX;
use IO::Select;

my $bufsiz = 8192;
my $buffer = '';

STDIN->binmode();
STDIN->blocking(0);
STDOUT->binmode();
STDOUT->blocking(0);

if($ARGV[0] eq '-s' and @ARGV == 2) {
    server($ARGV[1]);
} elsif($ARGV[0] eq '-c' and @ARGV == 2) {
    client($ARGV[1]);
} else {
    die "Usage: $0 { -s socketpath | -c socketpath }\n";
}

# $0 -c filename
sub client {
    my($socketname) = @_;

    STDOUT->blocking(1);

    my $client = IO::Socket::UNIX->new(
        Blocking => 0,
        Peer => $socketname,
    ) or die $!;

    STDOUT->syswrite($buffer);
    my $rs = IO::Select->new($client);
    my $ws = IO::Select->new();

    while(1) {
        $! = 0; my($rhs, $whs) = IO::Select->select($rs, $ws, undef);
        $! == EAGAIN and redo or $! and die $!;
        if(@{$rhs}) {
            $ENV{DEBUG} and warn "Socket ready for reading\n";
            # read what we can from the client socket
            my $c = $client->sysread(my $newdata, $bufsiz - length($buffer));
            # add newdata to main output buffer
            $buffer .= $newdata;
            # if end of input, block until buffer is written then exit
            if($c == 0) {
                STDOUT->blocking(1);
                STDOUT->syswrite($buffer);
                exit;
            }
            # if buffer is full, stop reading for now
            length($buffer) >= $bufsiz and $rs->remove($client);
            # if buffer is non-empty, ensure we are tring to write
            length($buffer) > 0 and $ws->add(\*STDOUT);
        }
        if(@{$whs}) {
            $ENV{DEBUG} and warn "STDOUT ready for writing\n";
            # write what we can to stdout
            my $c = STDOUT->syswrite($buffer);
            # if output error, exit
            not defined $c and exit;
            $buffer = substr($buffer, $c);
            # if buffer is non-full, ensure we are trying to read
            length($buffer) < $bufsiz and $rs->add($client);
            # if buffer is empty, stop writing for now
            length($buffer) == 0 and $ws->remove(\*STDOUT)
        }
    }
}

# $0 -s filename
sub server {
    my($socketname) = @_;

    not IO::Socket::UNIX->new(Peer => $socketname) or die "Socket already in use: $socketname\n";
    -e $socketname and (unlink $socketname or die $!);

    my $cleanup = sub { unlink $socketname; };
    $SIG{PIPE} = 'IGNORE';
    $SIG{__DIE__} = $cleanup;
    $SIG{INT} = $SIG{TERM} = sub { $cleanup->(); exit; };

    my $server = IO::Socket::UNIX->new(
        Blocking => 0,
        Local => $socketname,
        Listen => 5,
    ) or die $!;

    my @clients = ();
    my $rs = IO::Select->new(\*STDIN, $server);
    my $ws = IO::Select->new();

    while(1) {
        $! = 0; my($rhs, $whs) = IO::Select->select($rs, $ws, undef);
        $! == EAGAIN and redo or $! and die $!;
        for my $rh (@{$rhs}) {
            $ENV{DEBUG} and warn "Handle @{[$rh->fileno]} ready for reading\n";
            if($rh->fileno == STDIN->fileno) {
                # read what we can from stdin
                my $c = STDIN->sysread(my $newdata, $bufsiz - length($buffer));
                # blindly try to copy the new data to all clients (if client isn't keeping up, too bad)
                $_->syswrite($newdata) for @clients;
                # add newdata to main output buffer
                $buffer .= $newdata;
                # if end of input, block until buffer is written then exit
                if($c == 0) {
                    STDOUT->blocking(1);
                    STDOUT->syswrite($buffer);
                    exit;
                }
                # if buffer is full, stop reading for now
                length($buffer) >= $bufsiz and $rs->remove(\*STDIN);
                # if buffer is non-empty, ensure we are tring to write
                length($buffer) > 0 and $ws->add(\*STDOUT);
            } elsif($rh->fileno == $server->fileno) {
                # server read
                if(my $client = $server->accept()) {
                    $client->blocking(0);
                    push @clients, $client;
                    $rs->add($client);
                    $ENV{DEBUG} and warn "Created client filehandle @{[$client->fileno]}\n";
                }
            } else {
                # client read (only care about the eof signal)
                my $c = $rh->sysread($_, $bufsiz);
                not defined $c and warn $!;
                if(defined $c and $c == 0) {
                    @clients = grep{$_->fileno != $rh->fileno}(@clients);
                    $rs->remove($rh);
                    $rh->close;
                    $ENV{DEBUG} and warn "Destroyed clinet filehandle @{[$rh->fileno]}\n";
                }
            }
        }
        for my $wh (@{$whs}) {
            $ENV{DEBUG} and warn "Handle @{[$wh->fileno]} ready for writing\n";
            if($wh->fileno == STDOUT->fileno) {
                # write what we can to stdout
                my $c = STDOUT->syswrite($buffer);
                # if output error, exit
                not defined $c and exit;
                $buffer = substr($buffer, $c);
                # if buffer is non-full, ensure we are trying to read
                length($buffer) < $bufsiz and $rs->add(\*STDIN);
                # if buffer is empty, stop writing for now
                length($buffer) == 0 and $ws->remove(\*STDOUT)
            }
        }
    }
}
	#!/usr/bin/perl
	use warnings;
	use strict;
	use Errno qw(EAGAIN EADDRINUSE ECONNREFUSED);
	use IO::Socket::UNIX;
	use IO::Select;

	my $bufsiz = 8192;
	my $buffer = '';

	STDIN->binmode();
	STDIN->blocking(0);
	STDOUT->binmode();
	STDOUT->blocking(0);

	if($ARGV[0] eq '-s' and @ARGV == 2) {
	server($ARGV[1]);
	} elsif($ARGV[0] eq '-c' and @ARGV == 2) {
	client($ARGV[1]);
	} else {
	die "Usage: $0 { -s socketpath \| -c socketpath }\n";
	}

	# $0 -c filename
	sub client {
	my($socketname) = @_;

	STDOUT->blocking(1);

	my $client = IO::Socket::UNIX->new(
	Blocking => 0,
	Peer => $socketname,
	) or die $!;

	STDOUT->syswrite($buffer);
	my $rs = IO::Select->new($client);
	my $ws = IO::Select->new();

	while(1) {
	$! = 0; my($rhs, $whs) = IO::Select->select($rs, $ws, undef);
	$! == EAGAIN and redo or $! and die $!;
	if(@{$rhs}) {
	$ENV{DEBUG} and warn "Socket ready for reading\n";
	# read what we can from the client socket
	my $c = $client->sysread(my $newdata, $bufsiz - length($buffer));
	# add newdata to main output buffer
	$buffer .= $newdata;
	# if end of input, block until buffer is written then exit
	if($c == 0) {
	STDOUT->blocking(1);
	STDOUT->syswrite($buffer);
	exit;
	}
	# if buffer is full, stop reading for now
	length($buffer) >= $bufsiz and $rs->remove($client);
	# if buffer is non-empty, ensure we are tring to write
	length($buffer) > 0 and $ws->add(\*STDOUT);
	}
	if(@{$whs}) {
	$ENV{DEBUG} and warn "STDOUT ready for writing\n";
	# write what we can to stdout
	my $c = STDOUT->syswrite($buffer);
	# if output error, exit
	not defined $c and exit;
	$buffer = substr($buffer, $c);
	# if buffer is non-full, ensure we are trying to read
	length($buffer) < $bufsiz and $rs->add($client);
	# if buffer is empty, stop writing for now
	length($buffer) == 0 and $ws->remove(\*STDOUT)
	}
	}
	}

	# $0 -s filename
	sub server {
	my($socketname) = @_;

	not IO::Socket::UNIX->new(Peer => $socketname) or die "Socket already in use: $socketname\n";
	-e $socketname and (unlink $socketname or die $!);

	my $cleanup = sub { unlink $socketname; };
	$SIG{PIPE} = 'IGNORE';
	$SIG{__DIE__} = $cleanup;
	$SIG{INT} = $SIG{TERM} = sub { $cleanup->(); exit; };

	my $server = IO::Socket::UNIX->new(
	Blocking => 0,
	Local => $socketname,
	Listen => 5,
	) or die $!;

	my @clients = ();
	my $rs = IO::Select->new(\*STDIN, $server);
	my $ws = IO::Select->new();

	while(1) {
	$! = 0; my($rhs, $whs) = IO::Select->select($rs, $ws, undef);
	$! == EAGAIN and redo or $! and die $!;
	for my $rh (@{$rhs}) {
	$ENV{DEBUG} and warn "Handle @{[$rh->fileno]} ready for reading\n";
	if($rh->fileno == STDIN->fileno) {
	# read what we can from stdin
	my $c = STDIN->sysread(my $newdata, $bufsiz - length($buffer));
	# blindly try to copy the new data to all clients (if client isn't keeping up, too bad)
	$_->syswrite($newdata) for @clients;
	# add newdata to main output buffer
	$buffer .= $newdata;
	# if end of input, block until buffer is written then exit
	if($c == 0) {
	STDOUT->blocking(1);
	STDOUT->syswrite($buffer);
	exit;
	}
	# if buffer is full, stop reading for now
	length($buffer) >= $bufsiz and $rs->remove(\*STDIN);
	# if buffer is non-empty, ensure we are tring to write
	length($buffer) > 0 and $ws->add(\*STDOUT);
	} elsif($rh->fileno == $server->fileno) {
	# server read
	if(my $client = $server->accept()) {
	$client->blocking(0);
	push @clients, $client;
	$rs->add($client);
	$ENV{DEBUG} and warn "Created client filehandle @{[$client->fileno]}\n";
	}
	} else {
	# client read (only care about the eof signal)
	my $c = $rh->sysread($_, $bufsiz);
	not defined $c and warn $!;
	if(defined $c and $c == 0) {
	@clients = grep{$_->fileno != $rh->fileno}(@clients);
	$rs->remove($rh);
	$rh->close;
	$ENV{DEBUG} and warn "Destroyed clinet filehandle @{[$rh->fileno]}\n";
	}
	}
	}
	for my $wh (@{$whs}) {
	$ENV{DEBUG} and warn "Handle @{[$wh->fileno]} ready for writing\n";
	if($wh->fileno == STDOUT->fileno) {
	# write what we can to stdout
	my $c = STDOUT->syswrite($buffer);
	# if output error, exit
	not defined $c and exit;
	$buffer = substr($buffer, $c);
	# if buffer is non-full, ensure we are trying to read
	length($buffer) < $bufsiz and $rs->add(\*STDIN);
	# if buffer is empty, stop writing for now
	length($buffer) == 0 and $ws->remove(\*STDOUT)
	}
	}
	}
	}