Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
This is a sample of how to send some information to logstash via the TCP input from node.js or python.

This is a sample of how to send some information to logstash via the TCP input in nodejs or python. It assumes the logstash host is on 10.10.10.100 and the TCP listening input is 9563.

The logstash.conf should look something like the sample file.

The log message should be a stringified JSON object with the log message in the @message field.

To use, run the node script node sendMessageToLogstash.js, or the python script python sendMessageToLogstash.js

input {
tcp {
'charset' => 'UTF-8'
'format' => 'json_event'
'port' => '9563'
'type' => 'stucco-tcp'
}
}
output {
elasticsearch_http {
'host' => 'localhost'
'port' => '9200'
}
file {
'path' => '/usr/local/logstash/server/log/output.log'
}
}
/* globals require:true, console:true, process:true */
// This script will send a message to a [logstash](http://logstash.net/)
// server using the [TCP input](http://logstash.net/docs/1.1.13/inputs/tcp)
// and then quit. If there is no listener, it will just quit.
'use strict';
var net = require('net');
var logHost = '10.10.10.100'
, logPort = 9563
, sender = require('os').hostname();
var conn = net.createConnection({host: logHost, port: logPort}, function() {
var message = {
'@tags': ['nodejs', 'test']
, '@message': 'tcp test ' + Math.floor(Math.random() * 10000)
, '@fields': {'sender': sender}
}
conn.write(JSON.stringify(message));
process.exit(0);
})
.on('error', function(err) {
console.error(err);
process.exit(1);
});
import socket
import json
import sys
HOST = '10.10.10.100'
PORT = 9563
try:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error, msg:
sys.stderr.write("[ERROR] %s\n" % msg[1])
sys.exit(1)
try:
sock.connect((HOST, PORT))
except socket.error, msg:
sys.stderr.write("[ERROR] %s\n" % msg[1])
sys.exit(2)
msg = {'@message': 'python test message', '@tags': ['python', 'test']}
sock.send(json.dumps(msg))
sock.close()
sys.exit(0)
@sombralibre

This comment has been minimized.

Copy link

@sombralibre sombralibre commented May 22, 2018

this works!

import socket
import json
import sys

HOST = '10.10.10.100'
PORT = 9563

try:
  sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error as msg:
  sys.stderr.write("[ERROR] %s\n" % msg[1])
  sys.exit(1)

try:
  sock.connect((HOST, PORT))
except socket.error as msg:
  sys.stderr.write("[ERROR] %s\n" % msg[1])
  sys.exit(2)

msg = {'@message': 'python test message', '@tags': ['python', 'test']}

sock.send(str(json.dumps(msg) ).encode('utf-8') )

sock.close()
sys.exit(0)
@Zoya-bhavya

This comment has been minimized.

Copy link

@Zoya-bhavya Zoya-bhavya commented Jun 14, 2018

why node program sends data to logstash after quitting?

@Zoya-bhavya

This comment has been minimized.

Copy link

@Zoya-bhavya Zoya-bhavya commented Jun 14, 2018

Is there any way that i can keep sending data to logstash on particular intervals from my nodejs program

@Theraghav

This comment has been minimized.

Copy link

@Theraghav Theraghav commented Sep 3, 2018

@Zoya-bhavya, did you find an answer to your question?

@hbinduni

This comment has been minimized.

Copy link

@hbinduni hbinduni commented Jan 21, 2021

why node program sends data to logstash after quitting?

you need to end your msg with new line. so it should be something like client.write(msg + '\n')

@tamaroth

This comment has been minimized.

Copy link

@tamaroth tamaroth commented Jun 4, 2021

I stumbled upon this while looking for a simple way to send logs directly to Logstash.

I've used your basic idea and adjusted it to my needs:

  • i have a log file with structured logs, where each line is a JSON object and I wanted to feed this to Logstash
  • I do not care about exceptions, it's a quick hack and if it throws I'd prefer to see the entire stack
  • My personal ELK stack was installed from docker-elk, with docker-elk/logstash/pipeline/logstash.conf being the following:
input {
        beats {
                port => 5044
        }

        tcp {
                port => 5000
                codec => json
        }
}

## Add your filters / logstash plugins configuration here

output {
        elasticsearch {
                hosts => "elasticsearch:9200"
                user => "elastic"
                password => "changeme"
                ecs_compatibility => disabled
        }
}

The script I used is this:

import socket

host = "localhost"
port = 5000


with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
    sock.connect((host, port))

    with open("/path/to/log/file.log", mode="rb") as f:
        for line in f.readlines():
            sock.send(line)
    print("all data sent to Logstash")

If you don't mind, I'm dropping it here in hopes it helps someone else when they scour the internet for similar problem. Thanks for the base code!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment