The thing that trips most people up is that the parameters to the web cli (
--backend elasticsearch://blah:9300/blah) do NOT set the output destination.
The web app is its own process with its own args. It knows nothing about the agent config file.
Remember that you could simply use the same jar like so:
java -jar logstash.jar web --backend elasticsearch://elasticsearch:9300/clustername
and run the web interface without the agent process.
Port numbers and transport client
What also trips people up is port assignments. Logstash connects to elasticsearch as a
transport client. This means it becomes a part of the elasticsearch cluster but does not store any data.
If you're running
web and an external ES process on the same machine, all three processes will try and use port 9300. This won't work. The problem is that it MIGHT work depending on startup order. If you start ES first, it will grab port 9300. When logstash comes up, the agent and web processes will increment to
Now let's imagine you shut everything down and start logstash first. Likely
agent will get
9300 and web will get
9301. I'm not positive here but ES will I THINK jump to
9302. The problem is that you've told the web to talk to
9300 which is now the
The best bet here if you're running on the same machine is to lock ES to a different port and use that. That way there's never an issue. This could cause problems when you start to cluster though. So really if you're running ES externally from logstash (i.e. not embedded) run it on a different instance/machine.
Quick note on versions
Elasticsearch maintains compatibility only between minor versions. All clients and servers connecting via transport MUST be on the same major version.
This means that
0.18.x nodes can talk to a
0.18.y server regardless of what
y are. A
0.19.x client or server cannot talk to a
0.18.x client or server.
Long story short
If you're trying to run ES on the same machine as your logstash web or agent process, just use the embedded ES. You aren't buying yourself anything.