logstash wont' work unless starting as root

logstash-start-as-root.md

cat /etc/logstash/conf.d/logstash.conf

input {
file {
  path => ['/tmp/foo']
}
}
output {
file {
  path => ['/tmp/bar']
}
}

I create the source and destination files

touch /tmp/foo
touch /tmp/bar

chown root:adm /tmp/foo
chwon root:adm /tmp/bar

chmod 644 /tmp/foo
chmod 777 /tmp/bar

I start logstash using the init script provided by the offical rpm logstash starts as user logstash:logstash

service logstash start
ps aux |grep logstash
logstash 27222  1.7 24.0 1380572 121292 ?      SNsl 21:27   0:37 /usr/bin/java -Xmx256m -Djava.io.tmpdir=/var/lib/logstash/ -Xmx500m -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -jar /opt/logstash/vendor/jar/jruby-complete-1.7.11.jar -I/opt/logstash/lib /opt/logstash/lib/logstash/runner.rb agent -f /etc/logstash/conf.d -l /var/log/logstash/logstash.log --log /var/log/logstash/logstash.log

When I append to the first file (foo), it shows up in the second file (bar).

echo "testing permissions 644" >> /tmp/foo
cat /tmp/bar
{"message":"testing permissions 644","@version":"1","@timestamp":"2014-05-21T21:06:06.647Z","host":"foosball.ac","path":"/tmp/foo"}

If I stop logstash, change foo from 644 to 640 everything breaks.

service logstash stop; service logstash start; sleep 120;
echo "Testing permissions 640" >> /tmp/foo

cat /tmp/bar
(the new line never appears)

id logstash
uid=2000(logstash) gid=3001(logstash) groups=4(adm),3001(logstash)

The logstash user can read from /tmp/bar no problem

sudo -u logstash -g logstash -H cat /tmp/bar
{"message":"herp","@version":"1","@timestamp":"2014-05-21T19:46:57.761Z","type":"foo","host":"foosball.ac","path":"/tmp/foo"}
{"message":"herpy derpy","@version":"1","@timestamp":"2014-05-21T19:51:06.052Z","host":"foosball.ac","path":"/tmp/foo"}

Update

the logstash user is able to read /modify the file

sudo -u logstash -g logstash -H echo "im logstash, editing your files as 640" >> /tmp/foo

electrical in the irc channel says that there is a known bug where additional groups are not passed in.

It must have something todo with the init script -- running logstash as the user directly seemed to work

coolacid fixed this here https://github.com/coolacid/logstash/blob/init-fix/pkg/logstash.sysv

The diff of the file looks like this:

root@foosball:~# diff /etc/init.d/logstash-coolacid /etc/init.d/logstash

53,56d53
<   # chown doesn't grab the suplimental groups when setting the user:group - so we have to do it for it.
<   # Boy, I hope we're root here.
<   SGROUPS=$(grep "$LS_USER" /etc/group | grep -v "^$LS_GROUP" | cut -d: -f1 | tr "\\n" "," | sed 's/,$//'; echo '')
<
61c58
<   nice -n ${LS_NICE} chroot --userspec $LS_USER:$LS_GROUP --groups $SGROUPS / sh -c "

---
>   nice -n ${LS_NICE} chroot --userspec $LS_USER:$LS_GROUP / sh -c "

Logstash PR: elastic/logstash#1398