This is without any reasonable idea about SOLR configuration. It just happens to work here. It is also specific to my server, you might severly damage yours if you copy & paste this blindly!
All commands to be executed as root.
apt-get remove --purge solr-common