gistfile1.txt

App 262 stdout:
[ 2014-12-19 19:00:37.8352 243/7f44e3d53700 Pool2/SmartSpawner.h:298 ]: Preloader for /home/app/loewy-api started on PID 262, listening on unix:/tmp/passenger.1.0.113/generation-0/backends/preloader.10r4g3n
App 262 stderr: /usr/lib/ruby/vendor_ruby/phusion_passenger/preloader_shared_helpers.rb:69:in `fork'
App 262 stderr: :
App 262 stderr: Cannot allocate memory - fork(2)
App 262 stderr:  (
App 262 stderr: Errno::ENOMEM
App 262 stderr: )
App 262 stderr: 	from /usr/lib/ruby/vendor_ruby/phusion_passenger/preloader_shared_helpers.rb:69:in `accept_and_process_next_client'
App 262 stderr: 	from /usr/lib/ruby/vendor_ruby/phusion_passenger/preloader_shared_helpers.rb:139:in `run_main_loop'
App 262 stderr: 	from /usr/share/passenger/helper-scripts/rack-preloader.rb:159:in `<module:App>'
App 262 stderr: 	from /usr/share/passenger/helper-scripts/rack-preloader.rb:29:in `<module:PhusionPassenger>'
App 262 stderr: 	from /usr/share/passenger/helper-scripts/rack-preloader.rb:28:in `<main>'
[ pid=243, timestamp=1419015637 ] Process aborted! signo=SIGSEGV(11), reason=SEGV_MAPERR, si_addr=0x160, randomSeed=1419015578
[ pid=243 ] Crash log dumped to /var/tmp/passenger-crash-log.1419015637
[ pid=243 ] Date, uname and ulimits:
Fri Dec 19 19:00:38 UTC 2014
Linux 3.13.0-33-generic #58-Ubuntu SMP Tue Jul 29 16:45:05 UTC 2014 x86_64 x86_64
time(seconds)        unlimited
file(blocks)         unlimited
data(kbytes)         unlimited
stack(kbytes)        8192
coredump(blocks)     0
memory(kbytes)       unlimited
locked memory(kbytes) 64
process              524288
nofiles              524288
vmemory(kbytes)      unlimited
locks                unlimited
[ pid=243 ] Phusion Passenger version: 4.0.53
[ pid=243 ] libc backtrace available!
--------------------------------------
[ pid=243 ] Backtrace with 30 frames:
Using backtrace sanitizer.
*** backtrace-sanitizer warning: Permission denied @ rb_readlink - /proc/243/exe -> passthrough input
PassengerHelperAgent[0x54bcc7]
PassengerHelperAgent[0x54b579]
PassengerHelperAgent[0x54c874]
PassengerHelperAgent[0x54d52e]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x10340)[0x7f44e381e340]
PassengerHelperAgent[0x520788]
PassengerHelperAgent[0x4569f2]
PassengerHelperAgent[0x456665]
PassengerHelperAgent[0x45ae66]
PassengerHelperAgent[0x45fcdf]
PassengerHelperAgent[0x460e6b]
PassengerHelperAgent[0x528c89]
PassengerHelperAgent[0x52869d]
PassengerHelperAgent[0x50308c]
PassengerHelperAgent[0x4fb1f0]
PassengerHelperAgent[0x4f2ae5]
PassengerHelperAgent[0x4e8d57]
PassengerHelperAgent[0x49f78a]
PassengerHelperAgent[0x44788e]
PassengerHelperAgent[0x501192]
PassengerHelperAgent[0x4f83a4]
PassengerHelperAgent[0x4ef78f]
PassengerHelperAgent[0x4e490e]
PassengerHelperAgent[0x5a3627]
PassengerHelperAgent[0x50cd44]
PassengerHelperAgent[0x50c66f]
PassengerHelperAgent[0x50b9aa]
PassengerHelperAgent[0x59ece3]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x8182)[0x7f44e3816182]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f44e2b1afbd]
--------------------------------------
[ pid=243 ] Dumping additional diagnostical information...
--------------------------------------
### Request handler state
1 clients:
  Client 20:
    host                        = localhost
    uri                         = /
    connected at                = 2014-12-19 19:00:33 (4 sec ago)
    state                       = CHECKING_OUT_SESSION
    session                     = NULL
    requestBodyIsBuffered       = false
    requestIsChunked            = false
    requestBodyLength           = 0
    requestBodyAlreadyRead      = 0
    responseContentLength       = -1
    responseBodyAlreadyRead     = 0
    clientInput                 = 0x7f44c80014b0 fd=20, state=LIVE, buffer(0)="", paused=1, socketPaused=1, nextTickInstalled=0, generation=2, error=0
    clientInput started         = false
    clientBodyBuffer started    = false
    clientBodyBuffer reachedEnd = false
    clientOutputPipe started    = true
    clientOutputPipe reachedEnd = false
    clientOutputWatcher active  = false
    appInput                    = 0x7f44c80037c0 fd=-1, state=LIVE, buffer(0)="", paused=1, socketPaused=1, nextTickInstalled=0, generation=1, error=0
    appInput started            = false
    appInput reachedEnd         = false
    responseHeaderSeen          = false
    useUnionStation             = false

### Pool state (simple)
----------- General information -----------
Max pool size : 6
Processes     : 0
Requests in top-level queue : 0

----------- Application groups -----------
/home/app/loewy-api#default:
  App root: /home/app/loewy-api
  (spawning 1 new process...)
  Requests in queue: 1


### Pool state (XML)
<?xml version="1.0" encoding="iso8859-1" ?>
<info version="3"><passenger_version>4.0.53</passenger_version><process_count>0</process_count><max>6</max><capacity_used>1</capacity_used><get_wait_list_size>0</get_wait_list_size><get_wait_list></get_wait_list><supergroups><supergroup><name>/home/app/loewy-api</name><state>READY</state><get_wait_list_size>0</get_wait_list_size><capacity_used>1</capacity_used><secret>VlE9DaTiASPVMkpfCa9X8aDS6r0nTY3WxJ8dd2ENHzz</secret><group default="true"><name>/home/app/loewy-api&#35;default</name><component_name>default</component_name><app_root>/home/app/loewy-api</app_root><app_type>rack</app_type><environment>production</environment><uuid>nZTAEmSYZBjRKPz2C36H</uuid><enabled_process_count>0</enabled_process_count><disabling_process_count>0</disabling_process_count><disabled_process_count>0</disabled_process_count><capacity_used>1</capacity_used><get_wait_list_size>1</get_wait_list_size><disable_wait_list_size>0</disable_wait_list_size><processes_being_spawned>1</processes_being_spawned><spawning/><secret>I4SIw4qdWHKrLa4pIyGBIMRAbq5Sn81HEf54jQICNcL</secret><life_status>ALIVE</life_status><options><app_root>/home/app/loewy-api</app_root><app_group_name>/home/app/loewy-api</app_group_name><app_type>rack</app_type><start_command>/usr/bin/ruby2.1&#9;/usr/share/passenger/helper-scripts/rack-loader.rb</start_command><startup_file>config.ru</startup_file><process_title>Passenger RackApp</process_title><log_level>0</log_level><start_timeout>90000</start_timeout><environment>production</environment><base_uri>/</base_uri><spawn_method>smart</spawn_method><user>app</user><default_user>nobody</default_user><default_group>nogroup</default_group><ruby>/usr/bin/ruby2.1</ruby><python>python</python><nodejs>node</nodejs><logging_agent_address>unix:/tmp/passenger.1.0.113/generation-0/logging</logging_agent_address><logging_agent_username>logging</logging_agent_username><logging_agent_password>UjBbQcjkTIaA7tYNbz5mDgNDU12eJbmdfG1SclqkNEMEFhaf7Wm2DV4cscmBW2H1</logging_agent_password><debugger>false</debugger><analytics>false</analytics><group_secret>I4SIw4qdWHKrLa4pIyGBIMRAbq5Sn81HEf54jQICNcL</group_secret><min_processes>1</min_processes><max_processes>0</max_processes><max_preloader_idle_time>-1</max_preloader_idle_time><max_out_of_band_work_instances>1</max_out_of_band_work_instances></options><processes></processes></group></supergroup></supergroups></info>

### Backtraces
Thread 'Main thread' (0x7f44e3e4a780, LWP 243):
     in 'void Server::mainLoop()' (Main.cpp:571)

Thread 'MultiLibeio dispatcher' (0x7f44e3e48700, LWP 243):
     (empty)

Thread 'Pool garbage collector' (0x7f44e3df6700, LWP 243):
     in 'static void Passenger::ApplicationPool2::Pool::garbageCollect(Passenger::ApplicationPool2::PoolPtr)' (Pool.h:623)

Thread 'Pool analytics collector' (0x7f44e3e37700, LWP 243):
     in 'static void Passenger::ApplicationPool2::Pool::collectAnalytics(Passenger::ApplicationPool2::PoolPtr)' (Pool.h:754)

Thread 'Pool event loop' (0x7f44e221e700, LWP 243):
     (empty)

Thread 'MessageServer thread' (0x7f44e3db5700, LWP 243):
     in 'void Passenger::MessageServer::mainLoop()' (MessageServer.h:580)

Thread 'Request event loop' (0x7f44e1a1d700, LWP 243):
     (empty)

Thread 'Group process spawner: /home/app/loewy-api#default' (0x7f44e3d53700, LWP 260):
     in 'void Passenger::ApplicationPool2::processAndLogNewSpawnException(Passenger::SpawnException&, const Passenger::ApplicationPool2::Options&, const SpawnerConfigPtr&)' (Implementation.cpp:153)
     in 'void Passenger::ApplicationPool2::Spawner::throwAppSpawnException(const string&, Passenger::SpawnException::ErrorKind, Passenger::ApplicationPool2::Spawner::NegotiationDetails&)' (Spawner.h:687)
     in 'Passenger::ApplicationPool2::SmartSpawner::SpawnResult Passenger::ApplicationPool2::SmartSpawner::sendSpawnCommand(const Passenger::ApplicationPool2::Options&)' (SmartSpawner.h:678)
     in 'virtual Passenger::ApplicationPool2::ProcessPtr Passenger::ApplicationPool2::SmartSpawner::spawn(const Passenger::ApplicationPool2::Options&)' (SmartSpawner.h:747)
     in 'void Passenger::ApplicationPool2::Group::spawnThreadRealMain(const SpawnerPtr&, const Passenger::ApplicationPool2::Options&, unsigned int)' (Implementation.cpp:977)

Thread 'PipeWatcher: PID 262 stderr, fd 23' (0x7f44e3d01700, LWP 261):
     in 'void Passenger::ApplicationPool2::PipeWatcher::threadMain()' (Implementation.cpp:1539)
     in 'static void Passenger::ApplicationPool2::PipeWatcher::threadMain(boost::shared_ptr<Passenger::ApplicationPool2::PipeWatcher>)' (Implementation.cpp:1520)

Thread 'PipeWatcher: PID 262 stdout, fd 22' (0x7f44e3d94700, LWP 261):
     in 'void Passenger::ApplicationPool2::PipeWatcher::threadMain()' (Implementation.cpp:1539)
     in 'static void Passenger::ApplicationPool2::PipeWatcher::threadMain(boost::shared_ptr<Passenger::ApplicationPool2::PipeWatcher>)' (Implementation.cpp:1520)

--------------------------------------
[ pid=243 ] Open files and file descriptors:
ERROR: cannot execute command 'lsof': errno=2
Falling back to another mechanism for dumping file descriptors.
ls: cannot read symbolic link /proc/243/fd/0: Permission denied
ls: cannot read symbolic link /proc/243/fd/1: Permission denied
ls: cannot read symbolic link /proc/243/fd/2: Permission denied
ls: cannot read symbolic link /proc/243/fd/3: Permission denied
ls: cannot read symbolic link /proc/243/fd/5: Permission denied
ls: cannot read symbolic link /proc/243/fd/8: Permission denied
ls: cannot read symbolic link /proc/243/fd/9: Permission denied
ls: cannot read symbolic link /proc/243/fd/10: Permission denied
ls: cannot read symbolic link /proc/243/fd/11: Permission denied
ls: cannot read symbolic link /proc/243/fd/12: Permission denied
ls: cannot read symbolic link /proc/243/fd/13: Permission denied
ls: cannot read symbolic link /proc/243/fd/14: Permission denied
ls: cannot read symbolic link /proc/243/fd/15: Permission denied
ls: cannot read symbolic link /proc/243/fd/16: Permission denied
ls: cannot read symbolic link /proc/243/fd/17: Permission denied
ls: cannot read symbolic link /proc/243/fd/18: Permission denied
ls: cannot read symbolic link /proc/243/fd/20: Permission denied
ls: cannot read symbolic link /proc/243/fd/21: Permission denied
ls: cannot read symbolic link /proc/243/fd/22: Permission denied
ls: cannot read symbolic link /proc/243/fd/23: Permission denied
total 0
lr-x------ 1 root root 64 Dec 19 19:00 0
l-wx------ 1 root root 64 Dec 19 19:00 1
l-wx------ 1 root root 64 Dec 19 19:00 2
lrwx------ 1 root root 64 Dec 19 19:00 3
l-wx------ 1 root root 64 Dec 19 19:00 5
lrwx------ 1 root root 64 Dec 19 19:00 8
lrwx------ 1 root root 64 Dec 19 19:00 9
lrwx------ 1 root root 64 Dec 19 19:00 10
lrwx------ 1 root root 64 Dec 19 19:00 11
lrwx------ 1 root root 64 Dec 19 19:00 12
lr-x------ 1 root root 64 Dec 19 19:00 13
l-wx------ 1 root root 64 Dec 19 19:00 14
lrwx------ 1 root root 64 Dec 19 19:00 15
lrwx------ 1 root root 64 Dec 19 19:00 16
lr-x------ 1 root root 64 Dec 19 19:00 17
lr-x------ 1 root root 64 Dec 19 19:00 18
lrwx------ 1 root root 64 Dec 19 19:00 20
lrwx------ 1 root root 64 Dec 19 19:00 21
lrwx------ 1 root root 64 Dec 19 19:00 22
lr-x------ 1 root root 64 Dec 19 19:00 23
ERROR: Could not run 'ls' to dump file descriptor information!
--------------------------------------
[ pid=243 ] Dumping a more detailed backtrace with crash-watch...
Found gdb at: /usr/bin/gdb
ERROR: Cannot attach to process.

This may be the result of kernel ptrace() hardening. Try disabling it with:
  sudo sh -c 'echo 0 > /proc/sys/kernel/yama/ptrace_scope'
See http://askubuntu.com/questions/41629/after-upgrade-gdb-wont-attach-to-process for more information.
2014/12/19 19:00:38 [error] 146#0: *3 upstream prematurely closed connection while reading response header from upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "passenger:/tmp/passenger.1.0.113/generation-0/request:", host: "localhost"
[ 2014-12-19 19:00:38.5765 128/7f106bd03700 agents/Watchdog/AgentWatcher.cpp:96 ]: Phusion Passenger helper agent (pid=243) crashed with signal SIGSEGV, restarting it...
[ 2014-12-19 19:00:38.5912 300/7f2b307df780 agents/HelperAgent/Main.cpp:650 ]: PassengerHelperAgent online, listening at unix:/tmp/passenger.1.0.113/generation-0/request