kinetoscope ~: free total used free shared buffers cached Mem: 2048 1955 92 0 16 68 -/+ buffers/cache: 1870 177 Swap: 4095 85 4010 kinetoscope ~: top top - 12:12:29 up 2 days, 22:59, 2 users, load average: 10.00, 10.00, 10.00 Tasks: 95 total, 11 running, 78 sleeping, 0 stopped, 6 zombie Cpu(s): 29.3%us, 7.7%sy, 0.0%ni, 52.0%id, 0.2%wa, 0.0%hi, 0.0%si, 10.9%st Mem: 2097372k total, 2002740k used, 94632k free, 17264k buffers Swap: 4194296k total, 87796k used, 4106500k free, 69696k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 25156 www 20 0 309m 159m 2912 R 43 7.8 1637:21 ruby1.8 6937 www 20 0 311m 166m 2864 R 41 8.1 202:38.94 ruby1.8 10601 www 20 0 309m 157m 2816 R 41 7.7 1012:12 ruby1.8 5767 www 20 0 320m 177m 3160 R 39 8.7 214:45.31 ruby1.8 10884 www 20 0 304m 156m 1984 R 39 7.7 1006:24 ruby1.8 9353 www 20 0 514m 376m 4488 R 37 18.4 134:30.86 ruby1.8 10610 www 20 0 311m 173m 2768 R 37 8.5 135:01.54 ruby1.8 8777 www 20 0 344m 165m 4252 R 35 8.1 1020:31 ruby1.8 25564 www 20 0 310m 158m 3164 R 35 7.8 1631:51 ruby1.8 6624 www 20 0 311m 166m 2844 R 33 8.1 207:55.56 ruby1.8 1 root 20 0 4104 624 544 S 0 0.0 0:00.40 init 2 root 15 -5 0 0 0 S 0 0.0 0:00.00 kthreadd 3 root RT -5 0 0 0 S 0 0.0 0:01.38 migration/0 4 root 15 -5 0 0 0 S 0 0.0 0:00.48 ksoftirqd/0 5 root RT -5 0 0 0 S 0 0.0 0:00.62 watchdog/0 6 root 15 -5 0 0 0 S 0 0.0 0:23.08 events/0 7 root 15 -5 0 0 0 S 0 0.0 0:00.00 khelper 18 root 15 -5 0 0 0 S 0 0.0 0:00.00 xenwatch 19 root 15 -5 0 0 0 S 0 0.0 0:00.00 xenbus 29 root RT -5 0 0 0 S 0 0.0 0:00.56 migration/1 30 root 15 -5 0 0 0 S 0 0.0 0:00.54 ksoftirqd/1 31 root RT -5 0 0 0 S 0 0.0 0:00.64 watchdog/1 32 root 15 -5 0 0 0 S 0 0.0 0:19.32 events/1 33 root RT -5 0 0 0 S 0 0.0 0:00.58 migration/2 34 root 15 -5 0 0 0 S 0 0.0 0:00.42 ksoftirqd/2 35 root RT -5 0 0 0 S 0 0.0 0:00.62 watchdog/2 36 root 15 -5 0 0 0 S 0 0.0 0:18.76 events/2 37 root RT -5 0 0 0 S 0 0.0 0:00.60 migration/3 38 root 15 -5 0 0 0 S 0 0.0 0:00.38 ksoftirqd/3 39 root RT -5 0 0 0 S 0 0.0 0:00.62 watchdog/3 40 root 15 -5 0 0 0 S 0 0.0 0:19.14 events/3 66 root 15 -5 0 0 0 S 0 0.0 0:00.14 kblockd/0 67 root 15 -5 0 0 0 S 0 0.0 0:00.02 kblockd/1 68 root 15 -5 0 0 0 S 0 0.0 0:00.04 kblockd/2 69 root 15 -5 0 0 0 S 0 0.0 0:00.06 kblockd/3 79 root 15 -5 0 0 0 S 0 0.0 0:00.00 kseriod 128 root 15 -5 0 0 0 S 0 0.0 0:04.96 kswapd0 129 root 15 -5 0 0 0 S 0 0.0 0:00.00 aio/0 130 root 15 -5 0 0 0 S 0 0.0 0:00.00 aio/1 131 root 15 -5 0 0 0 S 0 0.0 0:00.00 aio/2 132 root 15 -5 0 0 0 S 0 0.0 0:00.00 aio/3 210 root 15 -5 0 0 0 S 0 0.0 0:00.00 accel_watch/0 211 root 15 -5 0 0 0 S 0 0.0 0:00.00 accel_watch/1 212 root 15 -5 0 0 0 S 0 0.0 0:00.00 accel_watch/2 213 root 15 -5 0 0 0 S 0 0.0 0:00.00 accel_watch/3 kinetoscope ~: sudo passenger-status [sudo] password for www: ----------- Backtraces ----------- Thread 'Main thread': in 'void Server::start()' (HelperServer.cpp:754) in 'int main(int, char**)' (HelperServer.cpp:815) Thread 'Status report thread': in 'void Passenger::ApplicationPoolStatusReporter::threadMain()' (ApplicationPoolStatusReporter.h:100) Thread 'Client thread 1': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 2': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 3': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 4': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 5': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 6': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 7': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 8': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 9': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 10': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 11': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 12': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 13': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 14': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 15': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 16': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 17': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 18': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 19': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 20': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) ----------- General information ----------- max = 5 count = 5 active = 5 inactive = 0 Waiting on global queue: 0 ----------- Domains ----------- /var/www/edisonnation.com/production/current: PID: 6624 Sessions: 1 Processed: 55 Uptime: 8h 38m 51s PID: 6937 Sessions: 1 Processed: 52 Uptime: 8h 25m 23s PID: 5767 Sessions: 2 Processed: 179 Uptime: 9h 6m 51s PID: 10610 Sessions: 1 Processed: 14 Uptime: 5h 55m 15s PID: 9353 Sessions: 1 Processed: 422 Uptime: 6h 48m 14s kinetoscope ~: sudo passenger-memory-stats ------------- Apache processes ------------- *** WARNING: The Apache executable cannot be found. Please set the APXS2 environment variable to your 'apxs2' executable's filename, or set the HTTPD environment variable to your 'httpd' or 'apache2' executable's filename. ------------- Nginx processes -------------- PID PPID Threads VMSize Private Name -------------------------------------------- 6640 1 1 26.8 MB ? nginx: master process /etc/nginx/sbin/nginx 6641 6640 1 27.8 MB 0.7 MB nginx: worker process 6644 6640 1 27.8 MB 0.7 MB nginx: worker process 6648 6640 1 27.8 MB 0.7 MB nginx: worker process 6650 6640 1 27.7 MB 0.7 MB nginx: worker process ### Processes: 5 ### Total private dirty RSS: 2.86 MB (?) ---------- Passenger processes ---------- PID Threads VMSize Private Name ----------------------------------------- 5767 1 320.7 MB 151.8 MB Rails: /var/www/edisonnation.com/production/current 6620 23 89.2 MB 0.6 MB PassengerNginxHelperServer /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2 /usr/bin/ruby1.8 3 4 0 5 0 300 1 nobody 65534 65534 /tmp/passenger.6617 6624 1 311.2 MB 140.5 MB Rails: /var/www/edisonnation.com/production/current 6637 2 56.3 MB 3.9 MB Passenger spawn server 6937 1 311.1 MB 141.2 MB Rails: /var/www/edisonnation.com/production/current 8777 4 344.4 MB 161.2 MB Rails: /var/www/edisonnation.com/production/current 9353 1 514.9 MB 372.2 MB Rails: /var/www/edisonnation.com/production/current 10601 1 309.3 MB 130.7 MB Rails: /var/www/edisonnation.com/production/current 10610 1 311.5 MB 170.3 MB Rails: /var/www/edisonnation.com/production/current 10884 1 304.6 MB 130.6 MB Rails: /var/www/edisonnation.com/production/current 25156 1 310.0 MB 135.1 MB Rails: /var/www/edisonnation.com/production/current 25564 1 310.0 MB 134.2 MB Rails: /var/www/edisonnation.com/production/current ### Processes: 12 ### Total private dirty RSS: 1672.07 MB ################################################## ## Instances stay around even after stopping nginx ################################################## lightbulb ~: sudo passenger-status [sudo] password for www: ----------- Backtraces ----------- Thread 'Main thread': in 'void Server::start()' (HelperServer.cpp:754) in 'int main(int, char**)' (HelperServer.cpp:815) Thread 'Status report thread': in 'void Passenger::ApplicationPoolStatusReporter::threadMain()' (ApplicationPoolStatusReporter.h:100) Thread 'Client thread 1': in 'FileDescriptor Client::acceptConnection()' (HelperServer.cpp:171) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 2': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 3': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 4': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 5': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 6': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 7': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 8': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 9': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 10': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 11': in 'FileDescriptor Client::acceptConnection()' (HelperServer.cpp:171) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 12': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 13': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 14': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 15': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 16': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 17': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 18': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 19': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) Thread 'Client thread 20': in 'void Client::forwardResponse(boost::shared_ptr&, FileDescriptor&)' (HelperServer.cpp:329) in 'bool Client::handleRequest(FileDescriptor&)' (HelperServer.cpp:438) in 'void Client::threadMain()' (HelperServer.cpp:484) ----------- General information ----------- max = 5 count = 5 active = 3 inactive = 2 Waiting on global queue: 0 ----------- Domains ----------- /var/www/edisonnation.com/production/current: PID: 14240 Sessions: 0 Processed: 248 Uptime: 57m 12s PID: 14879 Sessions: 0 Processed: 179 Uptime: 28m 11s PID: 8102 Sessions: 1 Processed: 71 Uptime: 6h 25m 35s PID: 8737 Sessions: 1 Processed: 2 Uptime: 6h 3m 0s PID: 8326 Sessions: 2 Processed: 16 Uptime: 6h 15m 30s lightbulb ~: sudo passenger-memory-stats ------------- Apache processes ------------- *** WARNING: The Apache executable cannot be found. Please set the APXS2 environment variable to your 'apxs2' executable's filename, or set the HTTPD environment variable to your 'httpd' or 'apache2' executable's filename. ------------- Nginx processes -------------- PID PPID Threads VMSize Private Name -------------------------------------------- 4928 1 1 26.8 MB ? nginx: master process /etc/nginx/sbin/nginx 4929 4928 1 27.8 MB 0.6 MB nginx: worker process 4932 4928 1 27.8 MB 0.6 MB nginx: worker process 4936 4928 1 27.7 MB 0.6 MB nginx: worker process 4938 4928 1 27.7 MB 0.6 MB nginx: worker process ### Processes: 5 ### Total private dirty RSS: 2.53 MB (?) ---------- Passenger processes ---------- PID Threads VMSize Private Name ----------------------------------------- 4908 23 89.2 MB 0.3 MB PassengerNginxHelperServer /usr/lib/ruby/gems/1.8/gems/passenger-2.2.2 /usr/bin/ruby1.8 3 4 0 5 0 300 1 nobody 65534 65534 /tmp/passenger.4904 4925 2 56.2 MB 1.3 MB Passenger spawn server 6523 1 311.2 MB 155.6 MB Rails: /var/www/edisonnation.com/production/current 8027 1 309.7 MB 131.4 MB Rails: /var/www/edisonnation.com/production/current 8102 1 326.1 MB 146.3 MB Rails: /var/www/edisonnation.com/production/current 8127 1 309.7 MB 131.2 MB Rails: /var/www/edisonnation.com/production/current 8326 1 310.9 MB 131.1 MB Rails: /var/www/edisonnation.com/production/current 8737 1 309.6 MB 130.3 MB Rails: /var/www/edisonnation.com/production/current 14240 1 310.2 MB 169.5 MB Rails: /var/www/edisonnation.com/production/current 14870 1 262.1 MB 96.2 MB Passenger ApplicationSpawner: /var/www/edisonnation.com/production/current 14879 1 313.6 MB 143.5 MB Rails: /var/www/edisonnation.com/production/current 22928 1 310.1 MB 156.5 MB Rails: /var/www/edisonnation.com/production/current 23359 1 311.9 MB 156.1 MB Rails: /var/www/edisonnation.com/production/current ### Processes: 13 ### Total private dirty RSS: 1549.23 MB lightbulb ~: sudo /etc/init.d/nginx stop * Stopping Nginx Server... ...done. lightbulb ~: sudo passenger-memory-stats ------------- Apache processes ------------- *** WARNING: The Apache executable cannot be found. Please set the APXS2 environment variable to your 'apxs2' executable's filename, or set the HTTPD environment variable to your 'httpd' or 'apache2' executable's filename. ------------ Nginx processes ------------- ### Processes: 0 ### Total private dirty RSS: 0.00 MB ---------- Passenger processes ---------- PID Threads VMSize Private Name ----------------------------------------- 6523 1 311.2 MB 155.6 MB Rails: /var/www/edisonnation.com/production/current 8027 1 309.7 MB 131.4 MB Rails: /var/www/edisonnation.com/production/current 8102 1 326.1 MB 146.3 MB Rails: /var/www/edisonnation.com/production/current 8127 1 309.7 MB 131.2 MB Rails: /var/www/edisonnation.com/production/current 8326 1 310.9 MB 131.1 MB Rails: /var/www/edisonnation.com/production/current 8737 1 309.6 MB 130.3 MB Rails: /var/www/edisonnation.com/production/current 22928 1 310.1 MB 156.5 MB Rails: /var/www/edisonnation.com/production/current 23359 1 311.9 MB 156.1 MB Rails: /var/www/edisonnation.com/production/current ### Processes: 8 ### Total private dirty RSS: 1138.52 MB lightbulb ~: sudo passenger-status ERROR: Phusion Passenger doesn't seem to be running. lightbulb ~: