Andy Thomason is a Senior Programmer at Genomics PLC. He has been witing graphics systems, games and compilers since the '70s and specialises in code performance.
Add the `replication` section to the mongod.conf file: | |
``` | |
$cat /usr/local/etc/mongod.conf | |
systemLog: | |
destination: file | |
path: /usr/local/var/log/mongodb/mongo.log | |
logAppend: true | |
storage: | |
engine: mmapv1 |
//events - a super-basic Javascript (publish subscribe) pattern | |
var events = { | |
events: {}, | |
on: function (eventName, fn) { | |
this.events[eventName] = this.events[eventName] || []; | |
this.events[eventName].push(fn); | |
}, | |
off: function(eventName, fn) { | |
if (this.events[eventName]) { |
@mafintosh asks: "Does anyone have a good code example of when to use setImmediate instead of nextTick?"
https://twitter.com/mafintosh/status/624590818125352960
The answer is "generally anywhere outside of core".
process.nextTick
is barely asynchronous. Flow-wise it is asynchronous, but it will trigger before any other asynchronous events can (timers, io, etc.) and thus can starve the event loop.
In this script I show a starved event loop where I just synchronously block, use nextTick
and setImmediate
############################## | |
## Java | |
############################## | |
.mtj.tmp/ | |
*.class | |
*.jar | |
*.war | |
*.ear | |
*.nar | |
hs_err_pid* |
// Restify Server CheatSheet. | |
// More about the API: http://mcavage.me/node-restify/#server-api | |
// Install restify with npm install restify | |
// 1.1. Creating a Server. | |
// http://mcavage.me/node-restify/#Creating-a-Server | |
var restify = require('restify'); |
Moved to git repository: https://github.com/denji/nginx-tuning
For this configuration you can use web server you like, i decided, because i work mostly with it to use nginx.
Generally, properly configured nginx can handle up to 400K to 500K requests per second (clustered), most what i saw is 50K to 80K (non-clustered) requests per second and 30% CPU load, course, this was 2 x Intel Xeon
with HyperThreading enabled, but it can work without problem on slower machines.
You must understand that this config is used in testing environment and not in production so you will need to find a way to implement most of those features best possible for your servers.
There are a lot of ways to serve a Go HTTP application. The best choices depend on each use case. Currently nginx looks to be the standard web server for every new project even though there are other great web servers as well. However, how much is the overhead of serving a Go application behind an nginx server? Do we need some nginx features (vhosts, load balancing, cache, etc) or can you serve directly from Go? If you need nginx, what is the fastest connection mechanism? This are the kind of questions I'm intended to answer here. The purpose of this benchmark is not to tell that Go is faster or slower than nginx. That would be stupid.
So, these are the different settings we are going to compare:
- Go HTTP standalone (as the control group)
- Nginx proxy to Go HTTP
- Nginx fastcgi to Go TCP FastCGI
- Nginx fastcgi to Go Unix Socket FastCGI
If you want, I can try and help with pointers as to how to improve the indexing speed you get. Its quite easy to really increase it by using some simple guidelines, for example: | |
- Use create in the index API (assuming you can). | |
- Relax the real time aspect from 1 second to something a bit higher (index.engine.robin.refresh_interval). | |
- Increase the indexing buffer size (indices.memory.index_buffer_size), it defaults to the value 10% which is 10% of the heap. | |
- Increase the number of dirty operations that trigger automatic flush (so the translog won't get really big, even though its FS based) by setting index.translog.flush_threshold (defaults to 5000). | |
- Increase the memory allocated to elasticsearch node. By default its 1g. | |
- Start with a lower replica count (even 0), and then once the bulk loading is done, increate it to the value you want it to be using the update_settings API. This will improve things as possibly less shards will be allocated to each machine. | |
- Increase the number of machines you have so |