perflib

gistfile1.txt

#Performance Testing Framework

Performance is a very important factor for JavaScript games. It becomes
a cruciual factor when you bring mobile devices into the mix.

Luckily we have some great tools for testing different methods of coding
such as JSPerf.com. This lets developers see which browsers perform best
doing what.

On the same vein, JSGameBench stress tested rendering methods for browsers
using WebGL, Canvas and DOM.

At Game Closure we want to look deeper and more thorough so I've been thinking
about different ways to design a framework that will allow us to run tests
with an assortment of different methods and combinations. With JSPerf one has
to write individual test cases for every combination. Here is my proposal:

##**Templates**

Code to test should have some templates for leg work, along the lines of
startup and teardown but finer grained.

##**Placeholders**

This is the key component to the power of this testing framework. Add
placeholders in the template code that will be replaced based on
the combinations being tested. The placeholders are denoted by a
line comment followed by a colon and the name of the placeholder.

	function render() {
		//:type
		//:x
		//:y
		//:end+type
	}
	
This looks like gibberish now but starts to make sense when you create
a list of different values for these placeholders:

	{
		type: [
			"DOM.css({", 
			"ctx.drawImage(cnv,", 
			"ctx.drawImage(img,"
		],
		
		x: [
			"this.x,", 
			"~~this.x,", 
			"this.x|0,"
		],
		
		y: [
			"this.y,", 
			"~~this.y,", 
			"this.y|0,"
		],
		
		end: [
			"});", 
			");", 
			");"
		]
	}
	
The test framework will use all different combinations of these values
testing to see which performs the best. You will notice a special
placeholder `end+type`. This a special placeholder that indicates that 
it should be in sync with the placeholder after the `+` symbol. This is
so we produce valid code and don't end up with a combination that will 
break.

##**Tests**

Whilst the templates setup what we are going to test, we need to know