Skip to content

Instantly share code, notes, and snippets.

@Stray
Created December 4, 2010 16:46
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Stray/728312 to your computer and use it in GitHub Desktop.
Save Stray/728312 to your computer and use it in GitHub Desktop.
How should I go about this in asunit 4?
I'm growing an end-to-end test which has more steps to it than my
usual end-to-end tests.
As a result I'd like to split my test-stages into separate test cases
(or something similar - could be helper classes of some kind)
and run them in a specific order.
The test cases follow on from each other - that is, they pick the
application up in the state it was in at the end of the previous test,
and then move it forward and verify that everything is working up to that point.
I tried implementing this in asunit3 but I've found that
beyond the first test case the tests don't really run.
If I switch to asunit 4 can I inject the same instance into each
test case, and can I control the order the test cases run in,
other than alphabetically?
The code below is what I've set up in asunit3. The child test case
run first (the one added in line 21) does run properly. The one added
afterwards doesn't. Obviously I *could* just not null out the RobotEyes
instance, as it is actually static... but that feels a bit gross.
Thanks!
Stray
package strategy.xendtoendtests {
// imports removed for brevity
public class PyramidGameEndToEndTest extends TestCase {
private var robotEyes:RobotEyes;
private var config:IGameConfig;
private var endToEndTests:Vector.<Class>;
private var runningEndToEndTests:Boolean;
private var dummyDispatcher:EventDispatcher = new EventDispatcher();
public function PyramidGameEndToEndTest (methodName:String=null) {
super(methodName)
}
private function appendTests():void
{
// each of these is a test case in itself, but it should report
// its results back into this test
endToEndTests = new Vector.<Class>();
endToEndTests.push( StartingConditions );
endToEndTests.push( FirstStoneSupplyOffer );
// and further stages of the end-to-end test,
}
override public function run():void{
config = new FirstGameConfig();
robotEyes = new RobotEyes(PyramidGame);
addChild(robotEyes);
robotEyes.visible = false;
appendTests();
// need to wait a while
var timer:Timer = new Timer(1000,1);
timer.addEventListener(TimerEvent.TIMER, timerHandler);
timer.start();
}
private function timerHandler(e:TimerEvent):void{
robotEyes.visible = true;
super.run();
}
override protected function cleanUp():void{
removeChild(robotEyes);
robotEyes = null;
}
public function testEndToEnd():void {
// this async is only to keep the test alive while
// all the child testcases run
var handler:Function = addAsync(application_tests_complete, 20000);
dummyDispatcher.addEventListener(Event.COMPLETE, handler);
runNextEndToEndTest();
}
private function prepareTest(testClass:Class):TestCase {
var nextTest:TestCase = new testClass() as TestCase;
nextTest['onTestComplete'] = runNextEndToEndTest;
nextTest['config'] = config;
nextTest['robotEyes'] = robotEyes;
return nextTest;
}
private function runNextEndToEndTest():void
{
if(endToEndTests.length > 0)
{
var nextTest:TestCase = prepareTest(endToEndTests.shift());
getResult().run(nextTest);
}
else
{
dummyDispatcher.dispatchEvent(new Event(Event.COMPLETE));
}
}
private function application_tests_complete(e:Event = null):void
{
// just to make the async keep the test alive
}
}
}
@robertpenner
Copy link

As you know, unit test cases traditionally are isolated from each other. You could make a custom runner to support your style. I envision the runner taking an ordered list of test classes and injecting into each. But it may be easier to maintain the integration test state in one test case, which is what I think you're doing here.

The difficulty you're running into is in running tests manually from within tests. In AsUnit 3, the TestCase runs itself (ick). I highly recommend moving to AsUnit 4 (use my repo). It has separate classes TestRunner and SuiteRunner. I think you'll find it easier to create a TestRunner inside your test and use it to run your sub-tests.

AsUnit 4 can run AsUnit 3 TestCases using LegacyRunner. Check out:
https://github.com/robertpenner/asunit/blob/master/asunit-4.0/src/asunit/framework/TestCase.as

Luke amazed me with this clever way to hook the old test style into the new framework.

Also, AsUnit 4 uses SwiftSuspenders for injection, as can be seen in AsUnitCore and the runners.

@Stray
Copy link
Author

Stray commented Dec 4, 2010

Thanks for that - it's pretty much as I suspected then - not really viable in asunit3 and time to move to asunit 4 :)

For the moment I've got the ugly static version working by (ick) prefixing the consecutive cases with A, B, C etc so that when sprouts re-builds the test suite they don't get rearranged. How gross is that?

Kristopher Joseph was pretty interested in how asunit 4 could do end-to-end testing so I might hit him as well to see what ideas he's got based on what you've said here and see if I can do something that can be a viable workflow rather than a hacky hack.

There's also Cucumber / Melomel kicking about. My motivation with robotEyes was to use AS3 to make it less daunting for flash devs - not having to also learn / install / update Ruby to get their end-to-end tests working.

I'll have a play with the new TestRunner then! If it also supports Legacy then it sounds like there's nothing to stand in the way.

Help much appreciated! Cheers, Stray

@Stray
Copy link
Author

Stray commented Dec 4, 2010

Any other folk with input - fire away.

@benbjohnson
Copy link

I would agree with Robert that the test cases should be isolated if possible so that they don't cause each other to fail. That being said, if your tests are built one on top of the next then it makes sense that the first one failing should cause the others to fail.

I can understand not wanting to introduce complexity into the environment for other Flash devs with Cucumber+Melomel but I find end-to-end UI testing using unit testing frameworks to be extremely difficult to read and maintain.

Let me know if you want a hand implementing Melomel & Cucumber on your project. It's been used on Flex projects but I haven't tried it on an AS3-only project yet. There's no Flex dependencies so it should work fine. I'm in the process of doing a screencast for Melomel but Camtasia hates me.

@Stray
Copy link
Author

Stray commented Dec 4, 2010

Thanks Ben - I shall give cucumber+melomel a spin tomorrow - so brace yourself for a bunch of questions!

They should cause each other to fail - I want it to fail at the first point at which it's not working - the next test would make no sense without the previous test completing.

The idea is that robotEyes makes it relatively simple to end-to-end test apps through asunit. So, until now I've found it to be fine.

The difference is that this is a game - I haven't had any problems end-to-end testing my more 'app' like apps, but this feels like it'll end up being a monster 3000 line test if it doesn't get broken up :)

Good luck with Camtasia... I find it helps not to take these things personally.

@benbjohnson
Copy link

Send any questions you have to me on Twitter.

I took a look at RobotEyes and it looks cool. I thought you said RobotLegs and I couldn't figure out how you used that library to end-to-end test. :) RE looks a lot easier to UI test in ActionScript than hand coding in AsUnit or FlexUnit. I've done that before and it's brutal. That's why I wrote an interface to Cucumber.

Melomel is a really simple remote interface to the Flash VM so you can still use RobotEyes in your Cucumber step definitions. I have some built-in utility methods for finding components but it looks like RobotEyes has more fine-grain control and is more advanced.

Is the game open source or is it a private project? If it's open, let me know where the repo is and I'll take a look.

The documentation on http://melomel.info should be up to date but if it isn't, feel free to update it via the Melomel GitHub wiki. The web site is just a mirror using Smeagol.

@Stray
Copy link
Author

Stray commented Dec 5, 2010

Excellent - thanks Ben! I was just checking out Melomel + Cucumber and I think at the moment it's not fine-grained enough for me on the non-flex side. Also, with RobotEyes is that you can listen for events to use as cues, which I have found to be pretty useful when you're picking up async stuff etc where timings are variable. AS3 doesn't have the busy cursor (afaik).

It's open source - repo is at: https://github.com/Stray/RobotEyes

I've added some more functionality since I last pushed - the nice thing about it being actionscript is that it's very, very simple to extend as your tests require something new. I'll add all my changes to the repo today.

Oddly the broken-down-tests thing is actually working really nicely now in asunit 3 - just using a really primitive approach of having a static instance of the application running in the RobotEyes master class, and using alphabetical naming to keep my tests in order. Crude but it's not getting in my way right now, so I can focus on the detail of what I'm testing.

So - as these tests are part of an open-source 'TDD a strategy game using robotlegs' thing, I'll do it all in this slightly dirty way, and then refactor to Cucumber / Melomel + RobotEyes afterwards because that'll be a really good learning path I think, and will produce something useful in terms of a complete "how" in asunit and then in Cucumber+Melomel as well.

It also potentially highlights where you'd want to switch from in-suite end-to-end testing to external end-to-end testing. If there are pain points then I'm sure I'll hit them :)

@benbjohnson
Copy link

I saw the RobotEyes repo. Is your game project's source available? Or is it private?

@Stray
Copy link
Author

Stray commented Dec 5, 2010

Oh! Sorry ... duh :)

Yes - game repo is at https://github.com/Stray/robotlegs-demo-StrategyGame

In the interests of full disclosure:

  • It's a project for a client. We provide their e-learning platform and content and it's an immersive learning exercise in project management that sits within an "introduction to project management" lesson.
  • But the client is pro open-source so when we develop something generic they are happy for us to share it.

So - dig in!

@newtriks
Copy link

I have (as Stray knows) just ported RobotEyes for use in Flex and written an end to end test case using ASUnit4. Keep an eye here for example link: https://github.com/newtriks/RobotEyes/tree/flex

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment