[Scummvm-devel] Thought about unit tests (automatic game testing)

[Rafał Rzepecki]

Dear colleagues,

Recent messages on the list got me thinking about automatic unit tests of the 
engines. The biggest problem currently seems that it's hard to test all the 
games with every release (being so huge amount of them) as this all has to be 
done manually and that makes the engines prone to regress silently.

I think it may be possible to make a framework to at least in part 
automatically test games. Technically this should not be that difficult, so 
let me focus on the API I imagine it would take.

To make it all work, necessary changes to the engines should be as minimal as 
possible -- and indeed what I feel would be necessary is just replacing 
whatever frame delay code there is with conditionally not delaying, but just 
bumping frame counter.

This would make it possible to create such test functions as (it remains open 
to debate on whether they should be C++-based, or some script language (I'd be 
more for the latter)):
moveMouse(framenum, position);
clickMouse(framenum, position, button);
pressKey(framenum, key);
compareImage(framenum, image);
...
and I feel that would be essentially it. It should also be pretty 
straightforward to create (record) such scripts automatically.

Having such a framework in place would make it possible to trivially test 
engine code (at least for common regressions) even on different platforms 
(perhaps conditional on game data availability by porters).

Let me know what you think.
-- 
- Rafał Rzepecki