[Scummvm-devel] Unit tests

[Jonathan Gray]

On Sun, 21 Dec 2003, Max Horn wrote:

> Yo folks,
>
> I would like to set up some unit tests for ScummVM. Right now, all I
> want to do is add tests for some things in common/ (in particular, the
> Map/List/String/Rect/Point classes).
>
> If you are not familiar with Unit tests, you may want to read up a bit
> on it, e.g. here:
> http://www.extremeprogramming.org/rules/unittests.html
> http://emw.inf.tu-dresden.de/de/pdai/Forschung/refactoring/
> refactoring_html/node7.html
>
> In short, having unit tests means that bugs may be found faster. In
> addition they can help "document" code -> if you are not sure how a
> certain method is meant to behave, you can take a look at the unit test
> to get an idea. And if we rewrite the Map class to use a red black
> tree, then any breakage introduced by this hopefully are caught by well
> written unit tests...
>
>
> I looked a bit around and examined various C++ unit test frameworks
> (see e.g. http://www.xprogramming.com/software.htm for an incomplete
> list). Right now I like "cxxtest" <http://cxxtest.sourceforge.net/>
> most for our purposes. It requires Perl or Python to run the tests, but
> that shouldn't be a problem - we don't have to run tests on every
> target system, after all. If you don't have perl/python, you can't do
> "make test", but that's the only drawback.
> Other than that, it doesn't need the C++ std lib in any way (thus
> making Ender happy), it's rather lightweight, and (I think) easy to use.

Although the argument against the C++ lib was portability to small
devices, I'd think that a series of tests wouldn't need to be run on
these devices themselves.  But then again there are the various compilers
by the folks at microsoft...

>
>
> If nobody objects, I would like to add a "test" directory to the
> scummvm dir, with the following layout:

Maybe call the directory "regress" or something similiar? Test is a bit
vague and something like regress implies regression tests/unit tests in my
opinion.

>
> test/
> |-- common (Contains unit tests for stuff found in the 'common'
> directory)
> |   |-- list.h
> |   |-- map.h
> |   `-- str.h
> `-- cxxtest
>      (All cxxtest 3.8.1 source files; I could also remove the
> examples/docs here, but I think it's useful to keep them)
>
> Also, a new "test" target in the Makefile which would compile and run
> the test suite when invoked.

See above comment on test vs regress.

>
> In the future, more tests could be added there, also including tests
> for gui/backends/sound, or even for the engines, if needed.

This sounds like a great idea, anything that helps find bugs quicker can
only be a good thing (well as long as it isn't creating bugs itself :)

It would be nice to be able to do unit tests using the game logic (ie byte
compiled scumm scripts for scumm) but obviously this requires additional
things that are currently not implemented for the most part (as fun as writing
 scripts by hand in hex is).

regards

Jonathan