[Scummvm-devel] Unit tests

Max Horn max at quendi.de
Mon Dec 22 07:39:01 CET 2003


Am Montag, 22.12.03 um 12:35 Uhr schrieb Jonathan Gray:

>
>
> On Sun, 21 Dec 2003, Max Horn wrote:
>
[...]

>> Other than that, it doesn't need the C++ std lib in any way (thus
>> making Ender happy), it's rather lightweight, and (I think) easy to 
>> use.
>
> Although the argument against the C++ lib was portability to small
> devices, I'd think that a series of tests wouldn't need to be run on
> these devices themselves.  But then again there are the various 
> compilers
> by the folks at microsoft...

Indeed. However, we still *can* use the C++ lib, we just don't *have* 
to use it with this lib. In fact, by default it does use iostreams, but 
can be configured to use stdio, too.


>>
>>
>> If nobody objects, I would like to add a "test" directory to the
>> scummvm dir, with the following layout:
>
> Maybe call the directory "regress" or something similiar? Test is a bit
> vague and something like regress implies regression tests/unit tests 
> in my
> opinion.

"test" is what about everybody else uses (well except for Perl tests 
which are in a directory called "t" :-). "regress" IMO only implies 
regression tests... While unit tests also work as regression tests, 
that is only part of their purpose.

[...]

>>
>> In the future, more tests could be added there, also including tests
>> for gui/backends/sound, or even for the engines, if needed.
>
> This sounds like a great idea, anything that helps find bugs quicker 
> can
> only be a good thing (well as long as it isn't creating bugs itself :)
>
It can't cause bugs because it's fully external to the actual source.


> It would be nice to be able to do unit tests using the game logic (ie 
> byte
> compiled scumm scripts for scumm) but obviously this requires 
> additional
> things that are currently not implemented for the most part (as fun as 
> writing
>  scripts by hand in hex is).

syke was "advertising" for doing engine unit tests several months ago 
on IRC. And I know he did work in this area for FreeSCI. I do agree 
that doing that would be very useful (essentially, if we added unit 
tests for each bug we encounter and fix, finding & fixing regressions 
would be a breeze). However, there are several hard problems in this 
regard. Some things to consider:
1) Work with real (game) data files vs. creating special test data files
    Real data is nice because it already exists, and actual bug occur in 
specific games/rooms. I.e. we could couple this with savegame for each 
bug to test, and then do tests from there
    OTOH, that means in order to be able to run the test, you need to 
have access to the game data in question. That is solved by artificial 
test data; however, we must first create it, which could be a problem, 
since we have no tools for this
2) Instead of working with data files (see 1), we could also just test 
functions/classes in isolation. This is somewhat orthogonal to approach 
1 since other problems are covered by this
3) The engines provide no real "hooks" for any testing right now. That 
might be no problem, though, really depends....

Ideally, it would be nice if we could create engine unit tests which 
work like this: "Load savegame XYZ, walk actor to object FOO; actor 
should come to a full stop at coords (x,y) eventually. If not, report 
error". I.e. translating bug reports more or less directly into unit 
tests would be very sweet, but quite hard to achieve in a useful 
fashion, I think... but of course already being able to test single 
building blocks (like the walking code in Scumm) would be nice.

However, first things first. This is mostly daydreaming :-)



Cheers,

Max





More information about the Scummvm-devel mailing list