[Scummvm-devel] GSoC gui tools

[Max Horn]

Hi Joost, all!

Am 23.11.2009 um 03:55 schrieb Joost Peters:

> Hi folks,
>
> As some of you will know, as part of Google's Summer of Code  
> programme,
> we had a student working on unifying our tools and creating a new GUI
> front-end using the wxWidgets library.
> Apart from some bugs, this is in a mostly finished state, so we should
> really aim to merge this into trunk sooner rather than later.  I  
> want to
> work on this and push the process forward the coming week(s).
>
> It seems there are 2 approaches we can take; 1) we slowly merge in a  
> few
> steps, update the old tools to use the new structure / classes,  
> etc.  or
> 2) we do (regression) testing and bug fixing in the branch, then  
> import
> it wholesale when we've fixed all outstanding issues.
> I'm personally in favour of option 2 since it seems the least amount  
> of
> work and easier distributable.   Any opinions?

Well, *if* we have enough man-power (e.g. if for every compress_FOO  
tool, the people who wrote it help out), then (2) sounds like the way  
to go. Drawback is that we are stalled until we are able to test  
*everything*. Or alternatively, until we tired enough to take the risk  
and just move the stuff anyway, relying on end users to discover  
regressions :/.

Anybody on this list willing to help? At all??

/me watches dustballs roll by...

We could of course try asking on the forums for help. Worth a shot,  
but based on past experience, I am not sure it'll really help.


(1) may sound more complicated at first, but at least it allows us to  
move tools to the new system at once after we tested them. If done  
right, I don't think it would be much slower, either: We just have to  
rename compress.cpp/.h in trunk to compress_old.cpp/.h and adapt the  
current tools. Then you can just merge the new versions of the tools  
you already tested to trunk right away, together with the new  
compress.* and the other new files. Presto.




> Regardless of which method we choose, the following jobs will
> (eventually) need to be done, and any help would be appreciated:
>
> - Testing of the GUI version of the new unified tools(s) on various
> platforms.

Last time I tried them they worked quite OK on Mac OS X (I didn't test  
all compression tools, though, only a few).

>
> - (regression) testing of the CLI version of the new tool(s) against  
> the
> old standalone ones. Either using a test suite/script or manually.
>    So far I've tested compress_scumm_sou (various games/demos),
> compress_queen, compress_agos (Simon 2 only), compress_tinsel (DW1  
> only).
>
> - Creating a test suite/script.
>   I've written a fairly ghetto bash shell script that aids in testing
> (compares the md5 sum of a known-to-be-good file against what the tool
> creates), but it's not great and could either do with a bit of work,  
> or
> a better alternative.  Is anyone familiar with an existing system for
> such a task?

I am not familiar with a specific system as such, but I believe a  
relatively simple python / perl script could take care of this  
already. Essentially, we'd need to define a list of records, each  
record describing a test case and consisting of at least these entries:

   source file name(s)
   source md5(s) -- to verify the input file is the right one
   then a list of these for { FLAC, MP3, Vorbis }:
     compression instructions -- i.e. which tool to invoke, and with  
which params
     output file name(s)
     output md5(s) -- to verify the output is bit identical to what we  
expect.


Then performing a test would be a simple matter of running over each  
record, compressing the input file and checking the output.

A refined version would only re-run a test if the compression tool  
binary changed (i.e. is newer than the output file), or if the test  
hasn't bee run yet (i.e. if the output file does not yet exist). This  
is something one gets for free with "make". So it might even be best  
to do the above with a Makefile, combined with a script:

test: dott-demo/monster.so3 ...
	run script which checks the md5 of every output file and prints  
"pass" or "failure"

dott-demo/monster.so3: dott-demo/monster.sou tools/compress_sou
	compress_sou PARAMS





>
> - Setting up tool builds for various platforms on our buildbot.

Indeed, that would be awesome. In addition, it would be nice if  
buildbot would run both the tools test suite as well as the unit tests  
in our code base on dedicated builders, too ;).

Sadly, we have already seem to have a hard time maintaining the  
existing buildbot stuff ... :/


Max