[Scummvm-devel] Detecting patched games / SCI bug reports

[Max Horn]

Am 16.08.2010 um 15:32 schrieb M. Kiewitz:

> 
>> Fair enough on the fan-games! But I am not sure how you
>> come to the "most of those" conclusion: Looking at random
>> non-assigned SCI bugs, and excluding fan-made ones (and even
>> demos), most of them seem to be uncommented, or have at most
>> comments by the submitter or by somebody who says "me too"
>> :). 
>> 
>> Anyway, here is a more extensive list of examples. Of
>> course, the numbers of bugs that were addressed, or even
>> fixed, is also huge, and the SCI folks are doing a truly
>> heroic task right now. They just could need more helping
>> hands, I'd say.
> 
> The problem with sci is that it's getting more and more complicated to fix tiny things. First you need to find out, if it's really just occuring in ScummVM SCI, you then need to go through sierra sci to find out what's going wrong. If our implementation is really incorrect, you can't fix it immediately. You need to go through all sorts of other interpreter versions just to figure out, when it was changed. Helping hands are nice, but we are now at a state, where it's getting really difficult to fix most of the bugs w/o introducing regressions. For example - just fixing the "I'm melting" sound issue in sq1 required me to go to several "special" places in multiple other games, just to find out how sierra sci behaves in those other cases.

Oh, and one more PS: I think it's great that you try to absolutely introduce no regressions with your changes. But IMHO, you shouldn't spend *too* much time on this -- some time, of course, just not *too* much. Simply because it quickly would turn into a bad use of resources: You are one of the few people who can fix those bugs; in contrast, we have a multitude of people who can perform game tests. Hence, it doesn't make sense to apply too much of your time to general regression testing; let others do that. Chance is, if there is a regression, somebody will spot it and report it, and then using "git bisect" (assuming you use "git svn"), plus the fact that we mark in each commit which bug(s) it was addressing, makes it pretty easy to find out when the regression was introduced. Sure, the drawback is that you then have to first re-evaluate the issue and "get into the code" again, but assuming all workarounds and bug fixes are sufficiently documented, that is not too hard. 

In summary, good change documentation (via source and bug tracker comments as well as clear commit mails), plus lots of testers, makes regression handling usually pretty smooth.

Cheers,
Max