[Scummvm-devel] Understaffing (Was: What is happening to the ScummVM team?)

[Max Horn]

Am 12.02.2009 um 21:27 schrieb Johannes Schickel:

> Hi,
>
> Max Horn schrieb:
>> regarding release / code quality: I concur, things seem to have gone
>> downhill when it comes to testing coverage, code/release quality, and
>> also availability of ports / testing of ports before the release.
>>
>> In short, the reason for that seems to be understaffing / lack of  
>> time
>> of people.
>>
>
> Actually that is understandable but it is IMHO no reason to force
> releases, like it seems to be the case with the last releases.
> I think unstable / bugged releases will have a worse impact on our
> reputation than a release less a year.

I agree. From my POV, we are in no way ready for a 0.13.0 release.  
E.g. the SCUMM subtitle speed bug is still in there.


>> It starts at the top: The two team leads (eugene and me) are super-
>> busy with life outside ScummVM, leading to a certain regretable lack
>> in leadership. And in bug triage. It might be nice to add 1-2 new
>> people to the head of ScummVM. We'd need somebody very active and  
>> able
>> to fill in the leadership role. Management is understaffed resp.
>> existing staff lacks time.
>>
>
> I guess this is the least important point. If all other developers  
> test
> their code properly we do not need too much managing seeing that our
> code base is mature enough for most tasks. Maybe we should add some
> guidelines / tips on various work processes.

I don't think it's that simple. For starters, people in general can't  
test their code properly: engine authors can't test all ports, porters  
don't have all games available. Bummer.

On the other hand, developers are humans and forget to do things, and  
sometimes need... coercing. I send tons of mails behind the scenes  
talking to people, pleading them to make changes or fixes, sometimes  
even sending patches I made myself blindly. Very often there is little  
to no reaction, but at least sometimes good comes from it. From my  
perception, a lot of the degrading quality indeed comes from the fact  
the Eugene and me put far less energy and effort into the whole thing  
than we used to. Maybe I am wrong and/or deluded, but I believe that  
we *did* make quite a difference when we constantly nagged people  
about fixing that bug, or just digged in ourselves to fix up stuff. I  
hope it doesn't sound arrogant, but looking at <https://sourceforge.net/tracker2/reporting/index.php?atid=418820&what=techall&span=&period=lifespan&group_id=37116 
 >, Kirben, Eugene and me really made an impact there ;-). Even though  
I'd actually prefer if bug handling was spread over more people,  
reality is that many devs don't like tracking and fixing bugs as much  
as writing new code...

But yeah, it's probably not our biggest problem :-).



>> It continues with engine maintanance: Not all our engines are being
>> very actively maintained. Right now, there is *nobody* assigned to  
>> BS1
>> (though I'd hope that the engine is stable enough to work well
>> anyway). Regarding e.g. the MM v2 bugs: 1-2 years ago I probably  
>> would
>> have already fixed those, as it is, I don't have time (other things
>> have higher priority for me) to even start debugging them. There are
>> some other people actively working on SCUMM, luckily, but I am not
>> sure how interested/equipped they are in/for fixing bugs in MM...   
>> The
>> engine teams are understaffed resp. existing staff lacks time.
>>
>
> Actually I guess if an engine is not maintained the probability that
> regressions are introduced is low. Of course there could be possible
> regressions when code in common/ etc. is changed and the engine would
> need to be adapted. But apart I guess regressions are mostly  
> introduced
> because someone working on an engine changes / adds functionality. If
> that is the case we have of course someone with knowledge about the
> engine. I guess the problem here is really that not every person  
> working
> on an engine is interested in all games supported by that engine :-/.

You are right to a degree, but nevertheless we *do* get regressions in  
these engines on occasion. Or discover old bugs. Or get problems  
specific o a port. And we do lots of changes to common, graphics,  
sound and backends all the time, and indeed, all of these can (and  
sometimes do) cause regressions, even without anybody touching the  
engine.


BTW, one thing that also bothers me a bit are engines that languish in  
a half-ready state. I wonder if any of cruise, igor, m4, made will  
ever see the light of day (= be completed) ?



>> It leads (but doesn't stop at) ports, where once again some ports
>> won't make it, for other ports we have heard little to nothing from
>> their porters. We certainly could stand getting some new people  
>> aboard
>> here! People like Chrilith, Neil, Kostas, etc. are working alone on
>> their ports, and the testing alone puts a huge burden on them. Port
>> teams are understaffed resp. existing staff lacks time.
>>
>
> I talked a bit with Joost about releases and we came to the conclusion
> that unit tests and/or a test engine to test certain backend / common
> code parts might help to find regressions in backend and common  
> code. Of
> course someone would need to work on that.

Yup, of course. As almost always, we only need to find "someone" ;-).  
I think more unit tests would be nice. And to pick up some things I  
said in the other person: While I would be fine discussing details of  
it here, too, I think in the end "he who does it" would be the one to  
decide mostly on how it will be executed ;-). *If* "someone" is found,  
that is :-)



> Not that I am not aware of our current tests in test/

Sure you are aware, you are one of two persons to have worked on them  
(BTW, I did introduce them to the list on Dec 23, 2003, but they were  
never really documented anywhere. Someone didn't do it :-).


> but it misses
> functionality to test for example graphics output, FS code, SearchMan
> code (related to FS code, just remember the latest problems on Wii and
> PSP with it), etc.

Sure. For starters, I'd already be happy if we could test "everything"  
on just Unix systems. Of course being able to run tests on all ports  
(e.g. testing the PSP FS code on an actual PSP) would also be great to  
have, but would require some more effort.

A first step would be to extend the build rules & the "main" code of  
the test suite to link against libbackends & libcommon (and then, in  
fact, out of necessity against graphics, sound, gui) to be able to  
test platform dependant stuff (including all you named).


So, if someone reads this and is interested in working on this, he is  
welcome to send / commit his patches :-)


Cheers,
Max