[Scummvm-devel] Bring testers to the team?

[Travis Howell]

Eugene Sandulenko wrote:
> There was a quite sound idea mentioned recently, that is bring "full
> time" testers to the team.
> 
> What do you think guys?

No, I think it is far better to stick with the more varied testing 
offered by public testing for each release cycle, even if we struggle to 
get full testing each time around.

The main problem with using the same people to testing games (especially 
those with multiple game paths) each release cycle, is it is far too 
easy to get tired of those games. With people only playing the essential 
parts more and more, each release cycle.

A variety of testers is much better, because people looking forward to 
playing a game or have not played game for a long time, are much more 
likely to give that game a more thorough testing.

A variety of testers, means there is much more chance of finding rare 
bugs in each release cycle, as different people tend to play games 
differently in many ways.

> What we can offer:
> 
>   - Place in our Credits list (on the site, in the docs, in the program)
>   - @scummvm.org e-mail
>   - +v on our IRC
>   - Access to Wiki (though we give it practically to everybody now)
> 
> And these need to be discussed:
>   - Access to our donations for purchasing games. (perhaps some max
>     amount per year? I don't really know, I don't want to abuse it, and
>     would rather save for the developers)
>   - Moderator status on the forums. (They will be part of the team, so
>     why not?)
>   - +o on IRC. (Will they really need it? IMHO +v is enough)

None, sounds far too easy to get access to the various resources offered 
to developers on the ScummVM project. Only wiki access could even be 
useful, and only during play testing part of release cycles.

On the credits issue, we would really need to add all other public 
testers to credits in order to be fair too. Which means would need to 
get the name of all testers, and what name type (nickname or real name) 
they want used. So more time wasted gathering that data, and compiling 
list into credits for each release.

> The requirements to these positions would be:
>   - Own more than 5 games. (There will be more mentoring that benefits
>     if they will have less. May be even increase the required amount?).
>   - Be around release time. That is twice a year
>   - Test their games
>   - Have more than one device to play ScummVM on is a plus
>   - Have non-English versions of the games is a plus

The requirement would need to be more like 4 times that amount of games 
(20), to order to test a different set of games each release cycle (for 
reasons stated above).

We currently don't have set times of the year for releases cycles, so 
your basically asking people to be available any time.

Playing testing their games on multiple platforms is going to make 
people get tired of their games even quicker, playing only the most 
essential parts of their games when testing.