[Scummvm-devel] Evolution of Scummvm-tools

[Johannes Schickel]

On 01/05/2016 09:16 AM, Arnaud Boutonné wrote:
> But that's only a part of the problem. My main issue at this point is 
> that the Common code it's currently using is extremely obsolete, which 
> results in the use of multiple unsafe functions that are currently 
> forbidden in Scummvm/Master by the use of common/forbidden.h. As it's 
> missing a lot of other basic stuff, a lot of code is duplicated 
> uselessly like the code in Common/utils, the required audio decoders, 
> etc.
>

In fact, common/ etc. code in tools is partly diverged from our 
mainline. Due to needs specific to our tools. Thus, parts of it might 
lag behind, and parts might just be different on purpose.

common/forbidden.h is only targeted at our main codebase on purpose. It 
serves to prevent engine etc. code from using functionality which might 
not work on all platforms we support (due to various reasons). However, 
our tools code has much less portability needs. Thus, there is no need 
for a mechanism like common/forbidden.h in tools.In fact, it might even 
be cumbersome in tools due to our tools partly focussing on ease of 
implementation over portability.

> I know there have been historical reasons for splitting apart the 
> scummvm-tools, typically because they are user-oriented at the 
> opposite of the devtools which are developer-oriented.
> Does it still make sense to keep them in a different repository, 
> rotting silently? I have been poked by users trying to compile the 
> tools and unable to do so, so I think that at least the user-oriented 
> approach failed for that matter...
>
> So my question is: could we consider to put the user tools in Master, 
> after making them compatible with the current Common code? and if not, 
> is there a git mechanism (criezy mentioned sub-tree) we could use, 
> along with some good coding practices?
>

Putting tools in master seems like a very bad idea for multiple reasons. 
I'll go through them in random order.

Our tools code has different portability needs than our mainline code. 
We target only desktop platforms with tools. This allows our tools code 
to be much laxer when it comes to portability. For example, we can 
assume a sane standard C library implementation etc. This simplifies 
coding for tools by giving programmers a wider range of preexisting 
functionality they can employ.

Our tools code is targeting completely different features compared to 
our mainline code. For example, we want native GUI support in our tools. 
This improves usablity for users which are unfamiliar with CLIs. This 
requires us to use external libraries which depend on functionality, 
which would be excluded in any ScummVM mainline code.

Our mainline codebase has many hidden in-code cross dependencies. For 
example, Common::File in mainline goes through the backends filesystem 
API to work easily with a lot of different platforms. Common::File in 
tools simply uses standard C libraries, because that works fine on all 
platforms our code are targetted to. This makes Common::File in mainline 
depend on an initialized and working backend/OSystem.
A result of this is: One can't easily link our mainline code in a 
scenario like tools (i.e. no OSystem present and all dependend 
functionality). There is no easy way around this. One would need to 
refactor our _mainline_ codebase to make it usable in tools.


Seeing how often we added new tools in the past years, it seems simply 
not worth the effort and additional complexity to refactor both mainline 
code and tools code. Simply moving tools to master and/or sub-treeing 
master into tools does not work unless we tackle the refactoring first. 
That also might give the (wrong) impression to -developers that all 
mainline code is usable in tools. Thus, I strongly feel we shouldn't do 
either. Updating code in tools for recent changes makes sense however. I 
don't think it should be too much work (at least in the case of audio 
codecs).

I have seen the MSVC project file related commits to -tools last night. 
I also remember you (Strangerke) asking about whether we can extend 
create_project to work with tools too. This makes me wonder about 
something we can definitely think of: Using a different build system for 
tools. That seems like a much more efficient way to make MSVC 
compilation work easily. As much as I dislike saying this: for example, 
we can think of using CMake, which would give us MSVC (and even XCode) 
project file generation for free in tools. Since our tools support far 
less platforms and have a much simple configuration stage CMake should 
be fine there.