[Scummvm-devel] Avoiding loading of all plugins during startup

[Angus Lees]

I understand if it is considered out of scope for this GSoC project,
but I would be very happy to see the detection code separated from the
engines.

For the Android port, I have the engine plugins all split out into
separate optional packages since having all of them at once is much
too big for a typical Android app.  At the moment, users have to
install the right engine "plugin" packages themselves - and it would
be great if I could instead suggest which ones were needed
automatically.

 - Gus

On Wed, Jun 23, 2010 at 20:26, yotam barnoy <yotambarnoy at gmail.com> wrote:
> On Wed, Jun 23, 2010 at 12:54 PM, Max Horn <max at quendi.de> wrote:
>> 1) Quick and simple refinement
>
> I think this solution should theoretically be easiest to implement and
> therefore should be the first 'milestone'. There's no reason the
> current code shouldn't work this way in general, and it really is a
> waypoint on the way to 2 since the code will have to handle having
> only one plugin in memory at a time.
>
>>
>> 2) Suggested long term approach
>> -------------------------------
>>
>> On the long run, I propose this: We add to every game target in the config file an "engine=BLABLA" key-value pair. This way, we can pick the right engine plugin right away. To ease transition to this new format, there are several ways: One is that if the "engine=" field is missing, we just revert to the old behavior. Another is that during startup, we scan through all config targets, and if we find any missing the "engine" key, we try to add it. By re-running the detector, matching the output with the gameid that is already there, and setting up the engine field. For this, the upgradeTargets()  function in base/commandLine.cpp could be adapted.
>>
>> In order to avoid having to do this re-scanning every time, I would suggest also finally putting the "version" field in our config file to some use: Right now it is worthless, because we simply set it to the ScummVM version whenever we load a config file. We could change that: If loading a config file made with a newer ScummVM version, issue a warning? And if the version is older than XYZ, run the target upgrader.
>
> I think this is the best method. We could avoid the version issue if
> our behavior is
> a) use the engine= field.
> b) If we can't find an engine, run the detector according to method 1,
> updating the config file only for the chosen game (to avoid too much
> work). This means we should have method 1 working first.
>
>> 3)...
>> All three approaches bear the risk that one may end up with a mismatch between the "gameid->engine" table used and the engine versions used.
>
> And I don't like them :(
>
>> * We currently support starting a game without a config entry, too: If you type
>>  scummvm monkey2
>> and have no monkey2 target configured, it will try to locate monkey2 in the current directory. We can't know the engine this way. Some "solutions":
>> ** If approach 3 is realized in some way, this problem does not exist ;)
>> ** Drop this feature (I don't think the feature is very important; then again, dropping a long standing feature just because your new code has difficulties supporting is nasty -- I don't like this :/)
>> ** Require the user to specify a --engine=FOOBAR option -- bad from a user's perspective, I don't like this much either
>> ** Fall back to the current mode of loading all plugins (possibly with the refinement from 1) above). Easy to do, but not perfect.
>
> The last one is my preferred option.
>
>> * We still need to support adding games in the launcher, including the mass add feature. We can just keep the current way: Load all plugins, then detect with each in turn. Or we do it iteratively: Load plugin 1, scan for games; load plugin 2, scan for games; etc.
>> For a plain "Add game", both should work well. If "Mass Add" is used, though, then we may need to traverse many many directories; if we use an iterative scanning approach, we now have to iterate all those directories N times, where N is the number of engine plugins. Depending on the system, this may be a lot slower than loading all engines, then traversing the directory tree only once.
>> I don't see an ideal solution for this right now, only various tradeoff. Of course one can argue that a huge "Mass Add" is something relatively rare, and probably mostly happens on "higher end" machines, so it's not so bad if it is slow... But maybe this thought is wrong? Porters?
>
> As was mentioned before somewhere in the depths of devel logs, Mass
> Add is becoming/will become the standard add command. This is the
> functionality that users generally want as far as I can tell. I think
> a good way to approach this may be to scan the directories in batches:
> place the filepaths in an array of fixed size, then load/unload engine
> by engine and iterate over that list as opposed to doing it at every
> directory level.
>
> As a side point, I just remembered that one of my non-techie friends
> who used ScummVM for the first time commented that the title of the
> "Go Up" button was unclear to her, whereas "Parent Directory" or
> something like that would have been much clearer.
>
>> * We need to still support "--list-games" which may require loading all plugins (but could be mitigated by approach 3)
>
> It shouldn't require loading all plugins together. Just load every
> engine individually and add the output to a list.
>
> Yotam
>
> ------------------------------------------------------------------------------
> ThinkGeek and WIRED's GeekDad team up for the Ultimate
> GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
> lucky parental unit.  See the prize list and enter to win:
> http://p.sf.net/sfu/thinkgeek-promo
> _______________________________________________
> Scummvm-devel mailing list
> Scummvm-devel at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/scummvm-devel
>