[Scummvm-devel] Changes to plugins and config manager

[yotam barnoy]

Hi guys. Just wanted to update you about some recent changes to the
infrastructure and to get your feedback.

The 'one plugin at a time' design has been renamed to 'uncached
plugins' and the implementation is now pretty much complete. Any
backend concerned about memory should probably use this scheme, since
the alternative ('cached plugins' aka the default option) requires
loading every plugin into memory at once, and our engine count is
constantly growing. The only exception I know of is the DC, which
stores all of its plugins on a disc separate from the game data and
would need constant disc swapping to alternate between plugins.

Finding the right plugin for a particular game using this scheme
usually takes time as each plugin must be loaded into memory and then
detected with. To speed things up, the first time a game is found by a
particular plugin, an entry is made in the ConfigManager linking the
gameid to the plugin filename. Any future calls to
EngineManager::findGame() will use that filename first.

To allow writing the entries in the ConfigManager, I had to create
another domain called plugin_files. The reason for this is that a) I
didn't want to clutter the AppDomain with strange entries, and b) I
also didn't have access to the game domain since I only had the gameid
to work with. To enable this new domain, I made some changes to
ConfigManager. From now on, every game domain must have a gameid key.
A domain that doesn't have a gameid key will be considered a
miscellaneous domain (miscDomain) when loading from the ini file. It
will still be available from getDomain() and all normal operations,
but it won't be included in operations that run on gameDomains only
ie. hasGameDomain() and getGameDomains().

Compared to the cached system, the uncached plugin system boots up
faster since it doesn't need to load all the plugins on startup, and
the same is true when returning to the launcher. Slowdowns occur only
when detecting games to add, since that requires loading every plugin
one by one.

Finally, to prevent the endless battles with memory fragmentation, I
followed Lordhoto's advice and created a Memory Manager for ELF
plugins that first scans all the ELF plugins for size only (without
loading them) and then allocates the size of the biggest plugin. This
allocation is never released. Of course, this method results in some
wasted space -- the bigger the biggest plugin, the more memory is
wasted. It also means that the work of engine authors to reduce the
size of their binary (for example by creating data files) has real
impact.

This waste of memory is also not appropriate for the DS, where every
drop counts, and the Memory Manager is therefore disabled there. The
DS will have to rely on other defragmentation techniques, but will
probably not be able to go from a game to the launcher and back into
another game just because of the fragmentation.

Later,
Yotam