[Scummvm-devel] R: Re: R: Re: R: Re: Dynamic Plugins (uncached) - optimization

[Johannes Schickel]

On 01/26/2014 07:07 PM, sunmax at libero.it wrote:
> Hi Johannes,
>
>> It's right that USB 1.0 isn't going to get any faster. However, what's
>> the throughput of USB 1.0? ~1.5MB/s? So for 30 MB (how much memory take
>> all the plugins anyway?) of plugins we get ~20s time loading all the
>> plugins from disk into memory. Even when the overhead would be 1/3 we
>> still are only at 30s. That sounds vastly different from the 15s vs
>> 1m40s you mention, which sounds like loading takes 85s.
> Those speed are purely theoretical.
>
> PS2 ScummVM main elf is ~2 MB (when compiled as dynamic plugins) and it takes
> ~15s to boot (uncached, no plugins).
>
> Subtracting the bootstrap portion (from loading done to GUI Launcher), is
> still 8-10s to load the main binary alone before it can be executed.
>
> So we are closer to ~300Kb/s than 1.5MB/s.
>
> It all depends on the driver (IRX) beside the HW implementation (of both port
> and specific USB key used).
>
> There are more optimized USB IRX (not part of the PS2 SDK) but their license
> is incompatible, or they are not available for free. The ones which ship with
> some PS2 DVD games are clearly much faster ;-)
>
> Now back to what we have and ScummVM: plugins are 20MB, 300Kb/s -> 70s +
> overhead + main binary (15s) = 1m40s.
>
> It's not the ScummVM implementation, it's PS2 low-level backend implementation
> (driver and C-libs).
>
> There is no simple way to re-implement the latter, so if we want to improve
> this on PS2 we need to tackle at the level where we can control it, by avoiding
> loading multiple plugins when we can know a-piori the exact one needed (whether
> to start or to add a game, when it's stored in a folder with a name matching to
> gameid).
>
> Even if the plugins loading & logic were 0s overhead, it's still 70s of
> pointless wait to load them all, which we could take down to 5s with the static
> table I'm suggesting.
>
> This will actually benefit all platforms, even the fast ones. Since loading 1
> plugin, is always going to be faster than loading 20 of them.
>
> Even if we take it down to 1s/plugin, waiting 1 sec is still better than
> waiting 20 sec.
>

Well, if file I/O is really that slow it makes more sense to work on 
something else indeed. Speeding up the loading of plugins would've been 
a benefit to other platforms using them too, but might not be worth it 
in case PS2 is still too slow anyway.

I thought a bit more about how we might speed up detection in a way you 
proposed: Instead of matching the directory name against gameids we 
might simply create a list of files engines look for in their detection 
and then map these to a plugin (maybe even prioritized in case multiple 
engines look for the same file). That should also catch cases where 
users name their directories not based on gameid. But one would need to 
check whether this is really (easily) feasible.

// Johannes