[Scummvm-devel] R: Re: R: Re: R: Re: R: Re: Dynamic Plugins (uncached) - optimization

[sunmax at libero.it]

Hi Johannes,

Cool!

>I thought a bit more about how we might speed up detection in a way you 
>proposed: Instead of matching the directory name against gameids we 
>might simply create a list of files engines look for in their detection 
>and then map these to a plugin (maybe even prioritized in case multiple 
>engines look for the same file). That should also catch cases where 
>users name their directories not based on gameid. But one would need to 
>check whether this is really (easily) feasible.

Yes, your approach is much more elegant.

Don't we already have a list of the files each engine is looking for as part 
of detection ?

I'm not sure (gotta check now) if this is consistent between different 
engines, or if they store those list each in a different way.

I think this will also make a difference for other platforms when we are about 
to prepare a release, since shaving even few seconds from each game you install 
/ start / exit, we cut down the time considerably.

Let me know if I can assist with tinkering.

Thanks,
-max


>----Messaggio originale----
>Da: lordhoto at gmail.com
>Data: 27/01/2014 21.08
>A: "sunmax at libero.it"<sunmax at libero.it>, <scummvm-devel at lists.sourceforge.
net>
>Ogg: Re: [Scummvm-devel] R: Re: R: Re: R: Re: Dynamic Plugins (uncached) - 
optimization
>
>On 01/26/2014 07:07 PM, sunmax at libero.it wrote:
>> Hi Johannes,
>>
>>> It's right that USB 1.0 isn't going to get any faster. However, what's
>>> the throughput of USB 1.0? ~1.5MB/s? So for 30 MB (how much memory take
>>> all the plugins anyway?) of plugins we get ~20s time loading all the
>>> plugins from disk into memory. Even when the overhead would be 1/3 we
>>> still are only at 30s. That sounds vastly different from the 15s vs
>>> 1m40s you mention, which sounds like loading takes 85s.
>> Those speed are purely theoretical.
>>
>> PS2 ScummVM main elf is ~2 MB (when compiled as dynamic plugins) and it 
takes
>> ~15s to boot (uncached, no plugins).
>>
>> Subtracting the bootstrap portion (from loading done to GUI Launcher), is
>> still 8-10s to load the main binary alone before it can be executed.
>>
>> So we are closer to ~300Kb/s than 1.5MB/s.
>>
>> It all depends on the driver (IRX) beside the HW implementation (of both 
port
>> and specific USB key used).
>>
>> There are more optimized USB IRX (not part of the PS2 SDK) but their 
license
>> is incompatible, or they are not available for free. The ones which ship 
with
>> some PS2 DVD games are clearly much faster ;-)
>>
>> Now back to what we have and ScummVM: plugins are 20MB, 300Kb/s -> 70s +
>> overhead + main binary (15s) = 1m40s.
>>
>> It's not the ScummVM implementation, it's PS2 low-level backend 
implementation
>> (driver and C-libs).
>>
>> There is no simple way to re-implement the latter, so if we want to improve
>> this on PS2 we need to tackle at the level where we can control it, by 
avoiding
>> loading multiple plugins when we can know a-piori the exact one needed 
(whether
>> to start or to add a game, when it's stored in a folder with a name 
matching to
>> gameid).
>>
>> Even if the plugins loading & logic were 0s overhead, it's still 70s of
>> pointless wait to load them all, which we could take down to 5s with the 
static
>> table I'm suggesting.
>>
>> This will actually benefit all platforms, even the fast ones. Since loading 
1
>> plugin, is always going to be faster than loading 20 of them.
>>
>> Even if we take it down to 1s/plugin, waiting 1 sec is still better than
>> waiting 20 sec.
>>
>
>Well, if file I/O is really that slow it makes more sense to work on 
>something else indeed. Speeding up the loading of plugins would've been 
>a benefit to other platforms using them too, but might not be worth it 
>in case PS2 is still too slow anyway.
>
>I thought a bit more about how we might speed up detection in a way you 
>proposed: Instead of matching the directory name against gameids we 
>might simply create a list of files engines look for in their detection 
>and then map these to a plugin (maybe even prioritized in case multiple 
>engines look for the same file). That should also catch cases where 
>users name their directories not based on gameid. But one would need to 
>check whether this is really (easily) feasible.
>
>// Johannes
>

>------------------------------------------------------------------------------
>CenturyLink Cloud: The Leader in Enterprise Cloud Services.
>Learn Why More Businesses Are Choosing CenturyLink Cloud For
>Critical Workloads, Development Environments & Everything In Between.
>Get a Quote or Start a Free Trial Today. 
>http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
>_______________________________________________
>Scummvm-devel mailing list
>Scummvm-devel at lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/scummvm-devel
>