[Scummvm-devel] R: Re: R: Re: R: Re: Dynamic Plugins (uncached) - optimization

[sunmax at libero.it]

Hi Johannes,

> It's right that USB 1.0 isn't going to get any faster. However, what's
> the throughput of USB 1.0? ~1.5MB/s? So for 30 MB (how much memory take
> all the plugins anyway?) of plugins we get ~20s time loading all the
> plugins from disk into memory. Even when the overhead would be 1/3 we
> still are only at 30s. That sounds vastly different from the 15s vs
> 1m40s you mention, which sounds like loading takes 85s.

Those speed are purely theoretical.

PS2 ScummVM main elf is ~2 MB (when compiled as dynamic plugins) and it takes 
~15s to boot (uncached, no plugins).

Subtracting the bootstrap portion (from loading done to GUI Launcher), is 
still 8-10s to load the main binary alone before it can be executed.

So we are closer to ~300Kb/s than 1.5MB/s.

It all depends on the driver (IRX) beside the HW implementation (of both port 
and specific USB key used).

There are more optimized USB IRX (not part of the PS2 SDK) but their license 
is incompatible, or they are not available for free. The ones which ship with 
some PS2 DVD games are clearly much faster ;-)

Now back to what we have and ScummVM: plugins are 20MB, 300Kb/s -> 70s + 
overhead + main binary (15s) = 1m40s.

It's not the ScummVM implementation, it's PS2 low-level backend implementation 
(driver and C-libs).

There is no simple way to re-implement the latter, so if we want to improve 
this on PS2 we need to tackle at the level where we can control it, by avoiding 
loading multiple plugins when we can know a-piori the exact one needed (whether 
to start or to add a game, when it's stored in a folder with a name matching to 
gameid).

Even if the plugins loading & logic were 0s overhead, it's still 70s of 
pointless wait to load them all, which we could take down to 5s with the static 
table I'm suggesting.

This will actually benefit all platforms, even the fast ones. Since loading 1 
plugin, is always going to be faster than loading 20 of them.

Even if we take it down to 1s/plugin, waiting 1 sec is still better than 
waiting 20 sec.

Thanks,
-max


>----Messaggio originale----
>Da: lordhoto at gmail.com
>Data: 26/01/2014 17.16
>A: "sunmax at libero.it"<sunmax at libero.it>, <scummvm-devel at lists.sourceforge.
net>
>Ogg: Re: [Scummvm-devel] R: Re: R: Re: Dynamic Plugins (uncached) -	
optimization
>
>On 01/26/2014 04:44 PM, sunmax at libero.it wrote:
>> Hi Johannes,
>>
>>> I think it would be better to find out why loading is that slow first
>>> and see whether we cannot optimize that before we try to implement some
>>> more complicated solution.
>> 1) PS2 USB 1.0 is not going to get any faster, no matter how long we
>> investigate it.
>>
>> It's only efficient if we load a single plugin to play and add new games.
>>
>> 2) The PS2 C libs FILE implementation is suboptimal indeed (not the ScummVM
>> backend/fs the PS2 SDK one).
>>
>> Rewriting PS2 C libs internal file handling is an order of magnitude more
>> complex than adding a static table to ScummVM, I think.
>>
>
>It's right that USB 1.0 isn't going to get any faster. However, what's 
>the throughput of USB 1.0? ~1.5MB/s? So for 30 MB (how much memory take 
>all the plugins anyway?) of plugins we get ~20s time loading all the 
>plugins from disk into memory. Even when the overhead would be 1/3 we 
>still are only at 30s. That sounds vastly different from the 15s vs 
>1m40s you mention, which sounds like loading takes 85s.
>
>Did you try to measure how long loading each individual plugin takes 
>(i.e. check how long DLObject::load runs)?
>
>// Johannes
>

>------------------------------------------------------------------------------
>CenturyLink Cloud: The Leader in Enterprise Cloud Services.
>Learn Why More Businesses Are Choosing CenturyLink Cloud For
>Critical Workloads, Development Environments & Everything In Between.
>Get a Quote or Start a Free Trial Today.
>http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk
>_______________________________________________
>Scummvm-devel mailing list
>Scummvm-devel at lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/scummvm-devel
>