[Scummvm-devel] Dynamic plugins and GNU ld

[Max Horn]

Hi folks,

so recently, adding support for dynamic plugins to various ports has  
come up again and again; here and elsewhere. See also e.g. this forum  
thread <http://forums.scummvm.org/viewtopic.php?t=5946>

Unix systems have an easy time here, they have dlopen/dlsym and can  
just use that without worrying. Windows might be more difficult but  
can cope.

But it seems that for most "small devices" out there, there exists no  
ready to use support for dynamic modules. This lack is, at least so I  
guess, the major obstacle preventing the NDS & PSP & GP2x (and  
probably more) from implementing dynamic modules.

Another small platform overcame this problem: Thanks to Marcus  
Comstedt, the Dreamcast has a nice working plugin system.

How did he do it? Well, Marcus should tell that story, not me, but  
I'll try to give a rough overview of it anyway (and am sure Marcus  
will correct all the wrong parts of it and fill in the correct stuff ;-)

 From what I can gather from the stuff in backends/platform/dc, he  
takes advantage of a special feature of the GNU linker, namely linker  
scripts (see <http://sourceware.org/binutils/docs-2.18/ld/ 
Scripts.html>). The DC port provides a linker script, backends/ 
platform/dc/plugin.x, which the custom Makefile of the DC port then  
passes on to the linker. The link script tells the linker to produce  
an elf32-shl binary, with certain content of the various sections; in  
particular, it also seems to generate a list of constructors/ 
destructors for global objects.

The core of Marcus' "dlopen" implementation can be found in backends/ 
platform/dc/dcloader.cpp. It knows how to parse the ELF files GNU ld  
generated, and provides code for relocating the loaded code; for  
loading & scanning the symbol list; and so on. It also makes sure that  
the constructors/destructors of global objects in the module are  
called at the appropriate time.


So, in principle at least, any port using GNU ld should be able to use  
the same techniques, right? With some adjustments for endianess maybe,  
or variations in the ELF format (and if a port doesn't use ELF at all,  
even more adjustments). Nevertheless, a good part of Marcus work might  
be reusable, it would seem, naively (I admit I have not scanned the  
code in depth, so I might have missed something, but again, I hope for  
Marcus to fill in the missing bits).

Many of our ports seem to use a GCC based cross compilation build  
chain. I would assume most (all) of them use the GNU linker. I have no  
idea how many of them use ELF, and which other formats are involved.

Nevertheless, it should be clear at what I am aiming here now: If  
somebody was to take that code and tweak it to work on any ELF-arm  
port, then it's very likely that same code could then be made to work  
on any ELF-arm port; with some more care, it might even be possible to  
use the same code for all the relevant ports.

What is nice about this is that now not each porter has to  
individually create a dlopen implementation, but rather you could join  
forces; also could take advantage of existing work; and by sharing the  
code, it gets wider testing for free, too.

Ah well. At the end of the day, it is not clear whether this is  
feasible or not, it might be just a pipedream; even if not, we would  
still need "somebody" who actually would be willing and *able* to  
implement this... So I am not holding my hopes up too much, but I  
figure it's at least worth a try to point this out... :)


Cheers,
Max