[Scummvm-devel] Dynamic plugins and GNU ld

[Marcus Comstedt]

Max Horn <max at quendi.de> writes:

[...]
> But it seems that for most "small devices" out there, there exists no  
> ready to use support for dynamic modules. This lack is, at least so I  
> guess, the major obstacle preventing the NDS & PSP & GP2x (and  
> probably more) from implementing dynamic modules.

At least PSP has some support for dynamic modules in the system
software.  Don't know about GP2x.  NDS has no system software that I'm
aware of.  :-)


> From what I can gather from the stuff in backends/platform/dc, he  
> takes advantage of a special feature of the GNU linker, namely linker  
> scripts (see <http://sourceware.org/binutils/docs-2.18/ld/ 
> Scripts.html>). The DC port provides a linker script, backends/ 
> platform/dc/plugin.x, which the custom Makefile of the DC port then  
> passes on to the linker. The link script tells the linker to produce  
> an elf32-shl binary, with certain content of the various sections; in  
> particular, it also seems to generate a list of constructors/ 
> destructors for global objects.

The linker script sorts out the global constructor and destructor
list for the plugin, yes.  That's basically all it does.  The main
magic is not in the linker script, but in two other options to ld.

The first is "-q".  This makes ld generate a relocatable binary.
This is different from a relocatable object (".o" file) in that
all symbolic references have been resolved.  It is basically the
same a regular static binary, except that it contains relocation
sections, which allows you to relocate the loadable sections to
new addresses and still get working code.  The code is not position
independet, it's just that the complete list of things that need
to be patched is available.

The second is "--just-symbols".  As I wrote, all symbolic
references in the relocateble binary are resolved.  This includes
references to the main binary (Common::whatever etc).  So this means
the main binary needs to be part of the link.  On the other hand, the
actual code of the main binary should not be included in the plugin
output.  Adding the main binary to the link with the option
"--just-symbols" takes care of that, it loads _only the symbol table_
from the main binary, and uses it to resolve references.  Hence, the
resulting plugin binary will only work if the main binary has been
loaded into RAM first (because it contains pointers to addresses
within the main binary, without actually containing that code), but
that's fine for the purpose of a plugin.  Also, the plugin will only
work with that particular build of the main binary, since the
references are by address, and not symbolic.

There is one more option, although it's non-essential:  In order
to reduce the size of the plugins, I remove all symbols from the
output binary except for the ones that are actually needed (i.e.
the ones I intend to call "dlsym" on).  This is done with
"--retain-symbols-file plugins.syms", where plugin.syms lists the
symbols which should be kept.


> The core of Marcus' "dlopen" implementation can be found in backends/ 
> platform/dc/dcloader.cpp. It knows how to parse the ELF files GNU ld  
> generated, and provides code for relocating the loaded code; for  
> loading & scanning the symbol list; and so on. It also makes sure that  
> the constructors/destructors of global objects in the module are  
> called at the appropriate time.
>
>
> So, in principle at least, any port using GNU ld should be able to use  
> the same techniques, right? With some adjustments for endianess maybe,  
> or variations in the ELF format (and if a port doesn't use ELF at all,  
> even more adjustments).

The major thing that needs changing is the processing of the
relocation records.  The thing is that relocations are architecture
specific.  So my code handles SuperH relocations.  MIPS and ARM
have different relocations, so they need different code.

Other than that it should just be a matter of changing the header
match values for endianness and architecture appropriately.  The two
variations of ELF which exist are 32-bit and 64-bit, and I don't think
GP2x is 64-bit.  I know for sure that NDS and PSP aren't.


  // Marcus