[Scummvm-tracker] [ScummVM :: Bugs] #14407: 3DS: make use of linear heap to make resource-intensive games playable

[ScummVM :: Bugs]

#14407: 3DS: make use of linear heap to make resource-intensive games playable
-------------------------------------------------+-------------------------
Reporter:  BallM4788                             |       Owner:  (none)
    Type:  feature request                       |      Status:  new
Priority:  normal                                |   Component:  Port: 3DS
 Version:                                        |  Resolution:
Keywords:  3ds nintendo stream memory texture    |        Game:
  tinygl freescape grim myst3 stark tetraedge    |
  wintermute                                     |
-------------------------------------------------+-------------------------
Description changed by BallM4788:

Old description:

> The 3DS has two types of heaps: a regular heap and a linear heap.
> [https://raw.githubusercontent.com/wwylele/misc-3ds-
> diagram/master/memory.svg Here is a visualization]. Regular heap is used
> via the typical methods (new, malloc, calloc, realloc, free, etc.), while
> linear heap must be accessed via
> [https://github.com/devkitPro/libctru/blob/master/libctru/include/3ds/allocator/linear.h
> these]
> [https://github.com/devkitPro/libctru/blob/master/libctru/source/allocator/linear.cpp
> functions] provided by ctrulib.
>
> On the O3DS (Original 3DS), the main program memory, the regular heap,
> and the linear heap combined can be no greater than 64MB by default; this
> can be increased up to 96MB when building ScummVM as a .cia file by
> [https://github.com/scummvm/scummvm/blob/master/backends/platform/3ds/app/scummvm.rsf#L62
> editing the .rsf file]. Similarly, on the N3DS (New 3DS), the combined
> memory cannot exceed 124MB by default, but can be increased to 178 by
> [https://github.com/scummvm/scummvm/blob/master/backends/platform/3ds/app/scummvm.rsf#L79
> editing the .rsf file]. However, per internal 3DS documentation from the
> Nintendo gigaleaks (find it yourself), the regular heap is capped to
> 96MB, even on the N3DS.
>
> Currently, the 3DS port of ScummVM uses the default regular and linear
> heap sizes
> [https://github.com/devkitPro/libctru/blob/master/libctru/source/system/allocateHeaps.c
> as defined in ctrulib] (<=24MB and <=32MB, respectively). Not only does
> it mean that there is tons of available memory currently being unused (up
> to 152+MB, not accounting for stack memory and screen buffers in the
> linear heap), but it also means the TinyGL-powered games all segfault
> during initial loading, if not immediately. As far as I know, the only 3D
> game that can currently be run on the 3DS port is Blade Runner, and
> that's only because it uses voxel-based character models.
>
> I have experimented with maxxing out the regular heap to its 96MB cap,
> and in both Grim Fandango and The Longest Journey, the intro cutscenes
> play in their entirety and the starting areas load up successfully (most
> of Manny's office in GF, and the Arcadia cliffside in TLJ). Attempting to
> progress beyond this (accessing Manny's inventory, moving right to the
> screen with his computer, or leaving his office in GF; walking right to
> the White Dragon's nest or further in TLJ) causes a segfault.
>
> If the 3DS's linear heap could be used to hold memory-intensive data,
> such as textures, streaming data, etc., it might very well be possible to
> get all the 3D games to run in the 3DS port. Except for Penumbra
> Overture, of course. That would be a fool's errand.

New description:

 The 3DS has two types of heaps: a regular heap and a linear heap.
 [https://raw.githubusercontent.com/wwylele/misc-3ds-
 diagram/master/memory.svg Here is a visualization]. Regular heap is used
 via the typical methods (new, malloc, calloc, realloc, free, etc.), while
 linear heap must be accessed via
 [https://github.com/devkitPro/libctru/blob/master/libctru/include/3ds/allocator/linear.h
 these]
 [https://github.com/devkitPro/libctru/blob/master/libctru/source/allocator/linear.cpp
 functions] provided by ctrulib.

 On the O3DS (Original 3DS), the main program memory, the regular heap, and
 the linear heap combined can be no greater than 64MB by default; this can
 be increased up to 96MB when building ScummVM as a .cia file by
 [https://github.com/scummvm/scummvm/blob/master/backends/platform/3ds/app/scummvm.rsf#L62
 editing the .rsf file]. Similarly, on the N3DS (New 3DS), the combined
 memory cannot exceed 124MB by default, but can be increased to 178 by
 [https://github.com/scummvm/scummvm/blob/master/backends/platform/3ds/app/scummvm.rsf#L79
 editing the .rsf file]. However, per internal 3DS documentation from the
 Nintendo gigaleaks (find it yourself), the regular heap is capped to 96MB,
 even on the N3DS.

 Currently, the 3DS port of ScummVM uses the default regular and linear
 heap sizes
 [https://github.com/devkitPro/libctru/blob/master/libctru/source/system/allocateHeaps.c
 as defined in ctrulib] (<=24MB and <=32MB, respectively). Not only does it
 mean that there is tons of available memory currently being unused (up to
 152+MB, not accounting for stack memory and screen buffers in the linear
 heap), but it also means the TinyGL-powered games all segfault during
 initial loading, if not immediately. As far as I know, the only 3D game
 that can currently be run on the 3DS port is Blade Runner, and that's only
 because it uses voxel-based character models.

 I have experimented with maxxing out the regular heap to its 96MB cap, and
 in both Grim Fandango and The Longest Journey, the intro cutscenes play in
 their entirety and the starting areas load up successfully (most of
 Manny's office in GF, and the Arcadia cliffside in TLJ). Attempting to
 progress beyond this (attempting to read the memo in the mail tube,
 accessing Manny's inventory, moving right to the screen with his computer,
 or leaving his office in GF; walking right to the White Dragon's nest or
 further in TLJ) causes a segfault.

 If the 3DS's linear heap could be used to hold memory-intensive data, such
 as textures, streaming data, etc., it might very well be possible to get
 all the 3D games to run in the 3DS port. Except for Penumbra Overture, of
 course. That would be a fool's errand.

--
-- 
Ticket URL: <https://bugs.scummvm.org/ticket/14407#comment:3>
ScummVM :: Bugs <https://bugs.scummvm.org>
ScummVM