[Scummvm-tracker] [ScummVM :: Bugs] #9705: FULLPIPE: AmigaOS: Crash on start

[ScummVM :: Bugs]

#9705: FULLPIPE: AmigaOS: Crash on start
------------------------+------------------------------
  Reporter:  raziel-    |      Owner:  (none)
      Type:  defect     |     Status:  new
  Priority:  high       |  Component:  Engine: Fullpipe
Resolution:             |   Keywords:  has-backtrace
      Game:  Full Pipe  |
------------------------+------------------------------

Comment (by raziel-):

 Some good soul helped with debugging the crash and i collected some more
 information.

 If another good soul from the ScummVM team could please take a look I'd be
 grateful:


 Fullpipe crashed for me as well.

 C++ new operator uses probably malloc too (what else?).

 Newlib and malloc started to ring some bells and it seems that ScummVM
 does sometimes realloc(). I wonder if this could break the allocation
 scheme? Here is some info: http://forum.hyperion-
 entertainment.com/viewtopic.php?f=14&t=1591&hilit=realloc

 I hacked custom malloc/calloc/realloc/free functions into ScummVM and
 Fullpipe started without crash 3 times. It crashed on each exit though -
 probably my fault :) By the way game sprite graphics looks buggy, perhaps
 endian-related.

 I think realloc issue is not fully confirmed but it's one potential clue.

 Tried also Fullpipe on Linux with Valgrind and Clang but didn't see much.

 ...

 Here is my patch. Currently I'm able to start and exit FP successfully:
 http://capehill.kapsi.fi/scummvm/

 There is a primitive attempt to profile drawing, shader compilation hack,
 memory allocation workaround in fullpipe.cpp and finally, some realloc()
 workaround attempt (but it didn't change a thing).

 Maybe it would be possible to capture all malloc/free/realloc/calloc calls
 and create a newlib testcase based on that data. Data might be captured
 over serial, for example.

 ...

 Could you maybe describe the malloc workaround a bit?
 What do you think is going wrong in the scummvm code and what are you
 "fixing".
 Maybe the devs at ScummVM don't want to investigate and simply add that
 exclusion for AmigaOS4, but history tells me they do want to support all
 platforms properly.

 ...

 Workaround just provides global symbols called malloc, free, calloc and
 realloc and linker then uses these to put the binary together. Newlib's
 symbols get by-passed: now heap allocations are requested directly from
 ExecSG.

 This impacts also other libraries that are linked with ScummVM and need
 those symbols, including SDL.

 I don't suspect ScummVM doing "wrong" currently.

 ...

 I recorded memory calls of Fullpipe startup and then tried to replay these
 with a pure Newlib-based test program and problem did NOT reproduce. It
 seems complicated.

 If there is a heap corruption by ScummVM then maybe it's somehow
 AmigaOS(+newlib)-specific and doesn't trigger Valgrind on Linux.

 On the other hand, after replacing Newlib-calls with IExec ones, issue
 didn't seem to occur.

 ...

 We could try to put canaries before and after each heap allocation. If we
 get lucky, one of the "canaries" gets hurt and then it may be possible to
 slowly debug more, knowing the size of the allocation for the next
 attempt.

 ...

 I see dead birds. Could be a bug on my side but after first tests, of
 150000 free'd allocations, there were about 300 allocations where the tail
 end was smashed. Have to investigate more.

 *** Birdwatch: good canaries 149822, bad canaries 340

 The canary patch (fullpipe.cpp part) is here if you want to play:
 http://capehill.kapsi.fi/scummvm/

 ...

 Debugged little bit more and put a trap instruction inside dead canary
 detector. It seems that there is some bitmap freed:

 {{{
 (0x64D93E40) [engines/fullpipe/fullpipe.cpp:162] scummvm:free()+0x1a4
 (section 8 @ 0x236C8)
 (0x64D93E90) [engines/fullpipe/fullpipe.cpp:162] scummvm:free()+0x1a4
 (section 8 @ 0x236C8)
 (0x64D93EC0) [engines/fullpipe/gfx.cpp:715]
 scummvm:_ZN8Fullpipe6BitmapD1Ev()+0x50 (section 8 @ 0x36EE8)
 (0x64D93EE0)
 scummvm:_ZN6Common14DefaultDeleterIN8Fullpipe6BitmapEEclEPS2_()+0x38
 (section 8 @ 0x3D548)
 (0x64D93F00) [engines/fullpipe/gfx.cpp:255]
 scummvm:_ZN6Common9ScopedPtrIN8Fullpipe6BitmapENS_14DefaultDeleterIS2_EEE5resetEPS2_()+0x34
 (section 8 @ 0x3D59C)
 (0x64D93F30) [engines/fullpipe/gfx.cpp:422]
 scummvm:_ZN8Fullpipe7Picture11freePictureEv()+0xf0 (section 8 @ 0x375BC)
 (0x64D93F50) [engines/fullpipe/gfx.cpp:412]
 scummvm:_ZN8Fullpipe7PictureD2Ev()+0x30 (section 8 @ 0x377C0)
 (0x64D93F70) scummvm:_ZN8Fullpipe11StaticPhaseD2Ev()+0x44 (section 8 @
 0xA47C8)
 (0x64D93F90) scummvm:_ZN8Fullpipe12DynamicPhaseD2Ev()+0x34 (section 8 @
 0xA482C)
 (0x64D93FB0) scummvm:_ZN8Fullpipe7StaticsD0Ev()+0x54 (section 8 @ 0xA48B0)
 (0x64D93FD0)
 scummvm:_ZN6Common14DefaultDeleterIN8Fullpipe7StaticsEEclEPS2_()+0x44
 (section 8 @ 0xA3A58)
 (0x64D93FF0) [engines/fullpipe/statics.cpp:145]
 scummvm:_ZN6Common8for_eachIPPN8Fullpipe7StaticsENS_14DefaultDeleterIS2_EEEET0_T_S8_S7_()+0x44
 (section 8 @ 0xA3AB4)
 (0x64D94010) [engines/fullpipe/statics.cpp:115]
 scummvm:_ZN8Fullpipe15StaticANIObjectD0Ev()+0x68 (section 8 @ 0xA21E0)
 (0x64D94030) [engines/fullpipe/scene.cpp:124]
 scummvm:_ZN8Fullpipe5SceneD0Ev()+0x118 (section 8 @ 0x84BBC)
 (0x64D94060) [engines/fullpipe/gameloader.cpp:421]
 scummvm:_ZN8Fullpipe10GameLoader11unloadSceneEi()+0xf8 (section 8 @
 0x2BB44)
 (0x64D94090) [engines/fullpipe/gameloader.cpp:387]
 scummvm:_ZN8Fullpipe10GameLoader12preloadSceneEii()+0x254 (section 8 @
 0x2CB2C)
 (0x64D940F0) [engines/fullpipe/gameloader.cpp:572]
 scummvm:_ZN8Fullpipe10GameLoader13updateSystemsEi()+0xc8 (section 8 @
 0x2CDC4)
 (0x64D94110) [engines/fullpipe/modal.cpp:233]
 scummvm:_ZN8Fullpipe10ModalIntro6finishEv()+0xc4 (section 8 @ 0x62268)
 (0x64D94130) [engines/fullpipe/modal.cpp:91]
 scummvm:_ZN8Fullpipe10ModalIntro4initEi()+0x54 (section 8 @ 0x64198)
 (0x64D94170) [engines/fullpipe/fullpipe.cpp:630]
 scummvm:_ZN8Fullpipe14FullpipeEngine12updateScreenEv()+0x13c (section 8 @
 0x24000)
 (0x64D94190) [engines/fullpipe/fullpipe.cpp:440]
 scummvm:_ZN8Fullpipe14FullpipeEngine3runEv()+0x5b8 (section 8 @ 0x25394)
 (0x64D94320) [base/main.cpp:327]
 scummvm:_Z7runGamePK6PluginR7OSystemRKN6Common6StringE()+0xd48 (section 8
 @ 0x80A0)
 (0x64D94840) [base/main.cpp:599] scummvm:scummvm_main()+0xde8 (section 8 @
 0x953C)
 (0x64DDA0E0) [backends/platform/sdl/amigaos/amigaos-main.cpp:79]
 scummvm:main()+0x260 (section 8 @ 0x6AF8)
 }}}

 It could be that heap corruption has something to do with endian issues.
 To fix endian issues, somebody would have to go through all file loading
 code in fullpipe engine.

 Here is one obvious bug (but doesn't fix graphics):
 https://github.com/scummvm/scummvm/blob/master/engines/fullpipe/utils.cpp#L260

 It gives just bogus reading for BE.****

-- 
Ticket URL: <https://bugs.scummvm.org/ticket/9705#comment:17>
ScummVM :: Bugs <https://bugs.scummvm.org>
ScummVM