[Scummvm-tracker] [ScummVM :: Bugs] #12942: Apple keyboard cursor keys don't work, accessibility bar covers bottom of screen

[ScummVM :: Bugs]

#12942: Apple keyboard cursor keys don't work, accessibility bar covers bottom of
screen
-------------------------------------+------------------------
Reporter:  Purduecoz                 |       Owner:  (none)
    Type:  defect                    |      Status:  new
Priority:  high                      |   Component:  Port: iOS
 Version:                            |  Resolution:
Keywords:  iOS 15, keyboard, cursor  |        Game:
-------------------------------------+------------------------
Comment (by criezy):

 I only have a quite old iPad that is stuck to iOS 12, but with that one I
 checked that the arrow key from an Apple bluetooth keyboard work as
 expected in King's Quest II. And I have no idea why this would not work on
 iOS 15 or with other keyboards.

 One thing to keep in mind is that the keyboard needs to be enabled in
 ScummVM (for example with a pinch in gesture) before it can be used. But
 since you mention issues with the accessibility bar at the bottom of the
 screen, I assume you knew that already.

 While testing the arrow keys I noticed that the escape key from the
 keyboard was not working. And it was indeed never implemented; we only add
 support for using the ` key as escape (which is useful for keyboards that
 do not have an Escape key). But now if there is an Escape key it also
 works properly.

 Shortcut with modifiers was never implemented, so it is expected that they
 do not work. Implementing that one a case by case basis would be possible,
 although not trivial, but we cannot do it for every possible keyboard
 shortcuts. I am not sure if there is a way with the iOS API to handle
 generic key press with modifier.

 Regarding the "accessibility bar", I cannot reproduce either, but then I
 am not sure what you really mean. I have never heard of an accessibility
 bar, so I don't know if this is something new in recent iOS versions, or
 if you mean either the input assistant bar or input accessory bar.
  - The input assistant bar is a standard bar that typically has undo, copy
 and paste buttons.
  - The input accessory bar is a custom bar that applications can
 implement. In ScummVM we implemented such a bar to have function keys,
 arrow keys, escape keys and more.

 I have attached screenshots for those two. Can you clarify if you mean one
 of those two or something else?

 In ScummVM we try to always hide the input assistant bar. This only works
 on iOS 9 or above (which is your case) when compiled with the iOS 9.0 or
 above SDK. I assume that this is also your case. You can see
 https://github.com/scummvm/scummvm/commit/3baad9e2a for details. If that
 is the bar you see, we may have an issue with this code. The
 `UITextInputAssistantItem` documentation suggests that setting the
 `leadingBarButtonGroups` and `trailingBarButtonGroups` to `nil` as well as
 setting the autocorrection type to `UITextAutocorrectionTypeNo` should
 hide the bar. And I just noticed that while we do the later, we set
 `leadingBarButtonGroups` and `trailingBarButtonGroups` to an empty array
 and not to nil. That works on iOS 12 at least, but maybe that matters for
 iOS 15. Or maybe there is something more that needs to be done. But not
 having such a device this is not something I can experiment with myself.

 We also try to hide the input accessory bar if a physical keyboard is
 connected. They is no API to know if a physical keyboard is connected so,
 so we have to use some guesswork. You can see some details on
 https://github.com/scummvm/scummvm/commit/cfd66173. The code that matters
 is at the bottom in `prepareKeyboard`. Basically it looks at the height of
 the keyboard frame. Because when a physical keyboard is connected only the
 bars are visible, the height of the frame is a lot smaller. So we assume
 that if this is less than 140 point we can hide the bar. If this is the
 bar you see, maybe this guesswork does not work on iOS 15.

 But I am actually wondering if it might not be better to have a user
 option to show or not the input accessory bar instead of some guesswork to
 show it when there is no physical keyboard and hide it when there is one.
 My understanding is that iPad keyboard does not have function and escape
 key. So users might want to see this bar even if they have a physical
 keyboard (and it also has a key to show the ScummVM menu).
-- 
Ticket URL: <https://bugs.scummvm.org/ticket/12942#comment:3>
ScummVM :: Bugs <https://bugs.scummvm.org>
ScummVM