Do you want to use hot keys: Ctrl+F4, F5, Ctrl+N, F12, Ctrl+W? [SOLUTION?] [FEATURES?]

[qyloxe]

Let’s assume, that you have an app with qtabs, qtrees. You want to close your active qtab after pressing Ctrl+F4 or Ctrl+W, you want to add tab with Ctrl+N, switch tabs with Ctrl+Tab, and finally you want to press F5 when your qtree is active and let all the nodes be refreshed instead of a whole window refresh. Well, you cant. But…

1. You cant, because those are “special” keys, and they are captured by browsers. The more or less complete list is here:
   https://www.w3schools.com/tags/ref_keyboardshortcuts.asp

2. You can, because new browsers have a “keyboard API” and during special circumstances, you can capture almost all keys, prevent default event processing and do what you want:
   https://developer.mozilla.org/en-US/docs/Web/API/Keyboard

3. Yay, but what are those “special circumstances”? The idea was, that some apps (games, 3D modelers) would need access to all the keys (almost). The trick is, that you need to put your app in full screen mode and you need to “lock” the keyboard. Then you will have access to all the keys (almost).

4. Why “almost”? Becuase browsers (or OSes) still will filter some keys. For example it could be Ctrl+Alt+Del on Windows, but also right menu key (on windows) or Esc pressed longer (this is a way to exit full screen mode in browsers). The full list of keys available to your specific user depends probably on browser, version, os and full moon phase.

5. What are steps to use those technologies in my Quasar app?

   • first we need to change programmatically view to full screen
   • then we need to lock the keyboard
   • then we need to capture the keys
   • … and process them when they are pressed by user
   • when the user decides to leave full screen mode, we need to uncapture keys, unlock keyboard and leave the full screen mode

6. Why full screen mode? Because locking and capturing special keys works ONLY in full screen mode. This is even more complicated, because this has to be a “programmatic” full screen mode and not “browser default” full screen mode.

7. What is the difference between “programmatic” and “browser default” full screen modes? Programmatic uses full screen API:
   https://developer.mozilla.org/en-US/docs/Web/API/Fullscreen_API
   Default uses browser means to change the view, be it by pressing F11 key or choosing some option on menu. Those are different modes, and keyboard locking is available only (as for now) in full screen mode activated programmatically. This is even further complicated in some browsers (most, all?), because this activation needs to be a direct descendant of user click o keypress javascript handler.

8. Does that mean that if user presses F11 then my app will not have a means to knowing that it works in full screen now? Yes, this is correct. As for now, the full screen API, all those change events etc. works only in programmatic mode. In effect, there will be possible for user to press F11 and then click your button “go to FSM” and then… depending on browser, user would need to maybe even click your button again and press F11 again for leaving FSM. But it depends, of course.

9. We’re lucky - Quasar has a full screen support - yes, it is here:
   https://quasar.dev/quasar-plugins/app-fullscreen
   And we will use this API, but IMHO the code for those functionalities could be slightly changed and I wonder if this will work on every platform/browser. More on this later.

10. How can I test this? I created a codepen for testing of special keys capture in full screen mode with Quasar. It is here:
    https://codepen.io/qyloxe/pen/mdJGPbR?editors=1010

11. Feel free to change code, BUT running of this test is only possible from main windows frame. If the navigator.keyboard.lock function is not run from main window context (it is run as iframe in codepen) then there is an error in console “Uncaught (in promise) DOMException: lock() must be called from a top-level browsing context.”. Because of that you should use a codepen feature “debug mode”, available here:
    
    The codepen will be opened in another tab, where it will be a main window, and all should work OK. This is the link to the test in debug mode:
    https://cdpn.io/qyloxe/debug/mdJGPbR/gaMeYzDRoLLM

12. The main test feature is checking what happens when user presses keys and what happens when specifically are pressed combinations: “Ctrl+N” (new window), “Ctrl+Tab” (next browser tab).

13. Test scenario #1 - what happens when user presses F11? Ideally, we want to know, if browser is in full screen mode activated by user. Unfortunately, all three tests for that are showing (chrome, windows) false. There is no way for app to know (as for now) that it works in browser activated full screen mode. Or is there?

14. Test scenario #2 - what happens during browser FSM when user clicks on “Toggle full screen” q-button? How are state flags changed, is it possible to exit from this “double” mode? (yes it is possible - firstly user needs to press Esc longer, and then press F11).

15. Test scenario #3 - what happens when user presses keys and our “special” combinations (Ctrl+N, Ctrl+Tab) in FSM and without FSM.

16. Test scenario #4 - what happens when user presses keys with enabled prevention of default keyboard events. This scenario is what we want to achieve ie we want to capture keydowns and bind our own actions to them.

17. Why there are “mounted” and “onbeforedestroy” events? This code comes from:
    https://github.com/mirari/vue-fullscreen/blob/master/src/utils.js#L50
    It is a simple way to bind to full screen change events in cross browser way. The docs are here:
    https://www.w3schools.com/jsref/event_fullscreenchange.asp
    and here:
    https://developer.mozilla.org/en-US/docs/Web/API/Document/fullscreenchange_event

  mounted () {
    document.addEventListener('fullscreenchange', this.onFullScreenChange)
    document.addEventListener('mozfullscreenchange', this.onFullScreenChange)
    document.addEventListener('MSFullscreenChange', this.onFullScreenChange)
    document.addEventListener('webkitfullscreenchange', this.onFullScreenChange)
    this.checkIsInFullScreenElement()
  },
  beforedestroy () {
    document.removeEventListener('fullscreenchange', this.onFullScreenChange)
    document.removeEventListener('mozfullscreenchange', this.onFullScreenChange)
    document.removeEventListener('MSFullscreenChange', this.onFullScreenChange)
    document.removeEventListener('webkitfullscreenchange', this.onFullScreenChange)
  },

Basically we want to know whether broswser is in full screen mode, even when user presses F11. Unfortunately both checks - the all events and the Quasar way from AppFullScreen are failing to recognize that. The Quasar way of binding to these events is a little different - I think it is a good occasion to check if the code in Quasar is still relevant:
https://github.com/quasarframework/quasar/blob/dev/ui/src/plugins/AppFullscreen.js#L73
(hint: the missing mozilla event binding and additional prefix “on”). As of now, this API is supported on Mozilla platform:
https://caniuse.com/#feat=fullscreen

18. Why we want to check for active FSM with specific button? It is about this code:

    checkIsInFullScreenElement () {
      this.isactivefullscreenelement = !!(document.fullscreenElement || document.mozFullScreenElement || document.webkitFullscreenElement || document.msFullscreenElement)
    },

Well, I wasn’t sure, but I wanted to check if this code:
https://github.com/quasarframework/quasar/blob/dev/ui/src/plugins/AppFullscreen.js#L68
till line 80, will correctly check for active FSM. I wanted to check more frequently than Quasar and also on Mozilla platform. Unfortunately, there is no luck on windows and chrome/firefox. Maybe for someone on Apple or Linux it would get some other results.

19. Why this watcher uses Quasar preferred way of event binding and not a custom one? This is the code:

  watch: {
    '$q.fullscreen.isActive': function (val) {
      console.log('fullscreen.isActive', val)
      if (val) {
        if (navigator.keyboard && navigator.keyboard.lock) {
          navigator.keyboard.lock()
          this.iskeyboardlocked = true
          document.addEventListener('keydown', this.onGlobalHotKeyDown, true)
        }
      } else {
        document.removeEventListener('keydown', this.onGlobalHotKeyDown, true)
        navigator.keyboard.unlock()
        this.iskeyboardlocked = false
      }
    }
  }

Well, as we 've seen from former tests, there is no difference in recognizing FSM state (chrome, windows) so it is perfectly OK to use a preferred way of FSM state subscription:
https://quasar.dev/quasar-plugins/app-fullscreen#Watching-for-fullscreen-changes
What we want to achieve is just bind the keyboard hook and lock keyboard. BEWARE this code is NOT production quality, because proper binding and hooking are promisified methods.

20. Why this keyboard hook method works this way? Here is the code:

    onGlobalHotKeyDown (event) {
      if (this.preventdefault) {
        event.preventDefault()
      }
      let msg = `key: ${event.key}, code: ${event.code}, ctrlKey: ${event.ctrlKey}, shiftKey: ${event.shiftKey}, altKey: ${event.altKey}, metaKey: ${event.metaKey}, charCode: ${event.charCode}, keyCode: ${event.keyCode}`
      console.log(msg)
      this.$q.notify({message: msg, group:false})
      if (event.code === 'KeyN' && event.ctrlKey) {
        this.$q.notify({message:'Ctrl+N', color:'positive', group:false})
      }
      if (event.code === 'Tab' && event.ctrlKey) {
        this.$q.notify({message:'Ctrl+Tab', color:'positive', group:false})
      }
    }

In Quasar this method should be bound to max top level element of the app. Maybe layout or toolbar or as app extension even. With this event we can capture almost all browser “system keys”. The example shows the processing of Ctrl+N and Ctrl+Tab. The key part of this hook behaviour is a decision - do we call event.preventDefault() or not.

21. What we’ve learned? That it is possible to hook browser “system” keys. That it is quite hard as of now and upredictable. That user, who wants to use those key combinations should know that pressing F11 is different than clicking app “toggle fullscreen” button. That exit from fullscreen is possible without Quasar knowledge.

22. What Quasar features are possible with that knowledge? Well, I do believe, that Quasar needs keyboard navigation for all its components. Not only interactive ones but most of them. Writing good keyboard navigation in non trivial apps is the most important and very, very hard. Without support at the framework level it will be an almost impossible task.

23. What could be added to Quasar to better support advanced app navigation? We are not talking only about keyboard hooks. Lets assume that we have those QTabs and QTree from the beginning of this post. User may want to add new tab and refresh tree. She can press Ctrl+N and a new q-tab should be created by app and press F5 and only q-tree items should be refreshed. So Quasar NEED to know which of all app component is active one in terms of “new tab” action and “refresh action”. As for now, there is no concept of “active” component in Quasar. But this is not only about keyboard. User may want to use a touch gesture for “new tab thumb slide” and “refresh my component quadruple tap”. Another user may want to use voice and just say “add new tab” and “refresh this tree”. All those intentions generate global events (key, voice, touch) which should translate to actions in specific Quasar components.

24. Why this post is sooooo long? Sorry… I just wanted to research Full Screen API and global keyboard hooks, and thought that maybe it will be useful to somebody. In the process I realized, that I like Quasar even more and that this framework has so enormous potential for the future. The possibility to use “user intentions” at the framework level (“add tab”, “refresh”) would be IMHO a game changer for many, many developers. Cheers!

[metalsadman]

@qyloxe great write up.