mozfullscreen

PointerLock API Updates

A quick update on the Firefox PointerLock API implementation

Lets start with mochitests. While writing mochitests for pointerlock we stumbled on two problems

  1. Not being able to specify how many tests should run (different platforms were running different number of tests)
  2. Mochitest iframe not allowed to go fullscreen, making us run all the tests on a different window

David Humphrey came up with a solution for our first problem and added an “expect” functionality to the mochitest framework.
So now we can specify how many tests should occur when making asynchronous tests, for example:
SimpleTest.waitForExplicitFinish(3)
Bug 724578

For our second problem, I added the attribute mozallowfullscreen=true to the mochitest iframe that runs all tests.
I’m not sure if there was a specific reason for not allowing fullscreen on the mochitest iframe, but if it wasn’t it will simplify a lot writing tests for pointerlock
Bug 728893

Spec Updates

The spec had two major changes

  1. Switching from callbacks to events
  2. Moving functionality to the Document and Element

For example:
Everytime the pointer is locked/unlocked a mozpointerchange event will be dispatched to the document
A mozpointererror event will be dispatched if there are any errors while locking the pointer
Now It’s possible to access the element with the pointer locked via the document

  
 var div = document.createElement("div");

document.addEventListener("mozpointerlockchange", function (e) {  
 if (document.mozPointerLockElement === div) {  
 // Pointer is locked  
 }  
 }, false);

document.addEventListener("mozpointerlockerror", function (e) {

}, false);

document.addEventListener("mozfullscreenchange", function (e) {  
 if (document.mozFullScreen &&  
 document.mozFullScreenElement === div) {  
 div.mozRequestPointerLock();  
 }  
 }, false);

div.mozRequestFullScreen();

Instead of something like this:

 
 var div = document.createElement("div");

div.addEventListener("mozpointerlocklost", function (e) {  
 // Dispatched when pointer is unlocked  
 }, false);

document.addEventListener("mozfullscreenchange", function (e) {  
 if (document.mozFullScreen &&  
 document.mozFullScreenElement === div) {  
 navigator.mozPointer.lock(  
 div, // Element  
 function () {  
 // Success callback  
 },  
 function () {  
 // Failure callback  
 }  
 );  
 }  
 }, false);

div.mozRequestFullScreen();

Updating PointerLock API - Callbacks, Events and Threads

The PointerLock implementation of Firefox is going great, we are close to having the patch ready to land, maybe Firefox 13.

The work being done now is mainly some final touches, specially on the mochitests and on the API.

Recently the W3C PointerLock spec has been updated, the changes are the following:

  • When locking the mouse, dispatch pointerlockchange/pointerlockerror events instead of firing callbacks
  • Locking the pointer by requesting pointer lock on the target element
  • Adding a reference to the locked element in the Document
  • Exiting pointerlock by calling exitPointerLock on the Document

Those were significant changes, since it affected a big chunk of the code we had it implemented. However, I believe these updates to the API are beneficial, since with them developers will have an API similar to the fullscreen to work with.

The first bit I started working on was to dispatch the pointerlockchange/pointerlockerror instead of callbacks.

To Dispatch the events, the nsAsyncDOMEvent object was used:

   
 static void  
 DispatchPointerLockChange(nsINode* aTarget)  
 {  
 nsRefPtr e =  
 new nsAsyncDOMEvent(aTarget,  
 NS_LITERAL_STRING("mozpointerlockchange"),  
 true,  
 false);  
 e->PostDOMEvent();  
 }  

Same logic to dispatch the pointerlockerror and pointerlocklost

One of the good things about having to go back and rewrite some code, is the fact that opens the possibility to analyse some of the decisions made before.
Specifically in this case, the use of different threads when locking the pointer.
At first, the callbacks were being fired on a different thread so the execution wouldn’t hang, and the Lock method would be able to return as soon as possible and not make the user wait for a result.

Before, the logic for callbacks was mainly based off the nsGeoLocation implementation. However, now with the pointerlock API looking more like the fullscreen api I went and looked how they handle setting the element into fullscreen.
I had written a blog post a while back inspecting the fullscreen API, so even with the API receiving some changes it was easy to locate the code path for requesting fullscreen on an element.

Here is a simple diagram I drew

The diagram shows that once mozRequestFullScreen is called on an element, the method returns really fast and all the heavy processing happens on a separate thread.

On the other hand, this is how PointerLock does it:

On PointerLock, different from the FullScreen, the heavy processing happens on the main thread, and the new thread only handles the callback firing. Now switching to events, even less processing happens on the new thread, so that made me rethink the logic for locking the pointer.

I remember hearing that all the code that involves changing the presentation, it needs to happen on the main thread, so maybe that’s why we’re not spinning the pointerlock check/validation to another thread, since it involves changing the UI presentation by hiding the pointer if the lock is successful.

Another thing that caught my attention was the fact that on the fullscreen code, the nsCallRequestFullScreen object was dispatched to a new thread using NSDispatchToCurrentThread and on PointerLock we are using NSDispatchToMainThread


NSDispatchToCurrentThread
NSGetCurrentThread
nsThreadManager::GetCurrentThread NS_DispatchToMainThread
mMainThread

Firefox Bug 714071

Working on Bug 714071 introduced me to another layer of Firefox.
So far all I’ve been doing was working with c++ code, specifically related to the MouseLock API.
Bug 714071 on the other hand was focused on the js layer.

A brief summary of the bug:

Fix a problem with the Statistics video control


When the showing statistics option of a video was on, and the fullscreen was toggled the video would stop displaying the statistics but the menu would not be updated.
The goal for the bug was to keep showing the statistics when toggling between fullscreen.

The code that displays the statistics on a video is the following:

   
 showStatistics : function(shouldShow) {  
 if (this.statsInterval) {  
 clearInterval(this.statsInterval);  
 this.statsInterval = null;  
 }

 if (shouldShow) {  
 this.video.mozMediaStatisticsShowing = true;

 this.statsOverlay.hidden = false;  
 this.statsInterval = setInterval(this.updateStats.bind(this), this.STATS_INTERVAL_MS);  
 this.updateStats();  
 } else {  
 delete this.video.mozMediaStatisticsShowing;  
 this.statsOverlay.hidden = true;  
 }  
 },

Everytime the video was loaded on the page, or toggled in fullscreen, the init method would be called and an event listener would be attached to listen for the “Show Statistics” option click. However, the init method would initialize the video with a fresh config, so if the statistics were being displayed it would be hidden after switching between fullscreen mode.

Solution

The solution was to add a check on the setUpInitialState method to activate the statistics on the video if they were being displayed before toggling the fullscreen.

   
 if (this.video.mozMediaStatisticsShowing) {  
 this.showStatistics(true);  
 }  

That apparently solved the problem and the statistics are preserved even when toggling fullscreen.

Working on this bug made me very curious. How could some javascript code interact with c++ at run time.
I knew about the XPCOM object model used in Firefox and lately I’ve started to read more about XUL and the Gecko engine . This was the perfect time to start digging more deep and learn more about the Firefox foundations.
Using the videocontrols as a starting point I went to mxr and started searching some code.
From my initial search I think I found where the controls were loaded in c++. Now I just need to figure it out how all that happens :)

videoscontrols.xml – define the video controls

nsCOMPtr mVideoControls;
nsVideoFrame::CreateAnonymousContent – where the menu gets attached to the video element

nsNodeInfoManager – apparently used to load the video controls as well the poster image
nsNodeInfoMangager::GetNodeInfo – looks like the loading happens here

Manage XPCOM objects?
PLHasTableLookup
PLHasEntry
PL
HasTableAdd

NSTrustedNewXULElement – where the XUL element gets created and casted to an nsIContent
NS
TrustedNewXULElement declaration
NS_TrustedNewXULElement definition

TODO:
Keep digging and find how the pieces fit together!