video

Bug 713608 Update

One of the bugs I’m currently working on is: Bug 713608 – HTML5 Video controls are missing in Fullscreen
A quick overview of the bug:
This bug involves working with html5 video controls.
The goal of the bug is to enable the stock controls when the video enters in fullscreen via the context menu. For example: the controls are hidden when the video isn’t in fullscreen, then the user right clicks on the video and selects enter fullscreen. When the video switches to fullscreen mode, the controls will be displayed, even though they weren’t before toggling fullscreen, and after exiting fullscreen mode, if the controls were disabled in the first place, disable them again.

After working on Bug 714071 – The Show Statistics setting is not preserved when toggling the full screen that also involved working on the videocontrols.xml file, I thought would be good to keep digging how XBL bindings work on firefox.
So far Bug 713608 has proven to be more complicated than I thought it would be.
At the beginning, I followed the same logic used on Bug 714071 and tried to find the method that shows/hides the controls. However, different from the Statistics menu option, it seems that hiding the controls uses a different approach.
This is the code that displays the statistics of a video

   
 showStatistics : function(shouldShow) {  
 if (this.statsInterval) {  
 clearInterval(this.statsInterval);  
 this.statsInterval = null;  
 }

 if (shouldShow) {  
 this.video.mozMediaStatisticsShowing = true;

 this.statsOverlay.hidden = false;  
 this.statsInterval = setInterval(this.updateStats.bind(this), this.STATS_INTERVAL_MS);  
 this.updateStats();  
 } else {  
 delete this.video.mozMediaStatisticsShowing;  
 this.statsOverlay.hidden = true;  
 }  
 },  

But I couldn’t find the equivalent method for the video controls.

Browsing through the code, I was able to find the methods triggered by the event listeners for mousemove/over/out used by the video element:

MouseMove

   
 onMouseMove : function (event) {  
 // If the controls are static, don’t change anything.  
 if (!this.dynamicControls)  
 return;

 clearTimeout(this._hideControlsTimeout);

 // Suppress fading out the controls until the video has rendered  
 // its first frame. But since autoplay videos start off with no  
 // controls, let them fade-out so the controls don’t get stuck on.  
 if (!this.firstFrameShown &&  
 !(this.video.autoplay && this.video.mozAutoplayEnabled))  
 return;

 this.startFade(this.controlBar, true);  
 // Hide the controls if the mouse cursor is left on top of the video  
 // but above the control bar and if the click-to-play overlay is hidden.  
 if (event.clientY < this.controlBar.getBoundingClientRect().top && this.clickToPlay.hidden) {  
 this._hideControlsTimeout = setTimeout(this._hideControlsFn, this.HIDE_CONTROLS_TIMEOUT_MS);  
 }  
 },  

MouseInOut

   
 onMouseInOut : function (event) {  
 // If the controls are static, don’t change anything.  
 if (!this.dynamicControls)  
 return;

 clearTimeout(this._hideControlsTimeout);

 // Ignore events caused by transitions between child nodes.  
 // Note that the videocontrols element is the same  
 // size as the *content area* of the video element,  
 // but this is not the same as the video element’s  
 // border area if the video has border or padding.  
 if (this.isEventWithin(event, this.videocontrols))  
 return;

 var isMouseOver = (event.type == "mouseover");

 // Suppress fading out the controls until the video has rendered  
 // its first frame. But since autoplay videos start off with no  
 // controls, let them fade-out so the controls don’t get stuck on.  
 if (!this.firstFrameShown && !isMouseOver &&  
 !(this.video.autoplay && this.video.mozAutoplayEnabled))  
 return;

 if (!isMouseOver) {  
 this.adjustControlSize();

 // Keep the controls visible if the click-to-play is visible.  
 if (!this.clickToPlay.hidden)  
 return;

 // Setting a timer here to handle the case where the mouse leaves  
 // the video from hovering over the controls.  
 this._hideControlsTimeout = setTimeout(this._hideControlsFn, this.HIDE_CONTROLS_TIMEOUT_MS);  
 }  
 },  

My plan was to find the event listeners that trigger the display of the video controls, then find how those events are added/removed to element.

I found where the listeners were defined

   
   
   
 if (!this.isTouchControl)  
 this.Utils.onMouseInOut(event);  
   
   
 if (!this.isTouchControl)  
 this.Utils.onMouseInOut(event);  
   
   
 if (!this.isTouchControl)  
 this.Utils.onMouseMove(event);  
   
   

However, I couldn’t find where they are added/removed from the video element.

I feel that I’m getting close to finding a solution.

Big thanks to Jared Wein (jaws) for helping me with this and other bugs :)

Firefox Bug 714071

Working on Bug 714071 introduced me to another layer of Firefox.
So far all I’ve been doing was working with c++ code, specifically related to the MouseLock API.
Bug 714071 on the other hand was focused on the js layer.

A brief summary of the bug:

Fix a problem with the Statistics video control


When the showing statistics option of a video was on, and the fullscreen was toggled the video would stop displaying the statistics but the menu would not be updated.
The goal for the bug was to keep showing the statistics when toggling between fullscreen.

The code that displays the statistics on a video is the following:

   
 showStatistics : function(shouldShow) {  
 if (this.statsInterval) {  
 clearInterval(this.statsInterval);  
 this.statsInterval = null;  
 }

 if (shouldShow) {  
 this.video.mozMediaStatisticsShowing = true;

 this.statsOverlay.hidden = false;  
 this.statsInterval = setInterval(this.updateStats.bind(this), this.STATS_INTERVAL_MS);  
 this.updateStats();  
 } else {  
 delete this.video.mozMediaStatisticsShowing;  
 this.statsOverlay.hidden = true;  
 }  
 },

Everytime the video was loaded on the page, or toggled in fullscreen, the init method would be called and an event listener would be attached to listen for the “Show Statistics” option click. However, the init method would initialize the video with a fresh config, so if the statistics were being displayed it would be hidden after switching between fullscreen mode.

Solution

The solution was to add a check on the setUpInitialState method to activate the statistics on the video if they were being displayed before toggling the fullscreen.

   
 if (this.video.mozMediaStatisticsShowing) {  
 this.showStatistics(true);  
 }  

That apparently solved the problem and the statistics are preserved even when toggling fullscreen.

Working on this bug made me very curious. How could some javascript code interact with c++ at run time.
I knew about the XPCOM object model used in Firefox and lately I’ve started to read more about XUL and the Gecko engine . This was the perfect time to start digging more deep and learn more about the Firefox foundations.
Using the videocontrols as a starting point I went to mxr and started searching some code.
From my initial search I think I found where the controls were loaded in c++. Now I just need to figure it out how all that happens :)

videoscontrols.xml – define the video controls

nsCOMPtr mVideoControls;
nsVideoFrame::CreateAnonymousContent – where the menu gets attached to the video element

nsNodeInfoManager – apparently used to load the video controls as well the poster image
nsNodeInfoMangager::GetNodeInfo – looks like the loading happens here

Manage XPCOM objects?
PLHasTableLookup
PLHasEntry
PL
HasTableAdd

NSTrustedNewXULElement – where the XUL element gets created and casted to an nsIContent
NS
TrustedNewXULElement declaration
NS_TrustedNewXULElement definition

TODO:
Keep digging and find how the pieces fit together!