New DOM bindings

In my previous post here I walked through some of the history behind the DOM bindings in the Mozilla code. This post is a followup to that where I’ll walk through the “new” bindings that we already have in part, and are working on completing over the coming months.

One of the big performance problem areas with the old style bindings was performance of accessing array like DOM objects, i.e. accessing properties of live lists of DOM objects such as NodeList (element.childNodes[n]), HTMLCollection (form.elements[s]), etc. Not only was that slow, but it also didn’t always work right, especially when dealing with lists from which objects were removed. The reason there was that the way those bindings were set up (and had to be set up given the binding mechanism worked) meant that any property that’s ever been touched from JS will forever be present on the JS object for that list. Its value will update if the underlying live list changed (may even become null), but if an entry in the list was no longer present, the property for that entry still existed on the JS object. This is something that’s hard (i.e. impossible) to do right with DOM bindings that are based on JSClass hooks, which all our bindings so far have been.

In mid 2011 we started thinking hard about how to move forward on this particular front, and faster bindings overall too. We chose to tackle the list like objects first for two main reasons, the current ones were both slow and broken, and there were relatively few of them so the change would be relatively isolated. The answer to the problems we were having with list like objects was to use the then fairly new support for JS proxies. Proxies are basically a new way to give JS objects custom behavior, one that’s in particular much more suited for live list like objects like NodeList etc. In May of 2011 we started to plan this work when Peter Van der Beken, Blake Kaplan, and Andreas Gal got together in Paris, France, (with Boris Zbarsky on many hours of video conferencing) to design new DOM bindings for our list like DOM objects. The outcome of this work was a code generator, one that read a configuration file, the relevant IDL files, and generated C++ template specializations of the ListBase template class implementations of proxies for list like objects. That work landed in August of 2011. It gave us better performance than we’d ever had for list like objects, and it also fixed the broken behavior of the old bindings. And while this work did give us performance improvements, there’s still room to improve in the new system as well, in particular with how the JS engine JIT works with JS proxies. So more to come there.

The last remaining large piece here is to write new DOM bindings for our regular DOM objects (i.e. objects that are not list like). That work (internally known as “Paris Bindings”) will be split up into several pieces as well, and the first one of that started early this year. In January of 2012 a group of people (Boris Zbarsky, Blake Kaplan, Mounir Lamouri, Olli Pettay, Kyle Huey, Bobby Holley, Ben Turner, and myself, plus Peter Van der Beken joining us on the last day that week) got together, again in Paris, France, to discuss the next piece of this work. The outcome of that set of meetings was a plan to tackle the bindings for a single object (plus a few dependent ones) as the first step. The object we chose was XMLHttpRequest, reason being that it’s a fairly self-contained object, it’s one that exists both on the main-thread and in DOM workers, and yet it’s a non-trivial object that poses some challenges (the fact that it’s an EventTarget being one of those challenges). We also had a plan on how to do that, part of which was preliminary code, even.

We then set out to hack this up, with Kyle Huey working on a WebIDL parser. Bobby Holley was working on the code generator (which uses the WebIDL parser), Boris Zbarsky working on argument and “this” unwrapping, return value wrapping, prototype setup, and a bunch of other low level stuff. Ben Turner worked on getting DOM workers ready for the new bindings (which was a significant amount of work given the then current worker bindings which were all hand-written JSClass based bindings). The rest of us helped out with various tasks as they came up, including writing some WebIDL files. In the following weeks these new bindings started to take shape, first we had the basic utilities working, then prototype setup, followed by basic code generation for methods/getters/setters. Things were starting to look pretty functional, even if there were still missing pieces to the puzzle. It was Peter Van der Beken who put the final pieces of the puzzle in place, and got a functioning build with new bindings in place for XMLHttpRequest. Then we of course found a bunch of details that were not right once we were able to push this stuff through the try  server etc, so a good bit of cleanup and missed pieces needed to be sorted out before finally landing this work at the end of March 2012. The landing went very smoothly, with one silly missing null check that Kyle Huey fixed being the most serious fallout. After this change having been in the tree for a bit more than a week now there’s some other stuff that was found, but all things considered, things are looking very good.

Writing the code generator for these new bindings brought up a set of interesting questions, one of which was what the signature of the method that the generated code would call should look like. We’ll be calling directly into a concrete class here (avoiding virtual calls when possible), so no need for any XPCOM goop, but at the same time we often do need to deal with propagating errors from the DOM method. That means we need an error code (i.e. an nsresult return value or out parameter), yet there’s plenty of cases where we simply don’t need one and requiring that the out parameter, or return value, always be present adds overhead we simply don’t want. What we finally settled on there was to have a nsresult out parameter, passed by reference, in the cases where we generate calls to fallible methods (which at this point can be controlled in the configuration file). That’s something we’ll likely move from the configuration file into per method/property annotations in the WebIDL itself as we do some pending cleanup now that the basics are working.

There’s also the issue of the name of the method that gets called. Since we’re calling methods on a concrete class, and that concrete class is likely to also inherit a bunch of XPIDL DOM interfaces (for now at least), we’ll likely run into conflicts where the signature of a method we’ll be invoking from the generated code causes conflicts with existing methods. There’s no obvious answer to this, but the fact that we return the error code in an out parameter (assuming there is one) helps as the signature in that case is likely different between the XPIDL method and the method we’ll call from the generated code. But if we’re dealing with an infallible method, like say xhr.abort(), then there will be a conflict due to the only difference between what our generated code will call and what we currently have from the implementation of our IDL interface is the return type. We decided to deal with this problem by renaming the binary name in the old XPIDL files, since we already had support for doing that (i.e. the binaryname attribute in IDL).

Another piece of this puzzle was making XrayWrapper work right with these new bindings. The existing XrayWrapper hooked into XPConnect and used its machinery to call the “original” unaltered method/getter/setter, but for our new bindings, XPConnect doesn’t know anything about our objects (or won’t, once we get further along here). The answer here was to write another flavor of our XrayWrapper, called XrayDOM, that specializes XrayWrapper such that it does the right thing on the new binding objects.

And the existence of QueryInterface() on the old bindings objects caused us problems too, particularly in our tests. The solution to this problem for now was to add a method named QueryInterface() on the new bindings as well, but calling it is a no-op. This of course led to callers of QueryInterface() expecting properties on the interface they were QueryInterface()’ing to to exist on the returned object (i.e. the binding object itself), fortunately the number of interfaces and properties we had to deal with was very low, and they’re only relevant to chrome code and thus only appear on chrome XMLHttpRequest objects, and that was enough to pass all our tests, and has yet shown any problems in the wild either. This is us paying the price of ever exposing QueryInterface() on DOM objects. Going forward this is something we need to keep an eye on, as this is likely to creep up with more objects, including some DOM elements. Our long term goal is to remove this completely, at least on non-chrome objects.

So what do these new bindings look like you ask? Well, we have a set of common utilities that are shared, and per DOM type we have roughly one JSClass, two lists of methods and properties (one for chrome, one for untrusted code, because there are things we expose to chrome that we don’t expose to untrusted code), and a list of constants (I’d love to link to examples here, but they’re all generated at build time so I can’t easily do that). And double that (minus the chrome only stuff) for each DOM type that’s available in DOM workers. Plus of course the implementation of the methods, getters, and setters. As you can tell if you start reading some of the actual code here (which ends up living in $objdir/dom/bindings/*.{cpp,h}), there’s very little XPCOM stuff left here. We don’t use XPIDL files, we use WebIDL, meaning that it’ll certainly be possible to write a DOM class for which there’s no XPIDL file in this setup. The bindings can simply call straight into methods on the concrete class, which is named in the configuration file. Inheriting from nsWrapperCache is a requirement for using the new bindings, that’s how we map from C++ object to JS object, and having a link from the DOM object to its parent (and ultimately its window or global object) is also a requirement in this new setup so that we can tell which JS compartment to create the binding object in.

We’re also still ironing out exactly what the configuration file and the WebIDL files will look like long term, but what we have for now is a good start, enough to learn how to move forward here. We’re also going to be consolidating some of the code generators we’ve grown over time, and eventually we’ll be able to remove support for quick stubs, slim wrappers, and probably a bunch of other optimizations that have been made in the XPConnect code too. We also went a bit overboard with the use of C++ namespaces in the generated code, so we’re cleaning that up now as well to be a bit more developer (and debugging) friendly.

The next steps here are to first do new bindings for a few more objects, that’ll let us iron out more details in our WebIDL parser and code generator by throwing more WebIDL at the system. The objects we’ve chosen for that are the canvas objects (because they’re very performance sensitive) and the CSS2Properties (because it poses several challenges on its own given how it is implemented, and it has a lot of properties whose presence is controlled by preferences). And Boris Zbarsky is also looking into the WebGL bindings, because they too are very performance sensitive. I would expect that we’ll land the above over the next weeks or couple of months.

After that the largest remaining piece is to write new bindings for our actual DOM nodes and the other miscellaneous classes. There’s a lot of those, so that’ll be a big chunk of work as well, but hopefully by the time we get there the infrastructure for all this will be pretty solid and we can focus on the bindings rather than the supporting code.

Posted in mozilla | Tagged | 17 Comments

History of Mozilla’s DOM bindings

The Mozilla project has gone through a number of different JS bindings for the DOM in Mozilla. This post is basically a trip down memory lane, walking through how JS running in the SpiderMonkey JavaScript engine in Mozilla has communicated with the C++ DOM throughout the years. But before I get into that, let’s clarify what DOM bindings are.

DOM bindings are the pieces of code that sit in between the JavaScript engine and the native DOM implementation. This goes for all browsers out there, there’s a JS engine of some sort, interpreting or executing JIT:ed JavaScript code, and there’s a DOM implementation, written in C++ in all popular browsers today. DOM bindings is the glue in between those two worlds.

Looking back to when the Mozilla source code was released, back in 1998, the DOM bindings that existed at that point were generated from IDL files that described the various DOM APIs. That generation happened using a compiler called midl, and it was part of the Mozilla build system at the time (though it didn’t run by default), but it was a compiler that was only runnable on Windows. If you were developing on other platforms you needed to get your hands on a Windows computer in order to change or add a DOM API. The output of running midl was a C++ function per method/getter/setter in the DOM API, plus some other stuff to get constants and other details right. The methods/getters/setters that were generated this way did what you’d expect, found the pointer to the C++ object that was being touched, did the appropriate argument conversions, made the call to the actual C++ method, and then potentially converted the result to a type that was suitable to pass back to JS, and in the midst of that it also dealt with exception throwing in case we ran into problems, or the caller called a DOM method incorrectly. The generated code was then committed to the CVS source repository so that others who were not working on DOM APIs didn’t need to re-generate the bindings every time. The generated code also grew to be a significant amount of code, to the order of 2MB of compiled code if memory serves me right. This was a significant chunk of code back in the late 90’s.

Then, late in the year 2000, it was decided that the current way of doing DOM bindings wasn’t good enough. It was a lot of (compiled) code, used a lot of memory (both on disk and in memory when running Mozilla), and it wasn’t very hacker friendly (everyone didn’t have a Windows development environment or easy access to one, etc). The then better alternative was to use a piece of technology that was being developed at Netscape called XPConnect. XPConnect was a generic way of communicating between JS and C++ (and vice versa). That meant there would be no more need for the then clunky intermediate step of generating new DOM bindings code whenever a developer changed a DOM API. Instead, we’d just use XPConnect which was then already capable of making an XPCOM object (which the DOM objects in Mozilla already were) look like a JavaScript object, and also make a JavaScript object look like an XPCOM C++ object. The problem then was that XPConnect was a bit too closely tied to how XPCOM worked, meaning that it naturally dealt with XPCOM interfaces, JS code that used XPConnect would need to call QueryInterface() in order to interact with multiple interfaces implemented by a given object. IOW, not how the Web and the DOM worked. That meant that XPConnect needed a good bit of work in order to be able to automatically reflect multiple interfaces on a single JavaScript object, and also a bunch of other work to get all kinds of details right for the web. This resulted in support for what we called scriptable helpers, which is C++ code that can be used to hook into the guts of how XPConnect reflects a C++ object to JavaScript. This grew very quickly into a lot of code to deal with the vast amounts of quirks that exist in the browser DOMs, quirks that are necessary for web compatibility.

The work to make the above change a reality landed on the CVS trunk on 5/8/2001, in what was then known as the XPCDOM landing. It was a massive change, done by several Netscape developers, including John Bandhauer, Peter Van der Beken, Mike Shaver, Mitch Stoltz, myself, and probably more people whose contributions I’ve since forgotten. The change gave us significant code size savings, memory usage savings, and it made the DOM code much easier to hack on. So over all, a good change, one that served us well for many years. It also introduced some problems though, some indirect ones, and other direct problems. One of the problems was that given the fact that we used this generic XPConnect library for reflecting the DOM to JS on the web that meant that some of the guts of XPConnect also became accessible to the web. That means we exposed the notion of calling QueryInterface() on things in the DOM to the web, and it also meant we exposed the global “Components” property on our global objects (i.e. window.Components). Neither of those things belonged on the web. Their use was never pushed, at least not intentionally, but they were there nonetheless, which inevitably meant that some sites started depending on them (fortunately only in small numbers AFAIK, but still). Another problem was that XPConnect depended on a particular prototype setup, in which the XPConnect-wrapped object’s immediate prototype was a flattened view of all interfaces that the object in question implemented. This lead to one problem in particular, which was that people were unable to override existing methods on inherited interfaces in Mozilla’s DOM. To give an example, if a site wanted to override Node.prototype.appendChild, they could do that, but their change would be shadowed by the flattened view of all DOM nodes’ interfaces that XPConnect put on the immediate prototype of every DOM node. With this setup a JS developer could still add to prototypes like Node.prototype, and those additions would be visible on all nodes. But changes didn’t work, and web developers kept stumbling over this problem.

Then over time this overall approach started showing other problems as well, beyond the functional problems I touched on above. The quirkiness of the browser DOM, plus the fact that more and more DOM APIs were added, led to nsDOMClassInfo.cpp growing significant in size, and it also grew to significant complexity, which lead to that file being pretty unwieldy and not very hacker friendly. Performance of the DOM bindings also started to become a problem. At first performance wasn’t a problem, the JS engine (then JS was fully interpreted) wasn’t very fast (at least not by any current standards), and likewise the C++ DOM wasn’t necessarily all that fast, which meant that the overhead of the bindings between the two worlds generally got lost in the noise of JS and the C++ code executing. But as the JS engine grew faster, and the C++ DOM likewise, the overhead of the bindings started standing out more and more.

Now XPConnect wasn’t necessarily slow, but it wasn’t necessarily fast either. It was a generic cross language communication layer, one that was even thread safe, which meant it needed to do a lot of stuff, including locking of various data structures etc. And the generic nature of the library of course meant that there’s few corners that can be cut to speed up cases that really matter for performance. The point is, in roughly the year 2005 or so, it was starting to become more and more of a bottleneck.

At that point, we started looking at optimizing XPConnect, w/o really changing how we used it in fundamental ways. There was some fat that got trimmed, and that helped, but those changes resulted in comparatively small improvements, not the significant gains we’d need long term.

Sometime before this point we had also added the cycle collector, which had the unfortunate side effect of making reference counting more expensive, and XPConnect was pretty reference counting happy. Peter Van der Beken, myself, and others pulled a good bit of heroics to eliminate a lot of the extra reference counting that was done, and that gave some good gains as well.

Then came 2008, with even more optimizations in the JS engine, including a tracing JIT. That made the overhead of the bindings stand out even more, again. Around that point, we had two plans to make significant improvements. The first one was Jason Orendorff’s work on quick stubs, which gave us shortcuts that bypassed a good bit of the slow paths in invoking certain methods/getters/setters on DOM objects. It wasn’t a catch-all approach, but it was one that we could explicitly use for things we believed were performance critical. What quick stubs did for us was that it gave us a code generator that could generate specific code for specific methods (based on a configuration file and the relevant IDL files), and this code could be made very fast. That was a big improvement. But it still left us with some XPConnect overhead in places where we didn’t want it, in particular with DOM object wrapping. Wrapping still went through the fairly heavy weight code that created new DOM object wrappers, or even looking up existing ones for objects that had already been wrapped (i.e. touched by JS before). The second of the two significant optimizations we did in 2008 was Peter Van der Beken’s work on caching the XPConnect wrapper on the DOM objects themselves. This was what became known as nsWrapperCache. That work left us with significant overhead in wrapping new objects, as in, the XPConnect wrapper construction code was still hurting us. But in the case where we were touching a DOM object from JS that had already been touched, we got a lot faster, partially because we were able to look up a wrapper for a DOM object w/o calling into QueryInterface(), which meant we didn’t do any reference counting on that path at all. Plus, we also avoid some thread safe hash table traffic, which helped too.

The next significant optimization after all that was Peter Van der Beken’s work in 2009 on lightweight DOM wrappers (a.k.a. slim wrappers, which is what I’ll call them from here on). These slim wrappers gave us the ability to wrap a DOM object w/o creating a heavy weight XPConnect wrapper (XPCWrappedNative). A slim wrapper is basically just a JSObject that we create and give it enough smarts to make the object look like a real DOM object. And a slim wrapper has built-in smarts that can morph the slim wrapper object into a real XPCWrappedNative object should the need arise, which did if for instance someone explicitly asked for the wrapper from C++, and there were other triggers too which caused a slim wrapper to morph. So with all that, we got to bypass even more of the thread safe hashes etc in XPConnect, which again sped us up. At this stage, the combination of slim wrappers, the wrapper cache, and quickstubs, finally started to give us some serious speed out of our DOM bindings.

Now, around this same time the JS engine team was in full swing making JS faster yet, the tracing JIT was getting even faster, and it was being used more frequently. And there was talk about JaegerMonkey, a full method JIT that would (and did) make JS performance significantly better once again. That again meant that DOM binding performance again got more important. We invested even more work in making our current infrastructure even faster. We started writing specialized hand coded quick stubs which would avoid even more QueryInterface() calls, and could also call straight into non-virtual methods in the C++ DOM. And we pulled all kinds of other tricks to cut out even more overhead, both in the bindings themselves, but also in the C++ DOM code. Lots of this work was done by Peter Van der Beken and Boris Zbarsky. Some of this work led us to some interesting realizations in the DOM code, one of which was that DOM tree traversal performance was heavily dependent on CPU cache utilization rather than actual binding instruction overhead. And this was not for the obvious reasons of us traversing a tree structure with not necessarily good memory locality, but instead it was the vtable reads that the code triggered due to us calling virtual methods during the tree traversal which was the bigger problem. The vtable reads were causing CPU cache misses, and that ended up being a significant performance hit.

At this point we had fairly well hit a performance wall with the current setup. We’d squeezed out pretty much all the performance we could realistically squeeze out of this code. We had created shortcuts around XPConnect, we had done what we could in the scriptable helpers, and we had optimized the C++ DOM implementation fairly heavily as well. Yet we were still behind the competition (i.e. WebKit) in raw DOM access performance.

And then there was type inference support on the horizon, which again made the JS engine faster, and DOM binding performance mattered even more again.

All this led us to start thinking seriously about a different approach to what we had here. And that will be the topic of my next blog post here, which is about our “new” DOM bindings.

Posted in mozilla | Tagged | 8 Comments