CSS Selectors – Speed Myths
On of the things updated in Ext 1.1 RC1 was DomQuery, the CSS selector implementation in Ext. There seems to be a trend sweeping through all of the popular JavaScript libraries of implementing or improving existing DOM querying using CSS selectors. In fact, since my original post with querying benchmarks for the various selector implementations, overall selector processing power has gone from Pinto power to a Mustang GT 500.
Has it really?
While there is no doubt that there have been improvements in every library, the manner in which each has gotten there is different. Some use XPath support in FireFox to gain a slight edge in that browser, but lag way behind in other browsers. I have tried to remain as impartial as possible in my evaluations of each library’s implementation.
Mootools
The latest on the scene with their slickspeed post, has implemented a nice test suite. In fact, it was so easy to set up I decided to use it for the tests included in this post below.
For some reason, although their test suite is titled “speed/validity selectors test for frameworks”, it leaves out many useful selectors such as div.dialog.emphatic (multiple classes) and div:not(.dialog) but includes completely useless selectors such as div div div. I have added those two selectors in my tests below.
The Mootools selector implementation relies heavily on XPath in FireFox. Performance in other browsers drops significantly.
Prototype
Prototype’s selector implementation is based on the initial Ext DomQuery code mixed with XPath support in FireFox. I had high hopes when seeing the first benchmarks for their implementation, but somewhere something must have gone wrong. The XPath picks up a lot of slack in FireFox but similar to Mootools, performance in other browsers drops significantly.
Dojo
Dojo’s selector implementation borrows ideas from various sources. It also uses XPath in FireFox but it’s more selective in the selectors it converts to XPath. Its cross-browser performance is also pretty consistent.
jQuery
jQuery’s recent 1.1.3 release gave it quite a big boost in performance. jQuery, like Ext, does not use XPath in FireFox. Credit goes to John Resig for the most efficient nth-child implementation I have seen to date. In fact, Ext 1.1’s new nth-child implementation is modeled very closely after it.
jQuery was the only library other than Ext that passed all the tests.
Ext
DomQuery’s performance is based on highly optimized code. In FireFox, even without XPath Ext is right there in the mix – I think that says a lot about the efficiency of the code. DomQuery by far has the most consistent performance cross browser. Like jQuery, Ext supports unlimited chaining of filters and selectors and querying XML Documents.
Results
Here are my results. Bold indicates the fastest time.
| Library | IE 7 | FireFox 2 | Safari 2.0.4 | Opera 9.2 | IE 6 | Errors |
|---|---|---|---|---|---|---|
| Ext | 202 ms | 239 ms | 330 ms | 96 ms | 902 ms | 0 errors |
| jQuery | 334 ms | 392 ms | 843 ms | 176 ms | 1948 ms | 0 errors |
| Dojo | 415 ms | 315 ms | 917 ms | 110 ms | 1775 ms | 4 errors |
| Mootools | 684 ms | 136 ms | 1446 ms | 129 ms | 2655 ms | 4 errors |
| Prototype | 868 ms | 172 ms | 2031 ms | 119 ms | 3743 ms | 2 errors (IE only) |
IE7, FF2 and Opera 9 tested on Vista, Core 2 Duo 6600, 4GB RAM
Safari 2 tested on Mac OS X 10.4.9, Core 2 Duo T5600, 2GB RAM
IE6 tested on XP, Athlon x64 3200+, 1.5GB RAM
Run them yourself
http://extjs.com/playpen/slickspeed
Why another test?
These tests were conducted to test the 1.1 RC1 release, and to dispel some myths about the selector speed of each library.
I would prefer it if these tests were restricted to valid CSS selectors too.
Great work again Jack…you never cease to amaze me.
Great post, Jack! I always appreciate reading your take on these issues.
Unfortunately, I think the slickspeed test has a few other problems worth mentioning:
The HTML that the test uses was taken from a slapdash speed test I put together back in 12/2006 to determine the relative speeds of selectors within jQuery. I had no idea it would be used in the battle for selector speed dominance, and I’m a little embarrassed that it’s still out there. Surely someone can come up with something better than Act 1, Scene 3 of Shakespeare’s As You Like It. It’s hardly representative of what the DOM tree will look like on corporate sites or blogs.
Also, the slickspeed test reports the total time it takes to run through all of the queries, but the results are misleading when, for example, one library (incorrectly) finds 0 elements but records 1ms while the libraries that get it right record 5-15ms.
It might be more appropriate to have two rows at the bottom of the slickspeed table. One would report the average time for all of the library’s successful queries, and the other would report the total number of failed queries.
By the way, I realize that you aren’t the one who put the slickspeed test together, and I’m not suggesting that you ought to do anything about it. Just thought I’d add my 2 cents and offer a couple more reasons that we should probably take the results with a grain of salt.
Karl,
I agree 100% that the test DOM is not very realistic. In my personal selector usage and Ext’s internal selector usage, selectors are always scoped to a specific element/container. This makes the number of nodes it is working with much smaller and makes the use of XPath a slow down instead of a speed boost. For the sake of not wanting to spur a new debate in that department, I specifically left it out of my post.
> offer a couple more reasons that we should probably take the results with a grain of salt.
I think it’s worth noting that Ext and jQuery had no errors, so their times are in fact the correct times. I agree there are spots where Mootools picks up time because when it fails, it fails fast.
Some suggestions to dispel more myth. You count the thrown errors but not the results with wrong count (like div[class!=madeup] and div[class|=dialog]) . Imo that should also count as error, since the results are wrong. Ignoring not supported selectors or throwing an error is simply a design decision during framework design.
I agree also that a more realistic HTML (jQuery used a snap from the digg site for an realistic test dom once: http://www.kenzomedia.com/speedtest/template.html, http://www.kenzomedia.com/speedtest/) would bring better real-life results.
Furthermore Safari3 should be also in your overview, since it should have totally different results because of his XPath support.
Karl, exactly what I noticed upon jQuery’s 1.1.3 release -> http://jquery.com/blog/2007/07/01/jquery-113-800-faster-still-20kb/#comment-34150
What is the version of Prototype used ? Isaw the last relased fix lots of performance issue on DOM Query ?
I’m using prototype, is there a solutions to bind Prototype DOM Queries functions (.up(), .down(), …) with other DOM Query libs to test in a real application if there is visible changes ?
Regards
Andrie, ha! That’s great! You know, I just might have read the comment on the jQuery blog and internalized your idea so much that I thought it was my own. Sorry if that was the case. In any case, I hope I added a little more to the discussion in my comment above. Cheers!
Safari 3 beta (Windows) is the fastest on my system and this in every library! Opera and Firefox clearly have some strengths and issues with some libraries. Safari 3 does an overall good and a very performant job in javascript.
Has anyone encountered similar results?
Jack -
I posted about this on my blog a couple of days ago, and I also introduce two variations on the selector test suite – both using a copy of a complex, realistic page from my company’s website, Zillow.com. One suite simply does analogous queries to the original, modified for the new document; the other will scan the document and create 25 arbitrary queries based on actual elements in the DOM.
I came to much the same conclusions you did, and congratulations – DomQuery looks to the the best-of-breed for CSS selector engines.
Jack,
I’ve used a variation on this test for testing a toy selector implementation I’ve been playing with (I’ve got to post it one day) that uses what I think is a very different approach to things (unfortunately its not as speedy as I thought it should be). I’ve noticed something about most selector implementations, one that is hidden by the test scenarios give. Failed selectors (that is those that do not return any nodes) take almost as long as successful selectors of similar length, depending on where the failed node is within the selector. This is actually one of the situations that I’ve optimized for in my toy; I find the speed of failed selectors more useful than successful ones.
Why would that be? I don’t write hand crafted web applications or sites. I belong to a company that builds online banking systems for a large percentage of credit unions in Canada. I have to support a large team of very smart Java Developers, who (generally) don’t know a thing about HTML, JavaScript or CSS. In my world, the ideal solution is to have well thought out mark-up and a single JavaScript Library that attaches all the necessary behaviour to the mark-up it finds. That means that I may have a lot of selectors running that don’t return anything on any given page.
And that means that I want selectors that won’t return any results to run FAST! Ideally, selector speed should be proportional to the number of results it retrieves, not the complexity of its query. Or to be more precise, the more complex (that is the more specific the query) the faster it ought to be. Which suggests that tree traversing methods should be abandoned, eventually.
Perhaps you could add some selectors to your test that are guaranteed to return 0 results?
Just wondering. How come some are packed and some aren’t?
Does that change the results?
It might be neat to have a way to point the speed test at a URL and test. That way each person could see how their site would measure up. I suppose many of the test results would have zero found, but that’s probably even better. That way they only see the speed results from the selectors they would possibly use.
Hope all is well,
Glen Lipka
@Glen
Packing shouldn’t affect results. I do agree that a better markup source and more direct queries should be incorporated into the test. I personally *never* use global queries like the one found in this test as I believe they are a bad idea for performance. Selectors should be scoped so they work with only the nodes they need.
@Adam
It’s difficult to determine that there are 0 results without running the query. A zero result would return a little faster though as it would do significantly less node processing.
@mg
Safari 3 is blazing!
Quick question: What version of Dojo was this?
Thanks for finally touching on the “fad” status of CSS selector speeds these days. I hate that.
@Karl – not at all a problem! It’s good to see that the logics behind my proposal are spread individually! Common sense and good logic don’t have copyright.
@mg – indeed, Safari 3 shows us a new Formula 1!
@Alec – I would assume it’s the latest (Dojo 0.4.3)
I did some test, especially with jQuery on Windows XP & Vista, Mac OS X, Linux etc. The results confirm your testresults: The fastest browser is OPERA!
Congratulations Jack. These are great numbers. I am not even sure if it is possible to query any faster the DOM.
Anyway, speaking of widest browser support (a must IMHO for a library), you may be interested in ext behavior on various linux browsers. Apropo linux – ext, jQuery and dojo are the only libraries that run on Konquerer. You can find these test results along with screenshot over at my site.
Nice listing, thanx for your work.
So, Opera is the fastest and by a decent margin.
Yay for opera \o/
I have to agree with Dean, only use valid selectors in the tests, otherwise the test isn’t a fair one.
Also I ran the tests on all browsers for the Mac and Prototype came out on top every time and found more and only threw exceptions(2) in safari2. How accurate is this test?
Thanks for the effort and time,
Kevin
Nice article/blog Jack
Good read
I got pretty different results with Safari on Win XP and Opera on same machine.
And Safari was not so bad at all as your Mac Safari seems to be.
Safari 3 is NOT that much faster. In fact: safari cheats the speedtests. Read this: http://www.howtocreate.co.uk/safaribenchmarks.html
I’ve got quite different results on 4-core Mac Pro:
Safari: 115 161 118 87 171
Firefox: 146 355 152 256 306
Opera: 137 215 138 138 146
The opera is the fastest and the best
The test seems to be flawed, vastly different results in spite of getting the same result: error.
Also, testing normal pageloading are both Safari 2 and WebKit much faster so there are likely a few bugs in the script.
Mac mini 1.42 GHz G4, 1 GB RAM.
WebKit (Safari 3): 228 435 258 232 445
Safari 2.0.4: 2898 798 1465 389 817
Firefox 2.0.0.5: 458 1023 407 836 771
Opera 9.22: 384 588 358 370 344
@David
Sorry your post got caught by WordPress and I just found it. I agree the markup used in the test is pretty bad and your test looks much more realistic. Thanks for sharing your results!
Cool to see Opera is actually as fast as they claims, alghough milliseconds isn’t a trouble for me during daily browsing
@mg @jack @andrie Safari for windows may not be comparable when doing javascript-based speed-tests: http://www.howtocreate.co.uk/safaribenchmarks.html
Would it be possible that you update your slickspeed to the recently released jQuery 1.1.4, and maybe to Prototype 1.6.0 RC if there aren’t too many bugs/regressions in it?
it is very good,
The opera is the fastest and the best…
Very useful, thanks.
Please, do so clean up to your (trash)blog
Selectors are one of the most important aspects of CSS as they are used to “select” elements on an HTML page so that they can be styled
thanks very much
Interesting – I never thought that there such differences in CSS Speed
What about Ext 2.0?
Hopefully no effect on existing code already in place.