2006/06/12
Microsoft Academic Search: Promising
It's become something of an article of faith among the technically savvy and open-source-minded: Google good, Microsoft evil. Google cool, Microsoft kludgy. And certainly, anyone who has nearly put their fist through their monitor after the 1,134th appearance of Clippy, who has gaped in bafflement at Microsoft Bob, or who has spent an hour trying to disable the "improvements" such as real-time "grammar checking," in a given new version of Word, isn't likely to have warm feelings for the Orcs of Redmond. So a comparison between Microsoft's new Academic Search beta and Google Scholar (also officially in "beta," though it's been on the scene, I believe, for nearly two years now) should be a layup for Google, right?
After playing with the MS application (spurred by a demo at a recent trade show), I'm not so sure.
Right now, the MS Academic Search is limited to references in computer science, electrical engineering, and physics, so it's a bit hard to draw direct comparisons of the reliability and comprehensiveness of the results. But a few things stand out with respect to the UIs. The Academic Search UI is very different from Google Scholar's in the search result; the MS application gives you a two-paned view with lots of AJAX skullduggery in the background, so that when you put your mouse over a search result in the left-hand pane, you get the article's complete metadata (including abstract, where available) in the right-hand pane. In that same pane you can view BibTeX and EndNote versions of the citation information. (Though initially I had thought this was rather cluttered, it's all very handy after you've played with it a while.) The feature I'd focus on, though, is the pull-down box that lets you re-sort results by author, date, journal, or conference -- particularly the reverse-chron date sort.
I tried the search string "XML bioinformatics" on both Live Academic Search and Google Scholar. For the relevance-ranked search, both engines returned the same top result -- a review article by Achard et al. from 2001 -- but after that the results diverged sharply. And both, strangely enough, looked like they could be useful, even though I'm not sure that the two relevance-ranked results had any of the top 10 results in common save the first. (It is indeed a big world out there.) Selecting the reverse-chron sort on Academic Search quickly re-sorted the results into a nice list beginning with a 2006 article from Bioessays, and proceeding in well-behaved reverse-chron from there.
The Google Scholar results page, meanwhile, has a toggle in the upper right-hand corner that lets to switch between "All Articles" (the default view) and "Recent Articles." Where would you expect the "Recent Articles" link to take you? A reverse-chron list of the most recent publications relevant to the search string, perhaps?
Me too. But on Google Scholar, the first article you get by selecting the "Recent Articles" toggle is . . . the same 2001 article that was at the top of the "All Articles" list. And you'll find only a single 2006 article on the first page of results (number nine on the page). This is nonsense, of course, but it isn't limited to this particular search string; I've had similar results on Google Scholar with hot-button topics like avian flu, for which it's reasonable to assume that something worthwhile has been published in the current year. And Google's Advanced Search page on Scholar doesn't give you the option of a reverse-chron listing.
So on at least one use case that would seem to be important to research -- "Show me what's recently been published on [insert your topic here]" -- the MS app would seem to beat Google. At least for now.
The MS tool has some clear disadvantages, too: Some of the AJAX witchery leads to unpredictable behavior in the search-results pane, for example, and the design choices in the results don't always seem to have been made with usability as the top priority. Some of the other displays, such as sorting by journal or by author, are awkward to navigate, and the rules underlying the sorts are baffling. And right now the Microsoft engine doesn't do anything with respect to citations, which is of course a core strength of Google Scholar -- though the pleasant chap from Microsoft at the booth told me that's on their things-to-do list (as, of course, is broadening the search universe beyond just EE, physics, and comp sci).
And, of course, there's speed. At the same trade show at which I saw this demo, one of the speakers made the interesting point that Google's big proprietary strength in the marketplace isn't so much search per se, as its ability to leverage and manage a distributed system of servers for incredibly fast results (the "13,900,000 results on Anna Nicole Smith in 0.11 seconds" syndrome). That's going to be hard for anyone to beat.
