Guns N’ Roses started recording their ‘upcoming’ album Chinese Democracy in 1994. George Broussard started work on Duke Nukem Forever in 1997. Both titles have become standalone jokes in the music and game industries respectively, commonly regarded as having a release date in the vicinity of Armageddon. In some way, Microsoft has been working on WinFS (on and off, and not always under that name or scope. To be honest, I’m probably not being to fair them in this paragraph) since 1990, promised at some point to be released with multiple versions of Windows, and always pulled before release. When it appeared in the Longhorn betas (the OS that would become Vista), it was slow and incomplete, and the eventual news that it would be pulled from Vista too wasn’t terribly shocking. Vista itself was at risk of becoming a perpetual vapourware joke like Duke Nukem Forever, and after five years development MS was very painfully aware that they needed to get something out the door. So to much jeering at having once again over-promised and under-delivered, one of the three pillars of Vista was dropped.
Not that there wasn’t good reason for taking a long time about it. It was actually a really, really tricky problem they were biting off. Or an entire set of problems. WinFS was Microsoft’s latest iteration on their ongoing attempts to unify (or at least bridge) the concepts of a filesystem and a database. It’s the sort of proposal that automatically intrigue computer scientists, (as can be seen in the many other attempts at it). Why the hard separation between data stored as files, and data stored in a database? Surely the two could be unified, and imagine what it would bring! You could store your music and movies in a normal directory structure, and access them through the normal file access means, but browse them by artist, genre, or with queries such as “All movies directed by Steven Spielberg from the 1990s”.
Why it died
WinFS died as a monolithic entity that integrated into Windows for a number of reasons:
The Web – The concept of an Object Filesystem was something MS had been touting since 1990. It also made more sense back then. In 2006, with the web taking off and obviously becoming the new place for data to live, it didn’t so much. Why bother maintaining your contacts as entries in a local database so you could perform interesting queries on them when Facebook et al could do a better job for the most part, in a centralised location? And if this trend of apps moving to the web continues, then WinFS as a client-side application is weakened drastically as your data moves out of its purview.
Desktop search: good enough – The biggest use case scenario for why WinFS would be awesome inevitably worked out as being desktop search. But when Apple introduced Spotlight for OS X, which was just a simple (compared to what WinFS hoped to achieve) file indexing service it made a mockery of the need for such a complex solution to the problem. The release of Google Desktop for Windows put pressure on this wound. Eventually Microsoft released their own desktop search client for XP, an embarrassing and face-saving move given that file-indexing had already existed as a turned-off-by-default option in XP.
Embedded databases: good enough – The other client-side stories for why WinFS would be a good thing often involved media files. Everyone likes movies and music, and they’ve got lots of meta-data associated with them, like genre, year, length, artists, albums, etc. Lots of ways to order and query a collection. The problem for WinFS was it was never clear why this couldn’t be just as easily handled by an application-specific database. Like iTunes. And the tides were shifting on this front: cross-platform development has become important again. And SQLite is a lot more Mac and Linux-friendly than a new Windows-only API would be. It’s also a lot more existent. Developers like that.
But who can really say what “dead” means?
Jon Udell has an interesting interview (for certain definitions of “interesting”) with Quentin Clark, a manager at Microsoft who used to be the project manager on WinFS.
– Microsoft proceeded to start rolling the WinFS development into its backend software efforts ADO.NET and SQL Server. This isn’t news. And it makes sense in its own way. If web-apps are going to be where we store our data in the future, then the databases backing them are going to become our filesystems in a sense. Although if you think about it that way, then we’re a good way already towards the WinFS vision, and WinFS as it was originally envisioned is once again undermined by good-enough approaches.
In the interview Clark talks about several features in SQL Server 2008 that keep alive the WinFS dream:
– They’ve added some filestream column-type, which reference files on the underlying NTFS partition rather than storing the data itself in the database, which makes for better performance for large binaries.
– They’ve added a hierarchical column-type. You know, hierarchical, like a directory structure.
– They’re going to add a Win32 namespacing function, which will expose the database to Windows as another file storage device you can then browse and do all the usual fun stuff. WinFS by complex stealth. There’s more than one project for Linux that does the same thing through FUSE.
So in short, SQL Server 2008 will/is able to store large files just as well as NTFS is able to. It will be able to describe hierarchial data structures. It will be accessible from the Win32 file-system APIs. It’s pretty much offering all WinFS did, except for the client-side specific schemas (such as contacts, music, etc).
It’s also still as useless for most users.
I think the interesting part about all this (certainly SQL databases are a subject I struggle to get excited about) is that once you examine the WinFS idea from the server end of things, stripped of the client-side schemas and vapourous UIs and dubious use-cases, it’s pretty mundane. That is, it’s everyday stuff. The critical step’s already tak[en/ing] place: we’re moving our own data off the filesystems and into the cloud, where they’re shaped behind the scenes into whatever schema is best for the application, with an interface on the front-end designed to fit that structure. Files can exist in multiple folders in Google Docs. Social networking sites deliver on much and more of the contacts functionality originally promised by WinFS. iTunes structures your music collection as a simple database and collection of playlists built from queries into it (dynamically if you wish). The battle WinFS was going to fight has already been won. The next one is one Microsoft was never going to fight anyway, one for the structure and open exchange of this data.
 Via OS News