Paragraft

At the moment my master’s work has me in the middle of what seems like an never-ending sea of runtime errors. It’s the largest Python project I’ve ever worked on, and I’ve been forced to re-learn a lot of practices that I really should have internalized a long time ago but never really did. I’ve also been able to play around with interesting meta-programming techniques (ie, modifying the language on the fly, so to speak, though not as much as a genuine macro system would allow) to stop certain types of bugs reoccurring.

As things progress, the new code settles down into something that actually runs more often than not, and becomes predictable. One thing I’ve relearned the value of has been the sanity check: throwing assert statements around like it’s going out of style, and I’ve found myself almost wishing Python had Eiffel’s native contracts[1]. Almost. I’m willing to pay the development time/code stability trade-off of this approach.

But it’s a whole other matter when you consider articles like this one from the Baltimore Sun (June 30). It’s about a forensic bug-hunter team at the FDA set up to address the growing dangers as software runs more and more medical systems. It talks up static code analysis as a possible solution, though after passing through the mainstream journalism medium it probably comes across as much more of a panacea than it really is. After all, there is no single magic pill for complex software like this.

One passage stood out though.

FDA officials declined to name the maker of the infusion pumps. (In 2006, Cardinal Health, of Dublin, Ohio, stopped production of its Alaris SE pumps because of a key-bounce error that reportedly killed two patients, including a 16-day-old baby that got 44.8 milliliters of intravenous nutrition, rather than 4.8 milliliters.)

During the investigation into the malfunctioning pumps, nurses complained about frequent keyboard errors, while the manufacturer blamed nurses for entering the wrong drug information and then failing to double-check, said Brian Fitzgerald, who heads the FDA’s software specialists.

A shade over 23 years ago an equally tragic disregard for a reported malfunction that appeared to have no effect gave radiation overdoses to 6 people. The Therac 25, a radiation therapy machine that had a subtle bug that led to massive overdoses, went down in software engineering history as being one of the most infamous cases of a race condition bug. There was a comprehensive (and quite readable) report that’s very much worth reading for all developers working on non-trivial software or electronics. And it makes for spine-chilling reading.

From the Boston Globe, June 20, 1986 (source):

MAN KILLED BY ACCIDENT WITH MEDICAL RADIATION

by Richard Saltos, Globe Staff

A series of accidental radiation overdoses from identical cancer therapy machines in Texas and Georgia has left one person dead and two others with deep burns and partial paralysis, according to federal investigators.

Evidently caused by a flaw in the computer program controlling the highly automated devices, the overdoses - unreported until now - are believed to be the worst medical radiation accidents to date.

The malfunctions occurred once last year and twice in March and April of this year in two of the Canadian-built linear accelerators, sold under the name Therac 25.

Two patients were injured, one who died three weeks later, at the East Texas Cancer Center in Tyler, Texas, and another at the Kennestone Regional Oncology Center in Marietta, Ga.

The defect in the machines was a “bug” so subtle, say those familiar with the cases, that although the accident occurred in June 1985, the problem remained a mystery until the third, most serious accident occurred on April 11 of this year.

Late that night, technicians at the Tyler facility discovered the cause of that accident and notified users of the device in other cities.

The US Food and Drug Administration, which regulates medical devices, has not yet completed its investigation. However, sources say that discipline or penalty for the manufacturer is unlikely.

Modern cancer radiation treatment is extremely safe, say cancer specialists. “This is the first time I’ve ever heard of a death” from a therapeutic rediation accident, said FDA official Edwin Miller. “There have been overtreatments to various degrees, but nothing quite as serious as this that I’m aware of.”

Physicians did not at first suspect a rediation overdose because the injuries appeared so soon after treatment and were far more serious than an overexposure would ordinarily have produced.

“It was certainly not like anything any of us have ever seen,” said Dr. Kenneth Haile, director of radiation oncology of the Kennestone radiation facility. “We had never seen an overtreatment of that magnitude.”

Estimates are that the patients received 17,000 to 25,000 rads to very small body areas. Doses of 1,000 rads can be fatal if delivered to the whole body.

The software fault has since been corrected by the manufacturer, according to FDA and Texas officials, and some of the machines have been retured to service.

… (description of the accidents)

The Therac 25 is designed so that the operator selects either X-ray or electron-beam treatment, as well as a series of other items, by typing on a keyboard and watching a video display screen for verification of the orders.

It was revealed that if an extremely fast-typing operater inadvertently selected the X-ray mode, then used an editing key to correct the command and select the electron mode instead, it was possible for the computer to lag behind the orders. The result was that the device appeared to have made the correct adjustment but in fact had an improper setting so it focussed electrons at full power to a tiny spot on the body.

David Parnas, a programming specialist at Queens University in Kingston, Ontario, said that from a description of the problem, it appeared there were two types of programming errors.

First, he said, the machine should have been programmed to discard “unreasonable” readings - as the injurious setting presumably would have been. Second, said Parnas, there should have been no way for the computer’s verifications on the video screen to become unsynchronized from the keyboard commands.

As the report makes devastatingly clear, there was far more wrong with the system than just those two issues in the last paragraph. The code-base as a whole was shoddy, there weren’t hardware failsafes to stop unreasonable behaviour, and there was insufficient or non-existent sanity-checking (i.e, checking an internal state of the program is sensible). From the report:

The operator can later edit the mode and energy separately. If the keyboard handler sets the Data Entry Complete flag before the operator changes the data in MEOS, Datent will not detect the changes because it has already exited and will not be reentered again. The upper collimator (turntable), on the other hand, is set to the position dictated by the low-order byte of MEOS by another concurrently running task (Hand) and can therefore be inconsistent with the parameters set in accordance with the information in the high-order byte. The software appears to contain no checks to detect such an incompatibility.

Take away point (and again, reading the report will give you several): Sanity checks are good. Make use of assertions. Code defensively.

But most staggering was the way the manufacturer AECL reacted to the news. The blame for why it took two years for the bug to be recognised can be laid at their feet for not responding adequately to reports of malfunctions. Their attitude can be summed up in this paragraph from the letter they sent to hospitals after the bug was found, describing how to keep the machine safely operational.

Effective immediately, and until further notice, the key used for moving the cursor back through the prescription sequence (i.e, cursor “UP” inscribed with an upward pointing arrow) must not be used for editing or any other purpose.
To avoid accidental use of this key, the key cap must be removed and the switch contacts fixed in the open position with electrical tape or other insulating material. For assistance with the latter you should contact your local AECL service representative.

Take away point: If problems occur because your software’s allowed to drift into a nonsensical state when the users use the keyboard too fast, it’s actually not really a fix to tell them to remove buttons so they can’t use the keyboard too fast. It seems comical in hindsight. But in 2006, the FDA noted in a press release how Alaris responded to their faulty infusion pumps:

In an August 15 recall letter, Alaris informed customers that it will provide a warning label for the pumps and a permanent correction for the key bounce problem once it is available.  In the letter, Alaris also provided recommendations to pump users on steps they can take to minimize key entry errors until the problem can be corrected. The steps are as follows:

Proper Stance
When programming pumps, stand squarely in front of the keypad (ideally with the pump at eye level for best visibility) to facilitate proper depth of depressing each key.

Listen
Focus on listening to the number of beeps while programming IV pumps; each beep will correspond to a single digit entry.  Unexpected double tone could indicate an unintended entry.

Verify Screen Display
When programming the pump or changing settings, always compare the patient’s prescribed therapy or the medication administration record, original order, or bar code device to the displayed pump settings for verification before starting or re-starting the infusion.

Independent Double Check
Request an independent double check of pump settings by another practitioner before starting or changing infusions with hospital-selected high alert drugs.

Look
Before leaving the patient’s room, observe the IV tubing drip chamber to see if the observed rate of infusion looks faster or slower than expected.  Adjust accordingly.

In a way, it’s worse than “take the key off the keyboard and tape it over”. At least that stopped the error as known from happening. So here’s a final take away point: Telling your users to be constantly checking the system hasn’t malfunctioned, saying you’ll send out a warning sticker and eventually a real fix, doesn’t really count as a fix. Relying on human vigilance isn’t a solution. Not when bugs really do matter.

[1] PEP316 actually proposed that contracts be added to Python. Current status: Deferred.

Here’s a case of state-sponsored photo manipulation that matches the boldness of Stalin’s celluloid liberties (like the disappearing commissar above).

Spot anything odd in this photo? It was attached to this news story.

Specifically that part. The part with the three layers of identical looking corn flakes.

The cereal’s been clone-stamped to make it look like a veritable cornucopia (Cornucopia. Heh). What’s the reality here? Is the media covering up a foodstuff shortage in NZ? Can we know that Hubbard is really smiling in the original? Maybe he’s casting his hands out in despair over the last cup of flakes left in the warehouse. Should I be hoarding Weet-bix and rolled oats?

Seriously, news photographers, clone-stamping does not improve the composition of your photos. It’s just ugly. It’s also like chartjunk in that it’s unnecessary, and though a very light shade of gray on the lie-spectrum, does say something about how you regard your audience. You’d think the bevy of stories about photographers being fired for taking similar compositional liberties would scare anyone in a newsroom a mile away from doing it.

On June 9th Apple CEO Steve Jobs will take to the stage in San Francisco to give the keynote address at his company’s 2008 WWDC. Rumours are strong that he’ll be unveiling the second generation (or at least 3G capable) iPhone. He usually does about two big keynotes a year, and they’re almost always worth watching. Undeniably the best showman in the technology industry, he packs his own reality distortion field, a charismatic glow that can convince you of the urgent need to buy Apple’s latest product if you are to live a fulfilled iLife™. The older keynotes for significant product launches like the Macintosh launch, the unveiling of OS X, and the iPhone launch are particularly good examples[1]  worth watching to see how he brings the audience into the palm of his hand. He brings them up to speed with what the company’s been doing. He then starts naturally turning the topic into a story of a missing product in the market. He’ll talk about competitors’ products (if they have any) in that space, and talk about their short-comings, shaping the descriptions so that it becomes increasingly obvious what the perfect example of that product should look like. And then he unveils it, pulling it out of his pocket, pulling a sheet off it, or pulling it out of a manilla envelope. He’s spell-binding, and it’s only obvious what a talent he has when you see other CEOs try to emulate it and fail decidedly (warning: video autoplays).

One of the props he often relies on are graphical charts to show things like how marketshare of an product has been doing, or how a new one compares with competition for performance. Data presentation guru Edward Tufte has an expression for Job’s style: chart-junk. One of the themes Tufte keeps coming back to in his book Visual Explanations (which I’m reading and really quite enjoying at the moment) is that the magician is worth studying, as an example of a performer who acts out a lie. Learn from what he does, says Tufte, and then seek to do the opposite when presenting information. So it’s appropriate when you consider Job’s style how often he’s described as a magician. And the elements of a Stevenote are in some ways closer to a magic show than they are an honest data presentation. It’s sadly to be expected from what is in essence a marketing pitch. I’m always slightly annoyed when good marketing crosses the line into subtle dishonesty, and Jobs really shouldn’t have to engage in it. His products usually advertise on their own merits well enough.

On Tufte’s forum I saw a comment in a discussion on the sins of pie graphs pointing out this photo.

It’s actually horribly blatant. The Apple segment gets the closer perspective, and somehow 19.5% ends up with a bigger surface area than 21.2%.

Wondering if he did this often, I delved off into the Engadget archives of their coverage of other Stevenotes. And found plenty of chart-junk. For each of these, spot the bias introduced by the 3D perspective, and how it always seems to fall in Apple’s favour when there’s a direct comparison to be made.

Above: It’s a performance comparison, so smaller numbers mean the faster software. Safari is Apple’s browser.

These two above aren’t so bad. But the parallax bias is there, and it is in Apple’s favour. Apple’s numbers fall the left of the vertical vanishing line, so get foreshortened.

He’s normally so good about have some numbers on a chart. In their absence, and the presence of a 3D perspective, it’s impossible to regard this chart as anything useful.

Tiger is the latest version of OS X in this case. Panther’s the previous one, and older is everything before that.

In a 2D graph you’ve got two opportunities to skew the data. Growth is good, so if you want to make the numbers at the right hand side of the graph look better, bring that end closer. But additionally in the vertical axis, we see the same parallax bias as the web browser performance comparison charts from above, so that the “Industry” figures get skewed to look even lower than the Mac’s.

This one’s angled and segmented slightly so that the Leopard (the latest version of OS X) area gets magnified, but there’s not much you can do about numbers that different.

This is the blurry successor to the first one. Apple’s the green segment. Again the upward perspective that the Apple segment benefits the most from.

Again, two perspective skews here for bias, and a style bias. The first is the obvious make-the-right-hand-side-closer angle to magnify the latest figures. Which is interesting, since the vertical bias (the vanishing point is in line with the top of the graph) is shrinking those numbers compared to the older ones. Which would appear to be counter-productive, until you read the title and realise that this is a cumulative sales graph. Which is an interesting choice in itself, since cumulative graphs are often hard to get meaningful data out of. They don’t go down after all. Which is why they chose this form for this slide. Growth is slowing. It’s inevitable with a product like the iPod, but there’s ways to soften the news. Combined with the choice of a cumulative graph, the vertical perspective now makes sense. The vertical perspective has the effect of playing down the early growth and maximising the latest, bringing the graph slightly closer to looking linear, than an S-shaped graph that’s beginning to hit the final plateau.

So come June 9 we’ll once again be invited to be spell-bound by Cupertino’s wizard, and expect more chart-junk. It’s fun to be sold a story like this, provided you’re aware that you are being sold one. It’s not a scientific presentation. Jobs is known for prizing aesthetic form, and in a way 3D charts look nicer than 2D. But I would question whether it’s honest to take this consistently a biased approach in presenting your data. And as Tufte asks in Visual Explanations, “Why lie?”

[1] Worth watching for the contrast to these celebrations of Appledom is the infamous Macworld 1997 keynote where he announced the company’s partnership with Microsoft to a congregration (any religious implications from that word are fully intended) that doesn’t buy into his forced enthusiasm. It’s rare to hear actual booing during a “Steve-note”, and the scene was recreated in the 1999 TV movie Pirates of Silicon Valley.

For both yours and my ever decreasing attention spans, in the race to distinguish and spice up the daily news product, here’s more news, shallower, and faster.

Spectra from MSNBC is a pretty terrible romp into 3D. Pretty, but completely unusable and just rather useless. You select what channels of news you want, and as you do a selection of stories from each channel floats into the display as a rotating ring. It wouldn’t be so bad if you could actually click on the floating news items. But no, that does something completely unexpected, it ejects that entire ring of stores. To get to a story you want, you have to navigate via a ridiculous horizontal scrollbar. I thought we had learnt in the 90s that 3D interfaces like this just don’t work. From Information Aesthetics via Data Mining.

Moving from the realms of insanity to just the slightly overwhelming comes Newsmap, based off Google News.

From the very epitome of fickle and populist news rivers comes a selection of cool-looking, fast moving and not really that value-additive visualizations at their Labs section.

Finally comes a low-key (and the most embeddable of the lot) Flash widget that just rotates geo-coded stories on a world map.

Guns N’ Roses started recording their ‘upcoming’ album Chinese Democracy in 1994. George Broussard started work on Duke Nukem Forever in 1997. Both titles have become standalone jokes in the music and game industries respectively, commonly regarded as having a release date in the vicinity of Armageddon. In some way, Microsoft has been working on WinFS (on and off, and not always under that name or scope. To be honest, I’m probably not being to fair them in this paragraph) since 1990, promised at some point to be released with multiple versions of Windows, and always pulled before release. When it appeared in the Longhorn betas (the OS that would become Vista), it was slow and incomplete, and the eventual news that it would be pulled from Vista too wasn’t terribly shocking. Vista itself was at risk of becoming a perpetual vapourware joke like Duke Nukem Forever, and after five years development MS was very painfully aware that they needed to get something out the door. So to much jeering at having once again over-promised and under-delivered, one of the three pillars of Vista was dropped.

Not that there wasn’t good reason for taking a long time about it. It was actually a really, really tricky problem they were biting off. Or an entire set of problems. WinFS was Microsoft’s latest iteration on their ongoing attempts to unify (or at least bridge) the concepts of a filesystem and a database. It’s the sort of proposal that automatically intrigue computer scientists, (as can be seen in the many other attempts at it). Why the hard separation between data stored as files, and data stored in a database? Surely the two could be unified, and imagine what it would bring! You could store your music and movies in a normal directory structure, and access them through the normal file access means, but browse them by artist, genre, or with queries such as “All movies directed by Steven Spielberg from the 1990s”.

Why it died

WinFS died as a monolithic entity that integrated into Windows for a number of reasons:

The Web - The concept of an Object Filesystem was something MS had been touting since 1990. It also made more sense back then. In 2006, with the web taking off and obviously becoming the new place for data to live, it didn’t so much. Why bother maintaining your contacts as entries in a local database so you could perform interesting queries on them when Facebook et al could do a better job for the most part, in a centralised location? And if this trend of apps moving to the web continues, then WinFS as a client-side application is weakened drastically as your data moves out of its purview.

Desktop search: good enough - The biggest use case scenario for why WinFS would be awesome inevitably worked out as being desktop search. But when Apple introduced Spotlight for OS X, which was just a simple (compared to what WinFS hoped to achieve) file indexing service it made a mockery of the need for such a complex solution to the problem. The release of Google Desktop for Windows put pressure on this wound. Eventually Microsoft released their own desktop search client for XP, an embarrassing and face-saving move given that file-indexing had already existed as a turned-off-by-default option in XP.

Embedded databases: good enough - The other client-side stories for why WinFS would be a good thing often involved media files. Everyone likes movies and music, and they’ve got lots of meta-data associated with them, like genre, year, length, artists, albums, etc. Lots of ways to order and query a collection. The problem for WinFS was it was never clear why this couldn’t be just as easily handled by an application-specific database. Like iTunes. And the tides were shifting on this front: cross-platform development has become important again. And SQLite is a lot more Mac and Linux-friendly than a new Windows-only API would be. It’s also a lot more existent. Developers like that.

But who can really say what “dead” means?

Jon Udell has an interesting interview [1](for certain definitions of “interesting”) with Quentin Clark, a manager at Microsoft who used to be the project manager on WinFS.

Take-away points:

- Microsoft proceeded to start rolling the WinFS development into its backend software efforts ADO.NET and SQL Server. This isn’t news. And it makes sense in its own way. If web-apps are going to be where we store our data in the future, then the databases backing them are going to become our filesystems in a sense. Although if you think about it that way, then we’re a good way already towards the WinFS vision, and WinFS as it was originally envisioned is once again undermined by good-enough approaches.

In the interview Clark talks about several features in SQL Server 2008 that keep alive the WinFS dream:

- They’ve added some filestream column-type, which reference files on the underlying NTFS partition rather than storing the data itself in the database, which makes for better performance for large binaries.

- They’ve added a hierarchical column-type. You know, hierarchical, like a directory structure.

- They’re going to add a Win32 namespacing function, which will expose the database to Windows as another file storage device you can then browse and do all the usual fun stuff. WinFS by complex stealth. There’s more than one project for Linux that does the same thing through FUSE.

So in short, SQL Server 2008 will/is able to store large files just as well as NTFS is able to. It will be able to describe hierarchial data structures. It will be accessible from the Win32 file-system APIs. It’s pretty much offering all WinFS did, except for the client-side specific schemas (such as contacts, music, etc).

It’s also still as useless for most users.

I think the interesting part about all this (certainly SQL databases are a subject I struggle to get excited about) is that once you examine the WinFS idea from the server end of things, stripped of the client-side schemas and vapourous UIs and dubious use-cases, it’s pretty mundane. That is, it’s everyday stuff. The critical step’s already tak[en/ing] place: we’re moving our own data off the filesystems and into the cloud, where they’re shaped behind the scenes into whatever schema is best for the application, with an interface on the front-end designed to fit that structure. Files can exist in multiple folders in Google Docs. Social networking sites deliver on much and more of the contacts functionality originally promised by WinFS. iTunes structures your music collection as a simple database and collection of playlists built from queries into it (dynamically if you wish). The battle WinFS was going to fight has already been won. The next one is one Microsoft was never going to fight anyway, one for the structure and open exchange of this data.

[1] Via OS News

In the world-popular board game Settlers of Catan, there’s six types of land tiles that you can occupy in order to obtain resources to fuel your growing empire. Quarries for brick, mountains for ore, forests for wood, plains for wheat, and grasslands for sheep. In your bog standard Settlers game, the most important two to secure are invariably brick and wood: you need them to build settlements and roads, and the game will often go to who can expand the fastest. But in the expansion set Knights and Cities, the games take longer, and the dynamic shifts. Invariably I find the most important land tile to secure in the long-distance run are plains. In the longer game you run up against limits of land. You reach the point where you can’t gain points from expansion anymore, and so you must develop what you have. Your settlements have to turn to cities, and that takes wheat (and ore). Then you need to raise and maintain knights to defend them, and that takes wheat too. The thing about the game is that if you only realise this need once you’re starting to turn your settlements into cities, you’re going to be starved by the players who occupied all the prime spots from the start. The game’s often made or lost by what tiles you initially claim.

Food supplies are one of those things you have to think long term about if you want to win. And two of the latest Stratfor podcasts[1] give an insight into how one of the governments more known for its long term thinking is viewing this particular phase of play, given the current food and fuel crisis.

The first discusses a proposal by China’s agricultural ministry to provide incentives (think tax breaks, subsidies for farm buildings and projects, etc) for companies to purchase or lease farmable land in foreign countries, and particularly Australia. This indicates the long term aspect of their planning: currently four out of ten Chinese are farmers (albeit in a peasant economy), and it’s still a net food exporter. But the podcast cites a McKinsey report saying that China’s urban population will reach 1 billion by 2030. By 2025 there’ll be 219 cities in the country with individual populations >1M.  And for a country with currently 19.6% of the world’s population, it’s only got 9% of the world’s arable land. Additionally, the east asia analyst that starts talking partway through points out that food imports are historically important to China as a result of their Great Famine (although looking back over the list, they’ve had a few).

So they’ve been acquiring foreign land for this purpose since the mid 90s in places like Cuba and the Philippines. This is not new. That they will be looking to Australia is. To continue the Settlers allusion, Australia can be thought of as two of the world’s largest quarry and plains tiles.

As the second podcast goes into detail to explain, although Australia is commonly regarded as drought stricken (which it still is in the wheat and rice-producing parts, but it’s hopefully emerging from this), its northern states particularly have great potential for agriculture (ie, it rains there in tropical amounts). And it’s very underdeveloped in this aspect. It’s more that the rice and wheat growers are partially where they are (ie, under drought) for historical reasons as much as anything else, and prefer the southern climate. Australia stands to gain from it in a small way (93% of all foreign owned land in Australia is Crown lease-hold, rather than freehold, and at least Victoria imposes a 20% additional land tax on foreign owners [2]), but it’s probably not how the government would like to see things going down. It’s far better for them to actually export it to China, and make use of the Free Trade Agreement they’re currently negotiating. I wouldn’t be surprised if the Australians do pass some more protectionist laws in this regard. Its use-of-local-labour and property ownership laws and are strong when it comes to foreign entities. Additionally in the north most land isn’t freehold but leasehold [3]. That makes things easier for the government, but they are still very much constrained by the free market. A lot of farmers there are hurting from the drought, and China’s willing to spend large on long term assets.

Other countries China’s interested in (and there are a few, in the interests of risk management by diversification) include that other big grain bowl of the world, Russia. But it’s one thing for them to talk about buying land in Australia, and another in their large, border-sharing neighbour. The Russians are even more sensitive about foreign ownership of land, and if the new president Dmitry Medvedev is anything like his patron Putin (who was rather strident in nationalizing gas and oil supplies), then he probably isn’t going to be a strong advocate for free market liberalization and foreign investment. The Stratfor podcast concludes that if similar farm purchases happen in Russia, we won’t be hearing much about them due to this sensitivity. I expect that if it does start happening in Australia, particularly near an election cycle, we’ll be hearing plenty about it. The Australian reaction will be worth watching.

This can be regarded as an over-cautious strategy in some ways. China’s economy will soon be the world’s largest again. It’ll have money to buy its food for a long time. So this is interesting (even if it doesn’t happen) in how it shows how sensitive the Chinese government is to food pricing. To keep a population that large happy, you need a lot of bread, and a lot of ciruses.

[1] If anyone’s feeling particularly generous and for some reason wants to spend several hundred dollars on a birthday present for me next month, I’d appreciate a StratFor subscription. Failing that, I’ll just keep on going with their free content features. Their daily podcasts I can particularly recommend as being a neat way to go beyond the typical media depths (shallows) of analysis.

[2]  Doug Cocks - Use with Care: Managing Australia’s Natural Resources in the Twenty-First

[3]  http://www.bmr.gov.au/education/facts/tenure.htm

John Resig just released a rather awesome Javascript library that implements the Processing language.

Jaw status: dropped. My empathy for the Reddit commenter who had nothing more to say on this release than “I give up on programming now.”

Make sure you check out the extensive list of demos. The long predicted competitor to Flash that Javascript + <canvas> could be may be soon upon us. Good. Or at least, some neat games should come out of it.

… is a social construct. Or so it would seem from the story of the lecturer at Dartmouth college who decided to sue her students for harassment when French Literary Theory and how it applies to science didn’t go down too well with them.

From the Wall Street Journal:

Priya Venkatesan taught English at Dartmouth College. She maintains that some of her students were so unreceptive of “French narrative theory” that it amounted to a hostile working environment. She is also readying lawsuits against her superiors, who she says papered over the harassment, as well as a confessional exposé, which she promises will “name names.”

…

Ms. Venkatesan’s scholarly specialty is “science studies,” which, as she wrote in a journal article last year, “teaches that scientific knowledge has suspect access to truth.” She continues: “Scientific facts do not correspond to a natural reality but conform to a social construct.”

The journal article in question. I can’t imagine why her science students objected to any of these arguments.

[COI declaration: I tried to read Foucault a few weeks ago. It was a 10 page essay. I got 3 pages in before I gave up.]

Via Gawker and Julian Sanchez.

There’s a question of design philosophies in software that describe two diametrically opposite ways of theoretically getting the same results: Top-down or bottom-up? Traditionally we’re supposed to do the former, designing the big picture first and then filling in the details until we’ve built all the way down from abstracted design to concrete reality. We usually do the latter, building little lego bits and then trying to connect them into a structure approximating the original design.

But in a sense in the world of web application design, where “best practice” isn’t just a moving target but one moving in quite different directions, the opposite is in effect. We’re doing top-down experience design, when we should really be doing bottom up. The distinguishing issue is that on the web, we’re not just creating one design, we’re creating a suggested design that will then be rendered in a whole multitude of ways.

Normal practice in web design/development is to work out what you want to functionally do, then make the call on what technology (Flash, Shockwave (remember that?), Java, AJAX, ActiveX, PDF, or even Silverlight) would be best for making that happen, evaluating the “best” as a measure of time, expense, longevity, security, and market support. And then if time allowed, you started designing fallbacks for clients without those technologies.

Chris Heilmann has done a good job advocating the opposite philosophy of progressive enhancement. This is the philosophy that involves you starting your site/web-app design with the lowest common denominator, and you produce a functional product at that tech level. If it can’t be done, you need a good reason for it to be so. Then you progressively layer on “richer” technology. It’s the humble and unassuming philosophy: you don’t presume more than you must about your user and their circumstances.

They’re two opposing philosophies that theoretically should give the same results. You start high-tech and work backwards, or you start low-tech and move forwards.

The problem that works against this is Hofstadter’s law: Work has a knack of taking longer than you expect. Unexpected new things to work on arise, and then you start budgeting your time and triaging things you shouldn’t. In the first design model, you would design low-bandwidth HTML versions of your all-Flash site. Unless a new feature request came in and you had to implement that first in the Flash. Eventually you just give up and require that your clients all use Flash. Then you wonder why Google isn’t doing such a hot job of indexing your site anymore. Or you bite the bullet and spend a lot of time doing things properly. As soon as you start prioritizing the high-tech experience as the primary and complete version, you’re just constraining yourself against future flexibility. And then you sometimes end up irrationally justify that primary experience in places that shouldn’t really exist.

The positive reasons for progressive enhancement then start flowing out of varied examples. There’s increasing numbers of users who use something like the Flashblock extension (because I’m sick of Flash-based ads, especially the ones that start streaming video, sucking bandwidth without your permission). Similarly, people have taken to using NoScript, an extension that imposes a white-list on allow Javascript. And don’t forget the disabled. Screen readers for the visually-impaired do a really bad job of handling Javascript. So does the Google web spider, for that matter. Or take the iPhone, a suddenly popular new platform that completely eschewed Flash. If you had invested into a site that required Flash, you were inaccessible. If you had built a site around progressive enhancement, you were much more well equipped to support mobile Safari. So adopting a philosophy of progressive enhancement in these cases improves support for niche users, accessibility, search engine coverage, and an unforeseen new platform.

This means things like coding HTML that’s completely functional without Javascript, or Flash. They’re technology it’s often reasonable to assume the average client will have. But unless you can really justify it, you shouldn’t.

It involves things like not using links with href="javascript:doThis()" or onClick event handlers hard coded into their HTML. Instead just give the links decent ids and then add the event handlers dynamically from Javascript. It’s not hard to do, if you do it right the first time.

There are some surprising offenders in this class. Try adding accepting a friend request on Facebook with Javascript turned off. You can’t actually click the button, and there’s no reason that should be so. Why did I run into that?[1] Well, if you’re the site owner, does it matter?

I had a Dynalink switch with firmware that broke the rule too. It used Javascript-powered links for its navigation, instead of plain HTML. I wouldn’t have noticed, if it weren’t for the Javascript not actually working on browsers that weren’t Internet Explorer. There was no earthly reason for those links to use Javascript, and every time I had to load up IE (particularly if it involved a reboot to Windows to do so) just to configure my switch, it didn’t do much for my opinion of Dynalink.

If you’re a web developer and you’re not already doing this or haven’t heard of the idea before, I strongly encourage you to read Chris’ full article on progressive enhancement. If you haven’t, but you’re exercising sound development principles (separation of code and content, observing standards, using semantically sensible markup, designing with accessibility in mind etc) you’re probably already most of the way there. But do skim over it all the same. It’s a descriptive philosophy that successfully captures much of what we should already be doing, but for reasons that fallen under different hats previous.

It was only a few days that I described how I bought NIN’s latest album Ghosts for US$5 on the strength of Trent Reznor putting the first quarter up for free, and selling it in a high quality DRM-free format. Then he released a single, Discipline, for free. Discipline was alright, but I’ve been loving the instrumental Ghosts. Makes for great programming or reading music (ambient yet interesting, without vocals or annoying bits), and it’s long enough for it to suck you in with its 110 minute length.

I can’t call Ghosts his latest album anymore. The ID3 tag on the Discipline file said to visit NIN.com on May 5. Reznor seems to have been bottling up the creative urges over the years, waiting until he could be released of the shackles of his contracts, because he’s somehow just released another album. Ghosts was only released on March 2. It’s called The Slip, and includes Discipline. And like the single, the album’s free to download. I would offer a first listen review right now, but I’m at Uni at the moment and going to wait until I can download it from somewhere where the bandwidth doesn’t cost so much (a situation I was reminded of the oddity of last week by a new PhD student here from the Netherlands complaining that we have to pay at all. Would Page and Brin have been able to start Google in a CS dept where they were paying 2.5c/MB during the day? But I digress).

There’s a promised CD version coming soon, but the monetization strategy (beyond just generating good will and fan interest) came in another email (downloading from NIN involves handing over your email address. It’s a pretty fair trade):

Nine Inch Nails is touring the US and Canada this summer. Premium tickets for all NIN headline dates will be made available to registered nin.com members in advance of public on sales. Pre sale tickets are personalized with the members legal name printed on the face of the ticket and ID will be required for pickup and entry into the venue on night of show. Pre sale ticket supplies are limited and available on a first come, first serve basis. Our goal is to put the best tickets in the hands of the fans and not in the hands of scalpers and/or brokers. Register at nin.com and check the performance page for additional tour updates.

And then it lists 26 concert dates and venues. It’s similar to the higher quality purchase options that were offered with Ghosts, which went all the way from a $5 download, to a $10 double CD, to a $75 deluxe edition, to a US$300 limited edition collector’s box which probably included a handcrafted figurine of Trent or something. They sold out of all 2500 of those. Clearly Reznor’s realised the value in catering to both the long end of casual listeners and the short end of dedicated fans. He’s now doing what many have predicted will be the best long term new business model for music: give away the music to act as a promotion for the concerts, and be varied in the product range you offer.

Update: Downloaded the album, and it’s great.