Together, these factors make Leo Hickman’s How I became a Foursquare cyberstalker, published in The Guardian on Friday. Lede: “It’s the coolest social networking tool in the world. But is the geo-location app Foursquare a stalker’s dream? Just how easy it is to uncover the intimate details of a complete stranger’s life?”.

Within a small pool of people that I follow the article garnered two kinds of response: Those regarding it an interesting read, and those who think it’s sensationalist bullshit (“‘Using Foursquare to tell people where you are results in them knowing where you are! SHOCK, PANIC’ Sod off, Guardian” — Dot, on the Twitter.)

We will not be discussing the comments on said article. Holy hell.

The article was interesting, and written in a somewhat even handed way, and the central premise that you can use new tools to find out where a total stranger is was clearly demonstrated. This is a problem for Foursquare, certainly. But the article is an infuriating read to anyone with intermediate knowledge of Foursquare for a number of reasons.

A plethora of technical errors and misrepresentations in Leo Hickman’s article

Throughout, Hickman only semi-qualifies a vast number of his criticisms. You read the article and are given conflicting information about what Foursquare does and does not share, and with who. Some things, as written, are outright falsehoods:

• “Glance down at your phone and – as I did with Louise – see the names of all the other users around you within a mile or so”. That is a terrifying feature. Also, it’s not true. Foursquare doesn’t do that, not even second hand through other services. Foursquare only shows you nearby people who are already your friends on the service.
• The writer’s obsession with ‘GPS’ as a buzzword is irrelevant and misleading in the context of a venue-based, check-in–based system. ‘GPS’ suggests that the service reflects each step you take. But Foursquare only uses GPS as a mechanism to look up venues. The sharing of that location is a separate step, with additional granularity. Technically, also, Foursquare doesn’t require GPS. The Mobile Web interface is entirely text search based. At one point, Hickman writes “Foursquare also uses your smartphone’s global positioning system (GPS) to broadcast your precise location to your friends”. That is not correct.
• At points, Hickman writes ‘friends’ in quotation marks, in the same way as you would about “friends” on MySpace. Again, this is misleading. It suggests that Foursquare’s social network is the same network of strangers and acquaintances as other, less sensitive networks like Twitter and Facebook, and it obscures the fact that you are maintaining a different, explicitly authorized friend list.
• “She was the user allowing a stranger such as myself access to the most personal information — photograph, full name, Twitter feed etc.” That is not the ‘most personal information’. Foursquare can share your telephone number with your friends, if you let it, and that is never exposed to non-friends.
• The reference to Jesper Andersen’s scraping hack is accurate, but the explanation that it was an oversight and a bug rather than deliberate sharing, and that Foursquare have substantially changed the interface in response to this is missing.
• Throughout, there is a total blurring of services: It’s really not clear from the narrative where a piece of information is sourced from Twitter, vs. where it’s sourced directly in Foursquare.
• “Some people are even checking in when they’re at home. Think of the implications. It’s crazy.” It is not documented that you can add your home as a venue on Foursquare with approximate (or no) address information, and you can actually replace the map pin to a nearby, inaccurate location (mine is placed on a nearby cross-street, for example.)
• The article refers to the use case of Foursquare being about getting freebies from coffee shops. That’s weird to me, because no-one I know in San Francisco cares about the freebies: It’s the social utility, enabling serendipitous meetings and spontaneous, real-world, social events that people here love. More on this complaint below, though.

Near to the end, as the article begins to conclude, comes this admission:

Louise’s setting on Foursquare automatically tweets her location whenever she checks into a location, which was how I could tell via her Twitter feed, without being her Foursquare “friend”, where she had been in recent days in such detail.

That’s a technically very important difference, the clear omission of which has shaped the reader’s perception of FourSquare.

People are leaking social information

Overall, this article is a misleading representation of Foursquare to the readership of the Guardian. But it is not my intention to tear apart the article or its author, only to document the substantial faults above, and hopefully serve as a reference to debunk those technical faults. But to rip into it emotively would be wrong, because although much detail and attribution of functionality is fuzzy (or wrong), the social problem that the article describes is absolutely real, to the point that it doesn’t really matter that the descriptions of Foursquare aren’t always correct.

The problem described is this: That through using Foursquare in combination with a linked service—namely Twitter—it is possibly for users to carelessly leak very sensitive information about some of their habits of movement.

That’s the actual problem, and it’s real.

The problem is caused by people not thinking about what they do. People are infuriating like that, and yes they’re ‘doing it wrong’, but it’s the truth of the matter. People see the function to pair their Foursquare and Twitter accounts so they do. They see the function to broadcast their check-in to Twitter so they do. They’re in an application context of closed friends, and they don’t consider that they’re spreading information into different places.

This problem is exacerbated by the user’s shallow understanding of Twitter. Plenty of people don’t seem to understand that their public Twitter account can be read by people who are not within their documented friend/follower network. They don’t think about it. I posit that most probably have never had an incident to discover someone unexpected has read what they’ve posted. As such, most are unaware of how far they are spreading a hyperlink to their current venue.

It’s also somewhat understandable that when a user sees the presence of a feature in Foursquare at all, they will extend whatever trust they hold in Foursquare to that integrated service. Twitter is a checkbox. There’s no gravitas to checking it.

It’s Foursquare’s problem. Though I’m annoyed at the article and the way it’s written, and annoyed because I’m again going to have to explain to my parents that I’m not an irresponsible moron, Foursquare has allowed it to be written. Misinformed and lax articles exist because they are written about misinformed and lax users; those people are using Foursquare now. Foursquare has to address the problems that those people face, regardless of how much the technically literate can separate the concerns. Foursquare have to provide assistance to use the product responsibly, because it has been clearly demonstrated that users don’t understand new, complicated socio-technical issues that we take for granted.

Foursquare has to do more

This means: Foursquare needs to explain in their apps more about how people should use it. Foursquare is not an open ended social environment: There are primary use cases, and others that fall outside of the service goals. As such, they can provide both positive education in how to use Foursquare, but also document anti-patterns in such a way that discourages known bad practice in a way that doesn’t hurt Foursquare’s core function, and also encourages the good practice of thinking about what users share in all parts of the app.

Foursquare itself has a secure design (bugs in implementation notwithstanding.) They don’t have to fear hurting growth by discouraging mistaken, over-personal over-sharing. Some options that cross my mind:

1. When adding a friend, emphasize to users that you will see all one another’s check-ins in real time. When responding to friend requests, use descriptive “Share Location” and “Do Not Share” labels that describe the consequence, rather than ‘Yes’/‘No’ buttons.
2. When friend requests come in, Foursquare needs to do more to help you identify the person. A name alone is woefully inadequate. Inclusion of the profile’s URL, and pulling in a Twitter bio would help. More advanced indicators (whether you are already friends on Facebook and Twitter, or if they’re saved in your phone address book) would also be smart. Right now, people are responding to friend requests that consist of a name, and two buttons to accept or reject. My response is to reject anyone I can’t identify, others will not. There’s no guidance. It’s a bad interface.
3. When clicking the ‘Share to Twitter’ checkbox for the first time, expand an inline tip that informs the user that Twitter is not restricted to their Foursquare friends, or even their Twitter followers. Educate the user about Twitter inside Foursquare, because it’s been demonstrated that the user needs that education.
4. When clicking the ‘Share to Twitter’ checkbox when also checking into a venue that’s been categorized or tagged as a ‘Home’, remind the user that this is someone’s personal address that perhaps they should think twice about publishing. Checking into home within Foursquare is fine, because it’s secure within your approved friends. But it’s not the primary case for Foursquare, so encouraging second thoughts doesn’t hurt the service uptake. Furthermore, it encourages good manners and sensitivity toward personal venues. You could actually go as far as disabling the sharing functionality for venues categorised as a ‘Home’, but it would be better to actively communicate, so the user understands the principal—they will inevitably check into homes that aren’t categorised correctly.
5. Encourage the user to use an approximate address when adding their own home as a venue. Again, it’s not a source of Foursquare’s income, so if someone decides not to do it at all, no-one gets hurt.

Ultimately, the interconnection of web services is mis-understood. We take it for granted that it happens at all, and by not taking the time or putting in the effort to explain it very clearly, the scenario in Leo Hickman’s article is not discouraged, and without the service in question filling in all the blanks at their end, article’s like Leo Hickman’s cannot be easily refuted.

What is the scale of the problem?

For all of the technical errors in Hickman’s article, my biggest complaint is actually this: That he didn’t investigate further. As a San Franciscan early adopter of Foursquare, my usage pattern is pretty safe: I only share with actual friends, I remove people when we lose touch, I only share a check-in to Twitter if it’s a non-sensitive location (and also, only when I write a message to accompany the check-in, because anything else is annoying.) I don’t automatically broadcast mayorship or badge achievements.

But whilst my observation is that this pattern of behavior is quite common to the first generation users (and those who came to Foursquare from its precursor, Dodgeball) I have no idea what the general behavior is around Foursquare’s newer generation of users, or when the shift in behaviour occurred. I have every reason to believe that there are entirely different attitudes now, such that I can’t dismiss the article’s cynical main narrative. If careless, uninformed leaking of location information represents the majority of Foursquare usage, then I and my way of using the app is irrelevant. If most people are irresponsible on Twitter via Foursquare, then Foursquare has to deal with the problem.

Unfortunately, the article makes no qualification of how common the problem is. It suggests that the risk of being stalked and of revealing your information is universal. It isn’t. So what percentage of Foursquare users are leaving themselves vulnerable? How many potential targets did Hickman fail to track down? He doesn’t say. That’s a real shame, and a real failure of journalism in the piece.

At the core though: A service can never do too much to communicate with its users (real users, and interested observers that pass through.) Communicate well and you’ll avoid misunderstandings and misuse. Communicate badly and the opposite will happen. Everyone’s a cynic, and some of them have the privilege to publish articles about you. Whether it’s fair or not, the onus is on Foursquare to communicate better, and prevent this scenario altogether.

Edit History:

This article was edited on Sunday 25th to correct the case of ‘Foursquare’ from ‘FourSquare’, add some additional quotes from Leo Hickman’s article, add the suggestion that Foursquare include Twitter biographic detail with friend requests, and make a handful of other tweaks to the prose.

The last few days have been tough for web evangelism. Or at least, tough for those of us who like to be regarded as level-headed.

First, HP purchased Palm, and immediately declared their commitment and belief in WebOS. A great move for both companies for sure, but cue comments like that of @bgalbs on Twitter: “The biggest tech company just bet its mobile strategy on the web”. Similarly, Joe Hewitt proffered “Hopefully we’ll look back on today as the day the mobile web began to eclipse proprietary mobile platforms.”.

Meanwhile, the feuding between Apple and Adobe regarding Flash on iPad took an unexpectedly public twist, with Steve Jobs writing at length everything that John Gruber had already described about defending his platform from third-party influence, but also that Apple are choosing to invest in native running open standards. Jobs incorrectly brands this ‘HTML5’. He also criticises Flash for being proprietary whilst evangelising the H.264 video codec (licensing for H.264 is not ‘open’, either.)

Adobe hustled a number of responses. Rapidly, a live interview with CEO Shantanu Narayen. Whilst everyone mentioned so far is guilty of muddying terminology, not saying exactly what they think they mean, and/or some general platform and framework-related confusion that I’ll elaborate on below, Mr Narayen is the first to throw some incomprehensible bullshit into the day. He refers to “open content”. What is that? Content wrapped inside an impenetrable, proprietary, single-vendor container format? Flash, by design, locks content away (in exchange for other supposed benefits.) Comments like “Flash is an open specification” is unhelpful too: The page hosting the Flash specification doesn’t specify a license, the document itself just exerts copyright and disclaimers on behalf of Adobe. Furthermore there are no native or competing implementations of Flash, nor is the development of Flash open to participation.

Adobe has for years relied on bullshit merchandise to remain relevant in the ‘future of the web’ debate. Three years since the launch of the iPhone visibly snubbed them, they have not shipped an acceptable, functional version of Flash on any other handset. If the upcoming Android release doesn’t perform well, there will be no reprieve.

Web Evangelism got hard. When Zeldman wrote Designing With Web Standards, there were two standards that mattered: HTML and CSS. They were the path to cross-browser compatibility, and thus they were the route to a brighter future (both for design, and for information publishing.) 12 years later, holy shit. JavaScript joined the standards party, and then became quickly obfuscated by frameworks. CSS3 stablised, Webkit extended CSS further (and it’s still called CSS3), a plethora of new standards on the server and client: OAuth, OpenID, Contacts, Connect, Geolocation, microformats, widgets, AJAX, HTML5, local storage, SPDY, ‘The Cloud’…

All of these things are vying for attention and evangelism. Some of them are great, some of them are stupid, but they’re all clubbed together under this vague banner of ‘The Open Web’. It sets expectations and demands from developers, who are all the while being wowed by the efficiency and quality of proprietary application frameworks like Flash and Cocoa.

Besides memory, the half-life of most things on the internet is short. Frustratingly, terminology is no exception. From the coinage, we lose meaning and specificity within a few years, sometimes only a matter of months.

‘HTML5’ now refers to a collection of related client-side technologies, branded together as a product. It is no-longer just a hypertext specification document, and everything that concerns document semantics is being ignored anyway. This is usage that Steve Jobs employed this week.

‘Open’ is lost.

• • The H.264 video codec is not open. It is patent encumbered, and there is a financial obstacle to license those patents. Yet, despite this, H.264 is bundled in with discussion of an open web stack. In practice it may be ‘open enough’ to function on the web. Operating Systems and Browsers will license it for playback, and the authority that grants the licenses is continuing to allow free (as in beer) usage of the codec by web publishers. But a change to those terms, and eventual enforcement of fees on big publishers is possible.
• • Adobe are certainly full of shit here, though. Absolutely full of it that it makes me angry. ‘Open content’?! Locked up inside Flash containers? That’s as closed as can be. At least if you uploaded it as a flat image you could run content through OCR it and recover the text! Let’s not get stated on their open screen project. To Adobe, ‘open’ means ‘cross platform, on platforms that Adobe provides support for, for as long as Adobe chooses to support them.’ That’s very different. Their use of ‘open’ is an outright lie.

Then, there’s ‘web’.

On this word rests most of our understanding (and misunderstanding) of what can and can’t be reasonably achieved with applications in the browser, over HTTP. What does it mean to be a ‘web application’? What are the expectations of a web application? Or of any web content, for that matter?

Off the back of the Apple verses Adobe mudslinging, Joe Hewitt went on a very, very long rant about the state of ‘the web’ as an application platform.

In parts, I agree, certainly with regard to the process of standardisation. Standards are supposed to be derived from implementation, and standardised technologies will be better for real-world iteration. Heck, our entire microformats process is based around codifying examples in the wild. The failure of CSS to define a layout syntax (choose from three) illustrates the need for implementations to lead.

However, the suggestion that Microsoft were bullied out of innovating the web, or that developers should pursue whizz-bang APIs to the point of single-browser dependency leaves me sour. This is the short, perhaps selective memory that the internet suffers from. It is not acceptable to me that 21st century knowledge retention has become so short and shallow as to be overwritten by influential ranting on Twitter. A greater tool for the dissemination of misinformation has never been known.

For those too young to remember, IE was innovating like crazy from 4.0–6.0, right up until the DOJ and web standards commies intervened.

Microsoft Internet Explorer did not stop innovating because of ‘standards commies’. That’s offensive and wrong. Web standards advocates went after Microsoft because they failed to adequately support basic, foundational web standards like CSS, necessary to publish an interoperable web of information in which Firefox and KHTML and everyone else could even compete, let alone succeed. That early CSS push was also a vital, huge step in making web content universally accessible beyond visual media.

Microsoft’s development of enhancements like ActiveX, and XMLHttpRequest were not being prepared to be standardised for the web with W3C participation, they were invented in such a way as to inject proprietary, Windows-only code into the web. Tools not to make to web better, but to make it dependent on Microsoft. Chris Wilson has since written more accurately about the plateau in IE development on his blog.

Contrast Microsoft’s method in the 90’s to the more recent CSS efforts coming out of Webkit: Extensions to CSS are followed promptly by proposed specifications (such as CSS Animations). Microsoft, from the top down were trying to ‘own’ the web. (More recently, to their immense credit, they’re in the right game. The aforementioned CSS Grid Layout module is from Microsoft engineers.)

Meanwhile, Sachin Agarwal is writing that The Web Sucks, again talking about the web as an application platform:

Web applications don’t have threading, GPU acceleration, drag and drop, copy and paste of rich media, true offline access, or persistence. Are you kidding me?

There, in that quote, is where I want to pull all of this together. Sachin’s complaint has absolutely nothing to do with the web. Think about that word; ‘web’. Think about why it was so named. It’s nothing to do with rich applications. Everything about web architecture; HTTP, HTML, CSS, is designed to serve and render content, but most importantly the web is formed where all of that content is linked together. That is what makes it amazing, and that is what defines it. This purpose and killer application of the web is not even comparable to the application frameworks of any particular operating system.

That’s the kicker. We talk about ‘web applications’, the ‘open web stack’. People are citing HP’s purchase of Palm and investment in WebOS as a victory for the web. We talk about applications built using HTML, CSS and JavaScript in the same breath as content published using HTML semantics.

Want to know if your ‘HTML application’ is part of the web? Link me into it. Not just link me to it; link me into it. Not just to the black-box frontpage. Link me to a piece of content. Show me that it can be crawled, show me that we can draw strands of silk between the resources presented in your app. That is the web: The beautiful interconnection of navigable content. If your website locks content away in a container, outside the reach of hyperlinks, you’re not building any kind of ‘web’ app. You’re doing something else.

Palm WebOS applications are awesome, but they are not part of the web. An app might interact with data on the web, and they are built with similar HTML, CSS and JavaScript technologies. That’s great, but they are not a connected, interlinked part of the web.

We’re talking about two very different things: The web of information and content, and a desire for a free, cross-platform Cocoa or .NET quality application framework that runs in the browsers people already use. The latter cause is louder, and risks stomping over the more valuable, more important, more culturally indispensable part of the web.

I’m not saying that better, more abstracted JavaScript APIs are unwelcome. I’m not saying that APIs to access devices like webcams and microphones aren’t useful or even important. Even WebGL, as far out of my field of interest as it is, would be a great thing.

But, the open publishing nature of the web, and the requirement that information be accessible, will inherently cause it to lag behind other platforms. The success of the web, the success of this impossibly huge network of information is because of the open, universally accessible, cross-platform, cross-device nature of web content. Cross-platform user interface sucks. It’s a nightmare of inconsistency and wrong, momentarily obsoleted assumptions. But cross-platform content? Well that is content. It’s articles and poems and pictures and movies and music, everywhere! How brilliant is that! Calling for browsers to make massive proprietary advances (even with the caveat of standardising later), suggesting that users should tolerate swapping between browsers, or even devices, to access particular content because you’ve obscured it behind a bespoke API is an absurd throwback to days we’ve left long behind; a proposition that would result in information and culture being locked away. Nothing is worth that, especially not ‘web applications’.

If you reach the point of building a browser-based application that you depend on so many proprietary enhancements that your users can only access it using Google Chrome, I think you’ve picked the wrong platform. If you want to built the most amazing user interface, you will need to use native platforms. A single vendor’s benevolent curation of their framework will always outpace the collaborative, interoperable developments of the web, whether it’s locked in a standards process or not. When they do a good job (like Apple have with CocoaTouch) their platform will succeed. But the web will always be the canonical source of information and relationships. That’s what it was built for. Blogging at length about how much the device APIs suck won’t ever undo that, nor change the fact that turning HTML in a rich application dialect is still a very new idea.

Personally, aside from all of this almost ideological disagreement over what the web is for, and what you can reasonably expect it to be good at, I honestly think that ‘Desktop-class Web Applications’ are a fools folly. Java, Flash, AIR and QT demonstrate right now that cross-platform applications are always inferior to the functionality and operation of the native framework on a host platform. Steve Jobs is right in his comments that third-party frameworks are an obstacle to native functionality.

Creating generic, non-native interfaces that run in a web browser won’t be any better than the cross-platform applications that came before (see Cappucino.) The idea of undermining the core function of the web to achieve that is detestable.

But first:

If I had an iPad rather than a real computer as a kid, I’d never be a programmer today.

—Alex Payne

Alex’ quote has been whipping around today, and when I first picked it up had been distorted to imply: “If I had only an iPad, rather than a real computer, and there were no open computers nearby, or at school, I would have never learned how to programme.” That’s a bit different. Thanks Twitter.

Alex goes on to talk about hacking on the machines he grew up with. Breaking them, repairing them. That’s a kind of activity that is dead and done in a closed, DRM-encrypted encrypted environment, certainly. However, the sentiment that has been echoed on his behalf (and I don’t think this is what he meant), is that people would not become programmers at all. That’s bullshit. Even if every computer on earth was a closed environment, people would be learning to programme, it just wouldn’t be the same way they do it now.

If root did not exist, it would be necessary to invent it. In a world with only closed systems, commercial success would still be tied to the development of third party apps, but the responsibility to train developers and introduce people to programming would fall to the gatekeeper of each system, rather than it being something that can organically happen. So, Apple, or Microsoft, or whoever, would be shipping programming sandboxes for their devices; maybe scripting tools, maybe programming games, maybe cut-down versions of the professional development environment. Whatever it would be, if there was no open ecosystem to learn in, a closed system to learn would be created. The idea that people would not become programmers in 1995 because they can’t do everything to a single closed system in an otherwise open industry today is a fallacy. If the first computers had been closed, everything would be different, right down to the seeds that inspired our initial passions for software development.

Which is not to say that I think everything would be fine. I think we as an industry, as a species, would be less creative and less innovative without 25 years of open computing. Our world and lives would be worse. But we would still have new programmers. And that is all I have to say about that.

Now, the possibility that we might find ourselves moving into a period of closed devices throws up a number of obvious fears: Apps getting rejected by gatekeepers, and the physical capabilities of devices being locked away behind private APIs, plus the inability to add new capabilities to a device without licensing a proprietary dock connector are three obvious ones. They are not my main cause for concern. They’re big roadblocks, but they’re also all enforced by policy and could simply cease on any given day.

Allow me to talk about some of favourite software on Mac OSX

Megazoomer is an add-in for Mac OSX. It’s a single feature, and once installed it injects a new piece of functionality into every Cocoa app; under the View menu, you can now choose Mega Zoom (or press ⌘↩. The window smoothly scales to true full screen, the Dock and Menu Bar slide out of the way, you can switch any application into a tranquil, productive WriteRoom.

Inquisitor was a beautiful hack that added auto-complete search suggestions, inline search results, and search provider customisation. It predated the simpler, native search suggestion that now ships in Safari 4, and is still more polished (without being bloated) than any other search suggest feature I’ve seen. It stopped being maintained when Snow Leopard forced a change to the Input Manager integration method, but Glims is a similar replacement.

The Microformats Plugin for Safari adds an hCard and hCalendar parser to Safari, and shows an alert when you visit a page containing embedded data. It lists the items, and lets a user add contacts and events to their native Address Book and iCal applications. It was simple, worked pretty well and was a hugely valuable demonstration of how microformats could be exposed in user interface (I wrote similar thoughts about this sort of UI at the same time.)

Airfoil is an application for OSX (and Windows) that allows you to stream any audio, from any application, over your local network to an Airport Express audio hub. It’s a feature of iTunes (AirTunes) that allows you to stream what you play there to Airport; this app makes it system-wide. It’s a fully supported, commercial piece of software, but one of the ways it works best is by injecting a piece of code into applications called ‘Instant Hijack’; a hook allowing Airfoil to grab audio from a running application right away, rather than having to relaunch the application. It’s very, very slick.

I want to draw particular attention to the fact that all of these apps are add-ins for the native, closed-source Mac OSX environment. They’re not changes to source code, nor do they use featureful extension mechanisms, like the plug-in interfaces of web browsers. These are hacks, add-ins that use a feature of the Mac’s Cocoa framework, able to change user interfaces and inject functionality in a raw way, at run time.

Reading the list, the functions I’ve highlighted here seem small. They’re enhancements; improvements; subtle. In these cases, their creators have developed single features and added them to existing applications. In doing so, they have prototyped and experimented with web browser functionality that may become native and expected in the future, but right now it’s new, and to those who install these hacks, they sink or swim based on whether your productivity improves.

In the environment of the iPhone, or the new iPad, it is this kind of development that is completely lost. Each app, locked up inside its own signed bundle, unmodifiable by design. If someone wants to ship a better search interface for a web browser on the iPad, they have to ship an entire browser. If someone want’s to ship user interface for microformats in the browser, they would have to build that browser. Some of the earliest demonstrations of microformats came from add-ins like Tails for Firefox, and later the Safari equivalent. The early success and adoption of microformats simply would not have happened without hackable software; allowing a single feature to be augmented onto a greater whole. The innovations of microformats themselves, the implementations we now see in search engines, as well as the momentum in other structured data mechanisms (RDFa, microdata) can be traced down to early, enthusiastic adoption of practical features like Tails, Operator and the Safari Microformats Plugin. Those plug-ins were not just innovative in themselves, but they supported a much broader effort.

In the case of Airfoil, though the iPad plays music through iPod or Safari, there’s no native AirTunes offering. If there were, you’d be beholden to Apple for it to function in third party applications. If they don’t ship it, no-one can.

None of these add-ins will exist on the iPhone OS for as long as it remains closed. Of course innovation will happen, of course amazing, standalone applications will be built, but that’s just it, they stand alone. Dreams of amazing features cannot be built without also building the whole. The nail for anything even close to this functionality is that current App Store politics dictate that you cannot ship applications that themselves interpret code; it’s not permitted for third parties to include extension capabilities in their apps.

No hacks. No scripting interfaces. Just apps. Many exciting things will still happen on this platform, even without policy changes. But I offer a lament to the the loss of extensibility.

The Web

There is one application on the iPhone that supports user scripting, and execution of user extensions, and that application is MobileSafari. Safari, like all web browsers, supports bookmarklets; strings of JavaScript code embedded in a URL, prefixed with the javascript: pseudo-protocol. The JavaScript code executes when you select the bookmark, and is able to manipulate the context of current page, and navigate to new pages.

The iPhone bookmarks interface is fairly clunky, and adding bookmarklets is impossible without preemptive effort by the writer, quoting instructions for installing Twittelator’s bookmarklet on iPhone:

I first had to navigate in mobile Safari to the Twittelator iPhone bookmarklet page and follow the lengthy instructions there. Basically you have to save that page as a bookmark, and then go back and edit that bookmark to delete everything before the javascript:window.location=%27twit://%27+window.location part of the URL. Once you do that, the bookmarklet executes the steps to post to Twitter.

—Safari iPhone Bookmarklets by Amy Gahran

Not very friendly. The other way to get Bookmarklets onto an iPhone is to add them to regular, desktop Safari and then synchronise your bookmarks using iTunes. As such, the Tumblr bookmarklet for blogging runs on my iPhone, since it’s in my regular browser too. Others have used JavaScript to recreate the useful “Find in Page” functionality you find in desktop browsers. Lifeclever documents that and others.

Since Bookmarklets run in iPhone Safari, it is my assumption they will also run in Safari on iPad. The bookmarks interface there is more conventional; a long menu that pops up over the current page; and as a menu, far more intuitive to pick functionality from. We’ll see whether there’s an easier way to add bookmarklets on the device itself.

The scenario that remains is this: The only extensible software on the closed iPhone OS is the Web. The only place you can enhance an application, rather than replacing it in entirity, is the Web. The only place your feature and someone else’s feature might co-exist is through two bookmarklets on the same website.

What we are heading toward is a reality (hopefully temporary, but I suspect for years) where the Web is the only extensible platform on Apple’s consumer devices. There are already components in place for building a JavaScript development environment within the browser; Bespin is a rich code editor, Firebug Lite is a JavaScript bookmarklet to add a DOM explorer and inspection tools. A web application that lets people write and execute code, and inspect the result, is the only real hacking tool we can offer the actual users of these devices.

It is interesting to me as an advocate of the Open Web that the tighter grip on the native environment drives people to web standards as their only alternative (both technical and political motivation.) But, as an admirer of Apple’s very polished native UI, and of the Mac hacks I mentioned at the beginning, I’m still disappointed.

It’s an exciting, inspiring time. If we are brave, then the imposition of these restrictions can only make us smarter when we work around them.

In summary, the iPad is an extremely well designed, meticulously constructed consumer device. The iPad is something which I think will fit perfectly into the life of someone with desktop computer at home. There, it provides lots of new functionality, and will prove useful a lot of the time. It’s more convenient, and will enable people to be more social with their cohabitants whilst still having a connection available. It is a device ideal in both size and price for integration information and media access with home life, rather than home-office life. My Father, sat in the living room, is regularly poaching my sister’s iBook, looking things up on the internet; searching for prices, fact-checking my siblings, keeping up with sports scores, weather and travel information. For him, an iPad would be a perfect addition (though, I think Jeff Croft makes a good point with regard to multi-user support; the living room and kitchen scenarios Apple presented on Wednesday set the tablet in a clearly shared environment, whilst providing access to personal information.) Overall, I’m very impressed.

For me, my caveat is that I already use a laptop on the sofa; that makes the iPad a harder sell for immediate adoption. I’m already sat on my sofa, computing. I’ll be waiting for functionality to be added to the Tablet over the next six months—from Apple or from third parties—and looking to see where it can really add something to my life. I like the form factor.

Every change we make to our email behaviour, whether it’s organisation, tools or simply trying to email less, we are trying to find an effective balance. I’ve just come out the other side of one change, and am spending a few weeks making a strong, conscious effort in response.

At this point, my email inbox tends to receive the following: Notifications (from services), content summaries, status changes, reminders; pushed information, newsletters, aggregate news; actual communication, enquiries, conversations, planning and social synchronisation.

My email clients of choice is Mail on Mac OSX, and Mail on my iPhone. Previously I used Gmail from Google, but I quit the service because I found the browser-based user interface clumsy, and the application as a whole aesthetically ugly. Mac OSX’s user interface is elegant, the display of email good for reading, and the desktop client remains robust. My email is on an IMAP server (because that’s how email should be accessed), and I can fall back to my service provider—Fastmail —for a clean and functional web interface if I’m ever away from my main machine. I have also developed a certain wariness of storing too much of my data with Google for free, and prefer to pay for the excellent service I now get.

Gripes with Gmail as an application remain, but Gmail did massively inform my email behaviour. Most importantly, archiving rather than deletion. I carried over that practice to my new host, and now have some four years worth of email archived on my machine, and backed up around the world. I can, if needed, find pretty much any piece of mail given the right parameters. Not that I will ever need to see most of that mail again. But archiving is easy for a while, and my compulsive nature makes thorough archiving desirable.

I organise my mailboxes annually. Each year I archive everything from the previous year into an annual parent mailbox, effectively removing all the top-level mail folders from my account hierarchy. I start each year with a bare ‘Inbox’, ‘Drafts’, ‘Sent’, ‘Archive’ folder set, and then the categorical mailboxes get recreated when the first relevant piece of mail comes in. Some get recreated within a day, others never come back. For what it’s worth, I’ve always found a simple tree structure to be entirely sufficient for organising email; I never proved multi-dimensional tagging to actually be useful in Gmail.

Although my archive routine is simple, it is slowing me down. Huge amount of archived data, most of it useless. So I have all of my domain renewal notifications? Why? To keep a record of when my registrars have been in contact, so that I know exactly when I was reminded of particular actions. That is an example of me choosing the simplicity of the archive action instead of making a good-value decision about what is worth archiving. I’ve now accrued too much information to effectively dig through.

So, my first email goal from now on is to archive less. To trash messages that are of low value. Being an archivist, completist and pedant is an unhelpful email trait.

Years of signing up for services means that I get a regular stream of automated or blind-mailed content from services. This email I do delete. But because the effort of performing a delete action is so low I have tolerated an increasing quantity of mail for years. The processing cost is low, but occurs frequently and at irregular intervals. The problem is exacerbated in a mobile environment, where a ‘new message’ notification triggers a significant interruption. That interruption would be appropriate for valuable, personal communication.

As well as notifications, I receive a small amount of actual content by email as well; most notably the daily Snowmail email from Channel 4 News in the UK. Unfortunately, that content is only distributed by email, and is not published as a blog. This is content I read, of value, but is not content that should be pushed to me, it’s content I should pull up at my convenience.

My second email goal is to reduce the amount of email I receive in my inbox to be 95% personal communication. This has meant that every day for the past two weeks, I’ve actually taken the extra time to find the unsubscribe link in the footer, or edit the communication preferences of whichever service is sending notifications, and permanently disable it. I suspect it will take months to clear all of them, since some services are very infrequent in their mailings, but already I’m finding that if my iPhone vibrates, then the message I’ve received is now most often of actual interest.

Newsletters require more work. Email is the wrong delivery mechanism; they should not be push content in the first place. So I’m going to experiment with subscribing a blog-by-email address to the mailing list in question, so I can convert the content into an RSS feed and have it consumed by Fever instead.

There is value in email, and that value is communication. Over years, service providers and publishers have taken advantage of email’s ubiquity to adapt it for push, notification and automation. Better solutions to those use cases are emerging (or already exist), so this is the time to reclaim the inbox, reduce your email throughput back to what the medium is really good for. I’ve already seen that a little bit of persistent effort can greatly increase the quality of email as a tool.

It doesn’t look good for Flash. But Flash isn’t going to die.

Firstly, consider some background. Flash has been around for a very long time, providing a platform for games, vector graphics, animation and media playback. It offers massive market penetration, and of course there is only one Flash player, so it purports to offer a consistent experience across platforms.

However, Flash comes with a number of significant downsides. Firstly, that ‘consistent experience across platforms’ might also be written as ‘inconsistent experiences on every platform’. By taking most of its user interface conventions from Microsoft Windows, the experience of using Flash on a Mac is not the same as using any other app in OSX using native controls. Simple stuff like the behaviour of keyboard chords in text entry controls are different, the text selection colour doesn’t match the system selection; low level stuff.

Similarly, the Flash plug-in itself is cited by Google and Apple as being a massive cause of crashes in their browsers, crashes that the browser maker gets the blame for, but is really caused by Flash. In Google Chrome and in OSX Snow Leopard, plug-ins are sandboxed away from the main browser process. Mozilla lists Flash as the biggest causes of Firefox crashes on Linux.

To top it off, for all the claims of ‘Accessible Flash’ over the past year, Flash content is still only accessible on Windows; on Mac OSX, Flash has no integration with VoiceOver. Screen readers are only supported on Microsoft Windows through the MSAA API.

The iPhone shipped two years ago without Flash support. At the time, some said it was a ‘missing feature’. 24 months later, the iPhone seems to be doing just fine without Flash, and users seem very happy. Adobe have stopped making vapourous comments about having ‘Flash for iPhone’ waiting in the wings.

There are two distinct threads to a Flash vs. HTML 5 discussion. Those are ‘Features’ and ‘Philosophy’. Let’s tackle them separately.

Features

HTML 5 is gaining mind-share because of a handful of key new features that it offers: <video>, <audio> and <canvas>.

The first two are quite self-explanatory, they are new elements dedicated to providing video and audio media directly in the browser, and provides a DOM API for controlling the media from JavaScript. Note, the idea is that the media is played back directly by the browser, not through a plug-in like Quicktime or Windows Media Player (which is how video used to work, before Flash).

This affects Flash because over the past few years, perfectly timed with the rise in available bandwidth to stream audio and video, it provided a solution better than the Quicktime/Windows Media/RealPlayer mangle that came before it. Before, to embed video in pages you needed to provide multiple codecs, depend on bespoke media player UI appearing in your page (all of which was different sizes, and so would break your layout), and half the time your visitors had the wrong version of the plug-in anyway.

Flash stepped in with a solution: Support for more platforms than any one of the other bespoke players, and you could design your own playback UI around it, too.

Flash won video from under the feet of Apple, Microsoft and Real by building something that was better, and bypassing their squabbling over codecs.

But, it’s just a better, bespoke solution. It’s still vendor dependent. Flash provided the use-case for ‘embedding video with author-defined playback controls’. The purpose of standardisation is to take that feature and define it, such that anyone can implement it. From there, comes video and audio in HTML5.

Flash also provides vector drawing tools. It’s another useful use-case (graphing, interactive charts, etc.) Again, the standardisation process for HTML is about taking the use cases from real content on the web and defining it so many people can implement it. canvas (via. Apple) is the implementation for that.

Three major pieces of functionality. Putting them natively into the browser responds to the needs of web developers. That’s what the standard is for. Does this mean that HTML 5 ‘kills’ Flash because previously Flash-only functionality is now native? No. But it means that those major use cases no-longer require Flash. There’s plenty of other, less trivial functionality that Flash supports for which widespread demand does not exist. But of course really common features of web pages are going to be supported in Open Web technology.

Additionally, you may cite sifr — using custom typefaces — as a use-case for Flash. That falls outside of HTML5, but is covered by an increasingly well supported CSS3 feature, @font-face.

Philosophy

I’ve avoided linking Jon Dowdell here as a major source because although he titled his post ‘Adobe on HTML5’, his blog also states that his opinions do not represent his company. His post is representative of Adobe’s general philosophy toward the web, though.

As far as Adobe are concerned, Flash is part of the web. It’s not just an optional, bolt-on plug-in for proprietary content. To Adobe, Flash is as much a part of the web as JavaScript or CSS. They regard is as a legitimate part of the stack.

“the “HTML5” publicity helps marginalize those few who still argue that images, animation, audio/video and rich interactivity have no place on the web. Flash will be able to deliver on those heightened expectations, regardless of what each separate browser engine does.”

The second part of Adobe’s philosophy is that consistent support for functionality on the web is non-negotiable.

“we really do need the ability to predictably deploy advanced capability across a range of device brands and browser brands”

This philosophy is wrong. One: Flash is not part of the web. The web is the Open Web and anything closed and proprietary is just riding on its back. I don’t mean that bespoke plug-ins are unwelcome or even ‘wrong’; they provide all sorts of useful functionality. I do mean that if you are a single-vendor creator of a proprietary, patent and license encumbered, undocumented, closed-source plug-in then you have no claim to be part of the infrastructure of the web. The infrastructure, from TCP/IP upwards, is open.

The consistent support aspect also flies in the face of techniques used with every part of the Open Web stack: Graceful degradation, progressive enhancement, and the fight against the misguided demand for pixel perfection are all battles that have been fought and won since Designing for Web Standards.

The web is about content. Everything above that is dressing (perhaps think of the web as fresh bread, perfectly coated in balsamic vinegar and olive oil). The fact that older browsers cannot render all the features of your page but can still provide the content to users is a feature. It’s the most important feature.

The Flash philosophy is opposite. Flash is about a complete experience (singular). It’s about every detail being precisely bevelled into place for every viewer. The consequence of this approach is that it resists the availability of content. The goal of perfect consistent rendering can only be achieved through a single version of this single vendor’s bespoke plug-in. If you need a feature of Flash 10, Flash 9 users must upgrade to see any of your content, not just the new feature.

The Flash approach is anti-content; anti-web. Adobe present the idea that Flash is a superior offering because the entire suite of features, in one big blob, is a compelling development offering. But the reason to write on the web in the first place is to make content available broadly.

In recent years, through multimedia, fonts and and vector drawing, we’ve seen more and more blocks of content moved into Flash, in the absence of a robust standards-based mechanism. HTML5 redresses that by supporting those use cases. HTML4 supports pictures. HTML5 supports moving pictures. HTML5 supports what people publish on the web.

Fuss

What is the fuss about? HTML5 doesn’t compete with Flash as a product, (you could never produce an implementable specification for so much functionality in one go). It just takes some specific, common use-cases for web content and supports them.

Yet, people on one side are crying for the absolute death of Flash, and clearly some from Adobe are on the defensive to outright dismiss the HTML5 effort.

Critics may be motivated by any number of those negative user-experiences this article opened with, but Flash won’t die. If HTML5 takes away one use-case that Flash fulfils, Adobe Flash will add new features that browsers don’t have. That’s what plug-ins have always done. Flash can and will iterate faster than browsers and can deploy wider all at once. That said, some of those existing use cases — namely video playback — are extremely lucrative for Adobe. Video took Flash from ‘optional’ to ‘essential’ for a certain slice of web content. The video sharing industry is dependent on Flash.

Adobe will lose their exclusive grip on that. But, what did they expect? That a massively profitable industry would tie themselves to a single vendor?

Flash offers only one advantage to video on the web, and I think this one will be genuinely interesting to see turned into a marketed feature. The HTML5 method of embedding video looks like this:

<video src='http://example.org/video/foo.mp4'></video>

There’s the URL to your video file, right there in the HTML source, downloaded in raw form. What can Adobe offer publishers? Two ‘features’ of software that run absolutely counter to the principals of the open web: DRM and obfuscation.

That could be interesting. The survival of Adobe Flash Video online will require them to take the closed, anti-content consequence of Flash’s model, and instead embrace it as a feature for media companies that fear distribution of their content.

Really, I think this whole issue is overblown. Maybe it’s all fuelled by scare-mongering from individuals Adobe, maybe it’s over-eager Open Web evangelists eager to see closed and proprietary scraped from the face of the web. In reality, it’s just the pragmatic, ongoing evolution of the web offering useful new functionality.

I was there for my work with microformats, and as someone who periodically pops up to suggest improving the user experience of OAuth.

People

MySpace, Facebook, Yahoo, Google, OAuth, Portable Contacts, Activity Streams, Open Social and OpenID. All under one roof. Everyone quite keen to move in positive directions. An exciting mix.

It was quite something to spend two days in these surroundings, participating in the conversations that determine what happens next. As is my nature though, perhaps the biggest value came through observation and listening. And to cut to the chase, herein lies my principal learning of Foocamp:

The Open Stack needs a Product

Foo had many, many sessions on the Open Stack; OpenID, OAuth, OpenSocial and Portable Contacts. Some of them concerned with user experience, some with data portability and distributed identity, some with the processes of open development itself. Every single one of those sessions at the very least mentioned Facebook. But more often, the sessions were outright dominated by Facebook Connect.

A session on building a start-up on the Open Stack was turned into a discussion about Facebook’s product, and what developers wanted from it.

Note the emphasis on Facebook’s product. The way in which we classify the technology of the open, social web and compare it to Facebook Connect is massively flawed.

Get this clear: Facebook Connect, from inception through API through user experience, is a single, self-contained, beautifully packaged product for developers. And it’s awesome. Facebook has the combination of detailed, well maintained user data, a huge user-base and excellent user interface design for the Connect experience. It ticks every box.

Compare to ‘the Open Stack’. There is no product. These are technologies — wonderful technologies — with which you could build something with the functionality of Facebook Connect. But at time of writing, there is no mature offering. The branded products from Yahoo and Google are not as strong as Facebook; they’re less mature in every way. Problematically, though, the tools have a stronger brand than the implementations.

It’s this that causes the Facebook/Open Web comparison to fall down so quickly. The open technologies are right and true. Using the same open auth and identity protocols is a massive win for developers. But what are you actually implementing?

The open stack itself doesn’t contain any data, nor provide any service. It is just the mechanism to provide those services. You don’t solve anything by ‘integrating OAuth’. OAuth isn’t a service. The publicity has to shift to actual service providers, where the end users are involved. Because really, it’s touching those end users that drives developers, not beautiful snowflakes.

We’re all imagining a world where you can implement OpenID+OAuth+PoCo and seamlessly integrate with Google, Yahoo! and any other social network using the same code. But that doesn’t exist yet. Only the foundations of it exist. And without the data provision from actual products, there’s no implementation to focus the open stack discussion on.

Not broken

Whilst it all sounds a bit bleak, nothing here is broken. Facebook has a massive head-start in the marketplace. Yahoo, Google, MySpace et al are playing catch-up in terms of the APIs and the user experiences of their own sites. Is Yahoo! Updates as rich an experience as Facebook? Not yet, no. There’s work happening everywhere to compete, across all aspects of all services. As such, of course Facebook is the more compelling option this year; it’s obvious that’s the case.

Further, as evidenced by Facebook’s Open Stream API launch last week, their strategy has been formidably well planned. Over the past twelve months they’ve been hit with sticks by openness advocates for being locked away in their walled garden, but their priorities have been elsewhere. They’ve been building a rock solid foundation that, once in place (now), they can start to open up and offer good data from the start. They have the luxury of the market lead, and they can use that to release better, more complete services. That’s their reward for being first, and they’ve earned it.

So, the feeling I come out with is that we should stop thinking about Facebook in the context of open standards (except where they implement them, of course). It’s a broken comparison. It’s hard, because the competitors have everything in the air at once, but it’s down to them over the coming months to turn their adolescent, open-powered APIs into compelling products. The part OAuth plays in this is just to continue becoming as transparent to users as possible.

It’s not the job of OAuth and OpenID as part of this ‘Open Stack’ to take on Facebook in mindshare. The roles of these APIs (PoCo, Open Social and Activity Streams included) is to be expected and taken for granted in any new implementation. These are the bricks on which houses are built. But people don’t buy bricks, and so our eyes need to focus on the products of this work.

The other point to stress here is that whilst the open stack needs stronger implementations, they don’t have to be ‘NotFacebook Connect’. That’s only one use for these standards. The big Open Web offering could be somewhat different. Better, even.

Be excited. The struggle of Open standards vs. Facebook is a fallacy, they’re just efforts a little out of sync. This year, with maturing, big products powered by open technologies, we’ll see things built that extend beyond the achievements of Facebook’s walled garden age. MySpace, Google, Yahoo! et al are all moving together toward something quite special. Developers will be able to take on exciting new, provider agnostic apps with this technology. Just accept that the second generation of competitors need a little time and encouragement to build out.

Oh, and don’t be surprised to see Facebook active in all this, too. In the end, they’ll be as open as anyone else.

Microformats.org is an interesting beast to work for. An informally arranged organisation of volunteers, overseeing a broad array of subject areas and points of interaction. 2008 was my first full year of administrative involvement with the group, for what value of ‘administration’ there really is.

This post is on my personal blog because there is no official line of policy at microformats.org. What I write here is just personal intent and what we achieve in the next twelve months is down to shared passions and collaboration, not the will of one person.

There are shared priorities, of course. The past few months have seen a surge of work on the awkwardly named value excerption ; a mark-up pattern and parsing rule derived from hCard. Honestly, I didn’t know ‘excerption’ was a real word until I started leading the work on this. Thankfully, naming is not as important as a good spec.

Basically, value-excerption in hCard got implemented in parsers globally, so we’re trying spec it more fully to reflect that. It’s a pattern for structuring data values, so in the process we can extend it do something to offer solutions to some long standing accessibility and localisation complaints. The work is sporadic; two weeks here, a month off there. That’s just how it happens. Being absent from Yahoo! this last month has helped me pull it together into a massive public test effort.

My other big task in 2008 was redesigning the microformats wiki, bringing it into line with the look and feel of microformats.org, adapting Dan Cederholm’s still-lovely design. It’s a piece of work I’m proud of, and besides being able to junk vast quantities of MediaWiki’s questionable and bloated default mark-up, it of allowed me to put microformats into the wiki mark-up itself: Each page is now an hAtom entry, with an hCalendar event for the last-modification date of the page.

This Year

I don’t care to dissect last year too heavily. It’s this year I’m excited about. There’s work coming to completion, there’s ongoing work that’s nearly ready to break cover, ongoing infrastructure improvements brewing and a desire to see a big step up in microformats toolkits for developers.

1. First up, I want to see the value-excerption work seen out within the next couple of months. Testing is going really well right now; it’s an effort beyond the scale of anything else we’ve done before. Knowing the accessibility and localisation issues we’re trying to overcome, it’s vital that we get it right. We can’t afford to push something that doesn’t solve the problems and complaints of authors as well as we can. I’m taking suggestions for a beer or red wine magnificent enough to open when we call this one ‘done’.

2. Secondly, we’ve built up a number of issues and enhancement requests against the core microformats — hCard and hCalendar. They’re stable, useful and are helping to change the web, but iterating stably is an important step to take as the community and formats mature. Just as HTML5 is not versioned like a piece of software, there won’t be an ‘hCard 2’. This is the web and we won’t be breaking existing pages or forking our specifications; that’s absurd. We will evolve. I would like a period of active editing and hope to see hCard and hCalendar ’Second Edition’ published this year.

3. Recipe and Audio formats. Two new drafts in 2008. Bearing in mind that many popular and quite stable formats like hReview and hAtom are actually still in draft, that’s a very significant step — it takes a lot of research and brainstorming to put together a good draft spec. These subjects have much stronger, stable momentum than some previous microformat proposals have had, so I’m confident they’ll move smoothly. Structured publishing of music and food is highly Relevant To My Interests. I worked with publishing some of the hAudio draft in my previous music round up. I think it’s getting there.

4. I’ve spend some off-time brainstorming on a new effort myself; ‘embed’. No dedicated wiki page yet as I’m still compiling the initial data to get it rolling. There’s nearly enough to push it though; a few more sites to grab examples from to get people thinking. It’s deriving some concepts from the oEmbed format Pownce supported, allowing sites to describe their ‘embed codes’ for reuse around the web. I want to be able to reuse linked content in an activity stream, and deriving embeds from mark-up rather than writing drivers for every site on the net. It would make reblogging the embedded content more graceful, too. More robust use cases coming soon.

5. Microformats have issues, feature requests, bug reports, tasks to do. At present we track them on the wiki along with the specification documents themselves. Personally I find it a nightmare. Tracking and triaging issues through versioned documents in various structures is harder and less transparent than I’d like, so fixing it would be nice. The wiki update last year has the facility to hook spec ‘issues’ links up to other systems, and I’m spending some time experimenting. Community feedback needed here, plus considerations to be made regarding self-hosting something like Trac or offloading to an external tool. It could happen quite quickly, since I don’t think there are many sane arguments defending the wiki method; it doesn’t scale.

6. Wiki rewrites. I’m good at writing. I’m too verbose for sure, but I communicate well. I’ve taken great pleasure in applying this to more recent microformats output and I like to think I do a pretty good job of improving the experience of interacting with microformats documentation. Many pages on the wiki aren’t as well written. I don’t mean to criticise other authors, I refer more to the way in which over time important pages like the process and how to play page have been edited and added to so many times that at this point, I fear they’ve become impenetrable to a new visitor, and if they can’t follow the rules and I want to see effort go into reworking those pages to be higher quality documents, more approachable and easier to reference when they need to be enforced.

7. Support transformation efforts. In 2008, I’ve noted a couple of repeat proposals and desires for using microformat specifications in other contexts than HTML. Being in HTML is part of what makes something a microformat, so we’ve had instances of proposed forking. Versions of hAudio exist republished for use in RDFa, there’s an entire page on the microformats wiki called jCard — putting hCard into JSON for interchange. Per-specification duplication is, in my view, wrong. Duplicating specifications leads to fragmentation, confusion, incompatibilities. If people have use cases for transforming a microformat into RDF, or JSON, or anything at all, the core spec needs be the same. What we need documented are consistant rules for transforming HTML into any of those other languages. ‘Transforming microformats into JSON’ could be a single wiki reference page for all current and future microformats, explaining how to convert different microformat patterns into JSON. Not a ‘jCard’ and a ‘jCalendar’ and ‘jAtom’, with an ‘rRecipe’ for RDF and xResume for raw XML. Just one set of rules to handle the transformations that are useful. Within that, defining the parsed object structures of the microformats goes most of the way to serialising into another language, and that’s a job for parser authors to settle on the best way to turn microformats into objects consistently.

All of the above is a reasonable ask, I think. It’s ongoing progress in an evolutionary approach in development of standards and infrastructure. My big wish for the year is perhaps a bigger step.

The next level: API Kits

Consider existing services: Google Contacts, Yahoo! Address Book. Standalone data providers, whose APIs offer high level methods to access the contacts held within.

A popular use case for hCard and XFN is contribution to the distributed social ecosystem. Data about people and social relationships is published all across the web, but consuming it is prohibitively hard for most developers.

Whereas someone developing for the YAB or Google data stores can download wrappers around the high-level methods those APIs offer, consuming microformats remains at the parsing level. There’s no Person::getFriends('http://ben-ward.co.uk')-like method returning an array of vcard objects. If we’re serious about evangelising consumption of hcard in social networks. We need high level, task centic toolkits, not just raw parsers.

A higher level means providing solutions to common problems and use-cases, rather than a solution to ‘microformats’. A ‘Distributed Contacts API’ that follows XFN links between hCards, handles crawling pages and/or interaction with the Google Social Graph API. Ultimately, you make one call to a high level function and it just happens. I want to see microformat-based tools that boom!.

I think XFN and hCard offer the two most appealing toolkits: Distributed user profiles (‘Distributed Profile API’) to the profiles information described with hCard, linked with rel='me' and the aforementioned ‘Distributed Contacts API’ for obtaining the profiles of other people you link to as friends.

I’m thinking that methods like these are needed to make it trivial for social applications to start consuming microformats more ambitiously:

Person::getProfile('http://ben-ward.co.uk', callback)

Get all profile info for the person at ben-ward.co.uk, and fire the provided callback function when completed (you need callbacks for all of this since it’s both asynchronous handle and crawling the web is going to take a little time).

Person::getConnections(
    'http://ben-ward.co.uk',
    [ 'friend', 'acquaintance' ],
    callback );

Return the profiles of all the people connected to the person at ben-ward.co.uk connected with XFN ‘friend’ or ‘acquaintance’ relationships.

Methods like these make it simple for developers to start using the huge wealth of published microformatted data to enhance and power their social applications. Right now, getting to those methods is a lot of labour. We need to build it once, and we need to do it in the open. I would love to be in a position this year that we can evangelise microformat consumption with as much strength as we do microformat publishing. OAuth and OpenID has a lot of evangelic traction because libraries exist to implement it in many languages; ‘You should use OAuth, here’s some code you can use!’ is rather more convincing than ‘You should consume microformats! Err…’.

We can’t legitimately push sites to consume hCard with an effort barrier so high. If a stable API kit exists that a developer can just drop in to their codebase — like the wrappers for OAuth — then we can make a strong case to see the open web realise a little more of its potential. I’ve written about the dream of a distributed, microformatted web before at Digital Web. I want to see if become real, rather than just ‘possible’.

You can see this sort of thing in practice already on a tiny but beautiful scale. If you have an OpenID, and an hCard at that same URL, go sign up on User Voice. You’ll auth using OpenID, and when you bounce back to complete your profile, User Voice already knows your name and email address. That information comes not from attribute exchange through OpenID (which the Yahoo OpenID provider doesn’t support), but through reading the hCard from my URL. I wondered for a moment what was going on. And then I just smiled. It’s the future, now. I want to see that user experience available at low cost to every developer.

So, there’s my forward looking. I see the above as pretty concrete ideas. Of course, there’s far too much to lead myself. So, who knows. I hope that others in the community will feel inspired and that we’ll see this kind of work happen. Just as much, I hope to see the visions of others. This community is diverse. I think I’m one of the most passionate about the actual core of the community (perhaps more so than any particular microformat itself), but there wealth of thoughts and ideas amongst all our membership. If you’re one of those, I invite you to write up your vision for the year.

Microformats are a huge deal. Where do we go next? More formats? Reinforcing what we’ve got? Appealing to new groups of publishers and developers that haven’t heard of us yet?

If there’s enough posts along these lines I’ll link them all together on the microformats.org blog.

OpenID and OAuth are the open, standards based and interoperable editions of this technology, but Yahoo’s deprecated BBAuth and FlickrAuth and others all came before. Also at the tail-end of last year came Facebook Connect, a system whereby websites can piggyback on Facebook profiles for building applications.

For example, take Fire Eagle. It’s a service that stores your location on your behalf, for use by other applications on the web. It uses OAuth to control access to that location; no application can see your location by default. When you visit a site needing your location, it asks Fire Eagle for that information.

Instead providing your Yahoo! username and password to this third party site (which would grant access to your entire Yahoo! account), you are taken to a special page on the Fire Eagle site, click a button to grant specific location permission and then jump back to the original site, which now holds a token to access to your location.

This process means that the site you shared your location with can’t access anything apart from your location (it can’t log into your Yahoo! IM account, for example, or send emails through Yahoo mail). Furthermore, you can log in to Fire Eagle and remove that application any time; you don’t need to change your password to do so.

It’s the future, it’s user empowering, and it’s going to be great. Eventually.

The user experience of this OAuth process — and OpenID alike — has been criticised a bit. Users don’t expect to be moved between different websites, but they are familiar with entering their passwords all over the place. The short ranty version of this article would go like this: If you stop whining and just get on with implementing the OAuth flow, users will get used to it and will be just fine. It’s is usable as-is, so shut up already. But this is the long, constructive version, so:

The user experience of OAuth and OpenID is immature, and can still be massively improved and smoothed out with concerted design effort.

Which brings me to Facebook Connect. Connect is a product as well as a proprietary technology. It’s a packaged and complete offering from Facebook, and as such, comes with a far more complete and polished user experience than the technology-focused, open standards have so far achieved. Polished and mind bogglingly stupid, in places, but, y’know.

Facebook Connect, whilst proprietary and product-specific and therefore irrelevant in the grand scheme of things, has UX that can be applied to OAuth and OpenID flows. If service providers support this, I think user experience gets much better, quickly.

How does Facebook Connect work?

The most common use case for Facebook Connect appears to be commenting on blogs, such as on Gawker sites. Rather than enter your details standalone, or uniquely register with a site, you log into Facebook, and Gawker uses those details instead.

So, you click the shiny ‘Facebook Connect’ button in the comments form, and an overlay appears:

This is the crux of the learning for OAuth. Rather than redirect to Facebook, this granting of permission happens right in the page in an embedded control.

It’s not quite as simple as this, mind. It’s ok that this action occurs in an overlay only because the user is already logged in to Facebook. No exchange of credentials takes place: The overlay is an iframe serving a page from Facebook’s server, so my current login cookie is used and there’s no need for Facebook to ask for my password. A malicious site would gain nothing by spoofing this dialog.

Since writing this article, Facebook have improved the behaviour of Connect. Now, if you are signed in you see an overlay as before, but if not signed in Connect opens a new window, where all usual browser functionality is available. This a huge improvement and fixes the complaints that follow.

Unfortunately, Facebook Connect then screws up. The whole point of delegated auth is that we stop users entering their passwords into third party sites. It has to stop. That means both actually entering their details into third parties, but also interface that gives the impression of giving your password to a third party. When you are not currently logged into Facebook, you instead see this dialog:

Millions of Facebook users, openly encouraged to enter their password into any site that asks. This is wrong. If the user is not already logged into the service, you should be redirected in a more traditional bounce between pages. That way browser-level phishing tools kick in, the URL in the address bar can be manually inspected by the user and, critically, the user is conscious of logging into a different service.

Facebook ranting aside, the first half of their Connect overlay UI would be very useful to enhance the user experience of OAuth and OpenID.

Here’s a hypothetical Fire Eagle app built into Last.FM.

In the current implementation of OAuth, clicking ‘Get Fire Eagle Location’ would redirect you to the Fire Eagle website, and then you’d redirect back again after clicking ‘Confirm’.

Instead, OAuth apps should do this by default:

No redirect, lighter weight UI and more responsive feedback. This, I think, is something that OAuth APIs should support out of box along with their other language wrappers; provide drop-in support.

Now, this behaviour applies for logged in users only. If you’re not logged in to Fire Eagle for any reason, you should still be moved to the separate site as before. We need to stay strict on keeping users spatially aware of where they are entering their passwords, otherwise the whole effort is undermined.

Overlaid OpenID

With one example down, here’s a mock of how Open ID could benefit from the same integrated flow, this time working with Dopplr, since they already support Open ID:

If not logged in to Yahoo, you get a prompt and just as before, are guided to step through the regular, separate-site process to sign in:

Clicking ‘Sign in to Yahoo!’ would take the user to Yahoo’s standalone page.

How to make this happen?

For this to happen, services need to provide support for it; it can’t be done just at the client side. The dialog-sized interfaces for authorising applications or logging into sites need to provided, and they need to support the ‘break out to enter passwords’ flow. But, sites like Fire Eagle already provide a mobile-scale version of the auth page, so further variants are not a major hindrance.

It also needs a JavaScript component to handle the UI side. With a bit of luck, this only needs to be done once and shared between projects.

The core technology behind OAuth and OpenID is pretty robust. Both have major adopters like Yahoo and Google. OpenID has a bit of a bit of a way to go before users need it, perhaps, but regardless, it’s well into the same phase where user experience needs to be a concerted effort, and the status quo needs to be challenged.

Everything in this post is just a small step from what we already have, it’s just smoothing out the edges. Maybe that’s enough, but I suspect there’s a long way to go and a wealth of other ideas out there.

The previous entry on microblogging was the start of a realisation. Realising that Pownce is useful in its capacity as a microblogging platform rather than as an alternative to Twitter, I think that my personal publishing breaks down cleanly into tiers, based on the depth of the content. And a wiki is perhaps the most natural part of that as anything.

Blogging, as in a site like this, is really only well suited to a certain style of publishing. I want to publish content of a consistent style on this blog. I want there to be a certain amount of depth to each entry, and I don’t want some detailed attempt all about the philosophy of personal publishing to be punctuated by a single line piece stating that ‘I could really murder a chip butty right now’.

The way I see this breaking down, and this is starting to feel quite natural, is as follows:

• Pownce is for short things. Thoughts, spontaneous musings, links that you think are remarkable (which contrasts with Delicious, which is a store of links I think are useful in some way). It suits short, sharp content. No room for depth. That’s microblogging.
• This blog contains longer, more considered content. I feel reasonably sure that the things I write here are somehow valuable, either as information or as an expression of my self. They have some depth. Comments are on. I should track responses on other blogs, too. Whilst communicating via separate blogs is a lamentably lost ideal of the original, pre-comments design of blogging, it’s a concept I like. In aiming to write content of substance, I’d want to support it.

The critical thing with a blog though, and something that should be embraced, is time sensitivity. What I write here is timestamped and could, upon further reflection in a month or a year prove to be dismissible rambling bullshit. But the timestamp validates that. The moment you read this you know that it’s old and that gives you the context to consume it. You can write safe in the knowledge that time will let your obsolete content fade away. Timeless, accidental masterpieces will look after themselves.

Which leads to wikis. A wiki will contain detailed content. Thoughts, projects, entire subjects documented through the eyes of an author. Wikis have long been complemented for being very close to the original ideals of the read/write web that Tim Berners-Lee envisioned (back before no-one had bothered to implemented the necessary HTTP verbs to do it). It’s back to a world of writing standalone pages. And in standalone, I mean to imply timeless. So, my ‘about me’ page isn’t a blog entry, it’s a page, and wiki is a superior publishing medium to maintain that kind of content. Similarly, documenting my ‘thoughts on personal publishing’, and my ‘current publishing practices’ is a standalone, timeless (and constantly updated) piece of information. Here I blog about how I publish, or rather, how I’m considering doing it. It’s driven by a desire for discussion. However, to publish my current publishing behaviour, a wiki is a superior platform. That one URL (let’s say, perhaps, http://ben-ward.co.uk/content/Publishing) will always represent current information and is far preferable over regular blog entries every time I change something. ‘Publishing Patterns, August 2008’, ‘Publishing Patterns, November 2008’… a blog is less suitable for versioned content.

So, Twitter is a slight oddball

I regard it as publishing ‘fragments’ of my day. By my reckoning it fits into the tier below (smaller than) ‘microblogging’. But it grew out from encouraging people to just publish their status and into its own social network. So as well as containing the little snippets of my day, it also contains pieces of social interaction. Twitter is great, but it’s a less pure publishing platform.

Combination. The lifestream.

The thing about blogging — an issue that produced some background resistance in me to the personal wiki concept — is that whilst you can better maintain content, you’re unable to push it to people. A blog has a feed and people consume that feed and therefore people read what you have to say. Sound vain? Get over that and accept that in some capacity we all want people to read what we write and we don’t want our output buried somewhere it’ll never be found.

If I were to produce a nicely combined life stream (which I will), Pownce, Twitter and the blog are chronological and so slot in neatly. Twitter gets filtered to avoid publishing those ‘social interaction’ posts, but otherwise fits in. But since wiki content is not time sensitive, it is not the content itself but the edits of that content which should be streamed. That in itself is a bit problematic. New pages are probably noteworthy, major edits are probably noteworthy; minor edits not so much.

The scenario I’m trying to support is this: Rather than someone come to this site and subscribe to just the blog feed, they would subscribe to the whole lifestream. But, the lifestream would be built such that the content is relevant enough they don’t get irritated by its content. Not an easy balance. Configuration seems like a grossly over-complex solution, but perhaps offering two predefined options would be manageable; substantial content containing blog entries, major wiki edits, and longer Pownce posts could be available separately from the whole life stream.

I suppose I should build it.

The curious thing about this is that really, tabs suck. They always existed as a simple hack around the operating system’s (Windows) inability to handle many windows together. The taskbar got full too easily, and when 75% of the items were browser windows, it all became unmanageable. As a result, tabs went into every browser on every platform, effectively providing a second, browser-context-specific taskbar.

The problem now (and likely then as well) is that the idea of one single ‘browser context’ is bogus. Browsers are now used for such a variety of tasks and applications that it makes less sense to keep, for example, Gmail and Google Calendar in the same context as a set of blogs you might be reading. Your cycle of looking at those pages is different. Mail more regularly than Calendar, and the blogs might just be a reading list to refer to later. Whilst browsers were very quick to add tabs as a feature, non of them have worked them into the idea of working contexts. New items always open in your last used window. Even if you manually break Gmail out into its own window, the moment you open a link you’re putting a reading list of pages into your email context. Tabs are implemented in a physical, window based manner, rather than in a workflow based manner.

On Mac OSX specifically, there’s an addition problem (actually, physical). The otherwise quite-useful Exposé function doesn’t work with tabs in any application. So whilst in Pages, Fireworks, Preview and so on I could hit F10 and see all my documents together, the tabbed browser hides all the content away behind tabs.

Which is a roundabout way of getting to a point. I’ve turned off tabbed browsing. Switched back to the old way of having each document or application in a separate window. Switching between then with Cmd+` rather than Cmd+Shift+], gaining the ability to see them visually with a swift tap of F10, and losing the recurring bug of thinking I’m finished with all the documents in one manually created context window, only to find the music stops when I accidentally close Last.FM.

To defy my muscle memory for hitting Cmd+T, I’ve used Mac OSX’s excellent Keyboard preferences to override ‘New Window’ to Cmd+T, and New Tab to Cmd+Option+T.

It’s just an experiment. Sometimes you do want tabs to keep things under control, for example working through a feed reader, opening links for reference later creates a single ‘reading list’ task, and you wouldn’t want dozens of those individual pages cluttering up the rest of the desktop. There will always be exceptions. But since the browser software has failed to handle work contexts properly, I think reversing the default behaviour is the way to go.

Initial reactions are that this is easier to manage, results in less accidents and no accidents in losing windows (Safari has a ‘Reopen last closed window’ option, but is less graceful with tabs).

Note, this problem with tabs is just a result of needing to break web applications and documents out of the browser context. Efforts like Fluid (a WebKit based browser that creates standalone executables for specific websites) also help break out these contexts for apps you use regularly, but is less suitable for infrequent or new apps. Also, this is not to say that all tabs are broken. Tabs in IM apps like Adium still work me, because the amount of content open at one time is fairly small, and managing two work contexts (‘Work Conversations’ and ‘Personal Conversations’) within a single window interface is trivial. The web browser falls down because the amount of content and the number of contexts now exceeds what I can manage.

We could build better browsers, but it strikes me that the better first action is to step back and stop bypassing the capabilities of the host operating system.

There’s so much that needs to be said about moving, but for now I’m thinking a bit about Fire Eagle itself, the gorgeous app I’m going to be working with for the foreseeable future. I’ve run sessions recently at Mashed 08 and then last week at Mobile Monday running through what we’re trying to achieve, and it’s quickly put me in a position of thinking about the field of location more generally.

Location is big. The killer apps aren’t there yet, but looking at early reaction to iPhone apps, quick criticism comes to those which don’t use the CoreLocation API where it could be advantageous. There appears to be a gigantic side effect though: rash, unintended publication of location. Right at the heart of Fire Eagle, and the feature that I feel most passionately about, is control over disclosure. Reducing the level of detail exposed about your location, on a per-application basis. You can disclose your location based on how much you trust the application or website asking to use it, you can disclose to a level suitable to what you’ll find useful. Fundamentally, we want people to be empowered and to realise that they do not have to tell anyone who asks precisely where they are to benefit from location-based applications.

Today, I saw this on Twitter:

I’ve obscured the user’s details, but there are a number of key things I want to highlight, without going into much more analysis.

• This user chooses to keep their content protected
• This user is using Twitterrific for iPhone.
• The location is added by this app is not abstracted at all, it’s just a raw, precise co-ordinate.
• The location is disclosed to everyone visiting the page; not just the user’s friends. There’s no trust mechanism.

The iPhone only offers a ‘yes’ or ‘no’ choice on whether to disclose your location to an app. There’s no local API to fuzzy that location when you work with it. Through the limited technology of the iPhone platform and the lack of thought on the part of a developer, that location has been made public in a way that it should not be.

Your location is some of the most sensitive information about you. I very much believe it to be a more complex piece of information than an ‘OK/Cancel’ prompt can handle effectively. I also fear that lazy attitudes toward location now, in these early days, could have a catastrophic effect on user perception of the entire location-application field.

Raising this issue is not really about Fire Eagle, we just happen to have approached the problem very thoroughly. It doesn’t matter whether you build an app using Fire Eagle’s API or not. The crux of this matter is that you absolutely must encourage your user to trust you, else you will lose them.

It’s an overview of distributed social networking in a microformats context, explaining how XFN and hCard gives us most of what we need to start exposing users to the benefits of having information made available between different social sites.

As well as explaining how to go about publishing these things and explaining what the first iteration of consumption could be, I’ve also tried to emphasise how important it is to guide users into this interconnected world gently, since many won’t appreciate that their information is quite so public.

It’s a little lengthy, but I hope it gets people thinking about the possibilities.

It’s a lengthy piece on the recent redevelopment of Kelkoo and the microformats we added to it. It’s focused on the consumption of the formats though; putting them to use to build a related products widget for Kelkoo. That was my Hack Day V creation (up on hack.ben-ward). I hope it provides a useful introduction to hListing, and the practical benefits of microformats in action.

Feedback so far has been excellent, so thank you to all who’ve linked it with kind words.