A few weeks ago, Google added a new AutoLink feature to the Google Toolbar 3 beta which automatically recognises certain structures including street addresses, ISBNs, package tracking numbers and automobile VINs; and converts them to useful links for the user. This feature has been met with a great uproar from the blogging community with many arguments for and against it.

Basically, the feature has been called by some ‘as evil as Smart Tags’, while being recognised as ‘useful for the user’ by others. Authors have cried foul over its modification of their documents without consent, yet it is praised by others for giving the user the added information; ridiculed for inserting links to a potential competitor’s site, yet applauded for giving the user some choice in the matter; and finally criticised for the lack of an opt-in or opt-out choice for the author or publisher, while some user’s are screaming back about ‘who’s user agent is it anyway?’

So, in light of all this discussion, I’m going to take an objective look at the debate so far including all the major arguments I’ve found, both for and against, giving the perspective of each affected group: content authors and publishers, users, and Google; and examine the desire for author control by means of an opt-in or opt-out mechanism.

The Problems

Many high profile bloggers have noted their strong objections to this feature, each relating to different aspects of the issue at hand. These problems include document modification and the publisher’s rights to control their content; the insertion of advertisements and the monopolistic effect this may have on competition; and the lack of choice for the content publishers.

Counter arguments for each of these issues primarily revolve around the user’s rights and ability to choose where and when such modifications take place, including the fact that it’s not an automatic process. This raises questions about the user’s ability to discern the origin of the links, how far software producers (including Google) may go to provide such features and whether or not the proverbial line has already been crossed.

Along with the arguments from those that don’t support AutoLink, there have been requests for the ability to either explicitly opt-in or opt-out on a page-by-page and/or site-wide basis, with most claiming that either would be acceptable, yet debating about which is the most appropriate. On the other hand, those that do support AutoLink claim that neither are acceptable and authors that should not interfere with their user agent functions.

In summary, this can be broken down into separate issues that need to be addressed, including:

• Document Modification
• The User’s Choice
• Linking to Sites and the Effect on Competition
• Opt-in and Opt-out

Document Modification

Robert Scoble’s main objections relate primarily to the document modifications, claiming that the integrity of the document is destroyed because the user agent is modifying it (despite it being at the user’s request) to include links to resources that the author did not intend to be there.

I believe that anything that changes the linking behavior of the Web is evil. Anything that changes my content is evil. Particularly anything that messes with the integrity of the link system.

In relation to this issue Scoble also claims that the rights of the publisher and of the user are in conflict with each other, and questions: in the favour of which group should this issue be resolved.

As a user I’d love all sorts of things. But don’t you see that the rights of the end-user are in conflict with the rights of the content producer? Will you ALWAYS settle the argument in favor of the user?

Similarly, other’s have agreed with this and request that the rights of the publisher be met by means of an opt-in or opt-out mechanism, which I will address later.

Counter claims have likened such document modifications to Ad blocker’s that remove advertising content from pages, site scapers that extract or alter the site’s content at the user’s request, the ability to control font sizes, colours and the overall ability to override or disable stylesheets; and other user agent extensions that perform document modification functions all at the request of the user.

Cory Doctorow wrote in Why you should love Google’s toolbar that the user should be able to request a user agent to perform any document modification at all, so long as the user agent is not, in anyway, committing fraud (i.e. The user must know and understand that the modifications are taking place).

I think I should be able to use a proxy that reformats my browsing sessions for viewing on a mobile phone; I think I should be able to use a proxy that finds every ISBN and links it to a comparison-shopping-engine’s best price for that book across ten vendors. I think I should be able to use a proxy that auto-links every proper noun to the corresponding Wikipedia entry.

Similarly, Anil Dash agrees that the document is fodder for processing however the user likes.

In the same way I can rip, mix and burn all the other media I bring into my computer, something as straightforward as an HTML page should be fodder for processing however I want.

In response to this issue, Dave Winer asks:

…what happens when Google isn’t satisfied to add links to our sites, suppose they were to change the actual words? I haven’t heard Google say they would never do that, have you?

That’s an interesting point, but it fails when you consider that such modifications by other user agents already occur. In a follow-up article on Boing Boing, Cory Doctorow and others proceed to list several user agents and tools that perform document modifications at the user’s request on a regular basis. In addition to this list, there are several other’s that should have been included, which I would like to point out.

• The Linkification extension for Firefox automatically converts plain text URLs and e-mail addresses to clickable links, in a way similar to Google’s AutoLink, by inserting <a> elements into the DOM.
• The W3C’s Semantic Extractor scrapes content from a document and outputs all the semantic information it can find.
• XSLT is designed to transform any HTML, XHTML or XML documents into another format.
• AltaVista’s Babel Fish and other translators do change the actual words of a document – that’s its purpose! As a result (because automated translations are not perfect) the intended meaning of the author may be unintentionally corrupted by the tool.

Yoz Graham explains, in response to some other questions from Dave Winer, that such modifications are completely acceptable, as long as certain conditions are met.

If a content-modifying function:

1. has a definition that is completely understood by the user
2. is only invocable at the user’s request and in isolation (i.e. not automatically)
3. has an effect limited to the user who invoked it

then it’s entirely within the spirit of the Web, no matter what modification it performs. No exceptions.

Indeed, I must agree in favour of allowing document modifications and note that those conditions are met by each and every one of those tools and user agents listed above. However the question is: are all of those conditions met in the case of AutoLink? Basically, the answer to that question all comes down to the user’s understanding and choice in the matter.

The User’s Choice

The user’s ability to choose the way in which they access content on the web is central to the way the web works. A user may choose which browser they use, which extensions they install, whether or not they want to see images, the allowed fonts and colours in a document, and many other presentational aspects. A user may choose whether or not they want to access a site and which links they want to follow from a site. They choose whether or not they wish to access or download resources, and which information or services they want from those resources.

AutoLink is just one service offered by the Google toolbar, which the user has the complete freedom of choice to use or not. However, while some choices are completely under the control of the user, others are not and are causing a great deal of controversy. In relation to this issue, Dave Winer states:

The issue for authors and publishers is whether readers know they’re reading text that’s been modified.

There are really two aspects to this question: whether or not the user knows that changes have occurred and whether or not the user is able to determine which changes were made. Google’s Chris DiBona and Marissa Mayer describe how the use of both AutoLink and the toolbar itself are entirely under the control of the user. Chris DiBona as quoted by Robert Scoble in Ahh, evil feature not evil if you force user to download, sign EULA, and click button? states:

We allow the user to never install the toolbar and never use the feature… User choice is the difference.

Chris DiBona states in Oh Please!:

Since the user explicitly wants to do this, the user is choosing to augment the content of a given website.

Danny Sullivan quoting Marissa Mayer in Google Toolbar’s AutoLink & The Need For Opt-Out – What’s Acceptable & What’s Not? states:

"It’s important to recognize that the toolbar is installed by people who want Google-enhanced functionality," Mayer said. "I would argue that the user is adding the link to the page. Google just provides the tool."

Yoz Graham also supports this by stating that the user has both requested the modification and has the right to do.

I say that it does nothing that the user has not specifically asked for. And if the user has asked for it, there is no reason why they should not have it; after all, they could save the HTML to their hard drive and edit it for exactly the same effect. (In fact, the user could do far more wilful damage to HTML than the AutoLink feature does.)

Since the user has full control over the feature and must request it each time, the user is aware that such modifications have taken place, but the question of whether or not the changes made are known to the user is much more valid.

Many have compared the appearance of AutoLink with Smart Tags, noting that Smart Tags used a dotted purple underline. Google’s answer to this problem is by providing a custom Google cursor and a descriptive title text, as Danny Sullivan quoting Marissa Mayer explains:

"The links that we add do look different. We work hard to help the user understand that this was a link added by the Google Toolbar, that it wasn’t a native link. We do this through a mouse rollover that is visible when you mouse over the link."

However, Danny Sullivan questions this by asking whether that is really enough and suggests that using a different underline, colour or other distinction as well, may be an improvement. However, it’s also important to point out that clicking the AutoLink button will sequentially highlight each recognised link, however my tests using the examples provided by Danny Sullivan seem to indicate it will highlight any match regardless of whether the link was added by AutoLink or already present.

I tend to agree that while Google have certainly taken some very good steps to make the user aware of which modifications have occurred, the feature could certainly use some improvement in this regard. Perhaps, as Danny suggested, different colours or underlines may be the way to go, but I think a special Google AutoLink icon inserted beside each link may be a better option, as it may be difficult to select distinct colours and underlines given the wide range of such styles chosen by every author using stylesheets. Thus, if only colour or underline was used, it would be potentially possible that the colours chosen may match or be very close to those already present in the document, as set by the author’s stylesheet. While the chances of that are slim, I think to be on the safe side an icon would be the better choice.

I mentioned earlier that there are choices that are mostly out of the control of the user. These choices relate to which content is automatically linked (ie. The user is restricted to ISBNs, US street addresses, several package tracking numbers for selected couriers and Automobile VINs). The user is also restricted by the choices available for which sites in particular are linked to.

Linking to Sites and the Effect on Competition

The sites that are linked to are, at this stage, chosen by Google. While the user has a limited choice for the map provider (Google Maps, Yahoo! Maps or MapQuest), ISBNs link to Amazon, Automobile VINs to Carfax.com and package tracking numbers link to their respective courier websites.

There is much concern over this issue with many pointing out that Google is essentially inserting advertisements for their and their partners’ services into other sites’ content. The most common example for this issue, first described by Gary Price, is that ISBN links to Amazon are inserted into Barnes & Noble’s site. While, as discussed earlier, the user has the right to modify the content of the document if they so desire, the tool is effectively leading the user to a competitor’s site of Google’s choosing.

Zeldman noted an objection to this, as it related to Microsoft’s earlier attempt with Smart Tags.

They extended Microsoft’s monopoly power into new markets, giving the Redmond giant the power to decide which non-Operating System companies would live and which would die. (Companies Microsoft’s Smart Tags division partnered with would live; their competitors would eat worms.)

If Google only includes features that make use of only their or their partners’ services, it may indeed unfairly harm the competition, and (in the worst case scenario) create a monopoly for those organisations. Danny Sullivan explained the Monopoly & Monetary fears of this issue; however, he also expressed the intention for Google to include more options (including Barnes & Noble) in the near future.

Don’t like that choice? When the tool emerges from beta in the near future, it is definitely planned for people to choose some of the content providers they want to tap into. If you want links to Barnes & Noble for ISBNs rather than Amazon, you’ll almost certainly be able to do that or pick from others.

If true, that will certainly be a huge improvement. Personally I feel that there should be some way for any online book store, map service, courier or whatever to include their services, perhaps by means of some kind of additional plugin or setting.

For example, I would like whereis.com.au as an option for my map service and a local book store like Angus & Robertson or Dymocks, and I’m sure others would like localised services too. However, the feasibility for Google to include each and every online book store, map service, courier and vehicle information service by default may not be feasible, which is why such providers and their users should be able to further configure the toolbar for their needs.

Opt-in and Opt-out

A user agent is a tool used by a user to access and perform certain tasks with resources. As such, a user agent should perform these tasks in a way that benefits the user, provide choices to the user where appropriate and provide feedback to the user about the tasks performed. Basically, the user agent must act on behalf of the user at all times.

However, a popular request from those against AutoLink is the ability for the publisher to temporarily disable the feature while users are viewing their site, with several suggested variations in the implementation details. Dave Winer’s preference is clearly for opt-in:

…we really want AutoLink to be opt-in… I might enable it to learn how it works to decide if I want to use it for other pages and other sites…

However, as Danny Sullivan correctly points out, opt-in is taking it too far, and indeed harmful to the usability of the feature.

I think that’s too far. Users do have rights. They have installed this software. Opt-out gives any publisher seriously concerned with the tool the ability to control it on their site. Many won’t be concerned, so requiring an opt-in is overkill that does hurt the user experience.

Yoz Graham agrees stating such features would never work if they required an opt-in.

Content creators should not have to provide specific opt-in permission; if they had to do this for every such feature out there, most of them would never work.

Others tend to agree also, yet still request the slightly less harmful opt-out feature through various methods such as: robots.txt extensions suggested by Dave Winer; or a meta tag similar to that provided by Microsoft for Smart Tags, as mentioned by Jeffrey Zeldman in response to the performance limitations of the various script solutions.

Client-side wear and tear could go away like a bad dream if Google would do what Microsoft did with Smart Tags: namely, provide a meta tag that disables them.

Drew McLellan (via Zeldman) explains how the script solutions, initially created by Chris Ridings and modified by others, work by cycling through the links in the page and removing those that are found to have been inserted by the Google toolbar.

Both the opt-in and opt-out requests have been met with opposition from Google’s Marissa Mayer, as quoted by Danny Sullivan:

"If you had opt-in or opt-out, that’s overall a lot less useful," Mayer said. "If the links sometimes won’t show because there’s a publisher opting-out, that’s bad for the user experience."

It is also noted that the toolbar will not modify any existing links, thus being a form of opt out – an option which has been taken up by Barnes & Noble by making all ISBNs on their site links to their own resources. However Danny replies by stating that doing so for every link just to block AutoLink is just not an effective nor feasible solution for everyone. However, in such cases, I question the publisher’s right to prevent the user obtaining more information where the publisher had failed to do so.

In response to the script solutions, Phil Ringnalda has reminded us of exactly who’s user agent it is and provided a work around to bypass Chris Riding’s original checklinks() function. However, this solution will fail (without modification) on the site of any publisher that alters the name of the function; and as Danny Sullivan points out, this is getting absurd.

…it highlights how quickly things have become absurd. You have third-parties working to prevent AutoLink and potentially others working to prevent preventing AutoLink. It’s a mess.

In my opinion, the valid issues against AutoLink seem to relate to the monopolistic effect of the links; yet the requested opt-in or -out solution does not seem to address this problem directly. Rather it gives authors control over how a user agent processes their document in order to completely prevent document modifications; particularly by implementation specific user agent features. Publishers that take such action to interfere with user agent behaviour can be seen very user-hostile; and user agents that obey such directives are acting on behalf of the publisher rather than the user.

Also, in order to opt-in or -out of such user agent features, it requires the use of proprietary extensions; regardless of whether they validate or not. This was, and is, true of the nofollow relationship, and will always be true for AutoLink and other similar features.

For example, Microsoft have introduced various proprietary extensions in order to disable user agent features including the Smart Tags meta tag, the image toolbar meta tag and attribute and other (though more widely implemented) features like autocomplete. Some of those validate and others don’t, but that doesn’t change the fact that they are proprietary extensions intended for use by a single user agent to perform a very user hostile act – namely, to disable a user agent feature against the user’s request. Thus any opt-in or opt-out functionality introduced by Google would be quite harmful, just as the script solutions have already proven to be.

As discussed, the document modifications performed by AutoLink are done so at the user’s request, and the publisher has no right to interfere. While some publishers may feel that the user is violating their rights, the fact is that once a user has received your document, they have the right to perform whatever functions they like with it, so long as the results are limited to the user that performed them.

While Google, in an effort to provide a tool for performing useful functions for the user, have certainly shown some desire to put the feature under the user’s control by allowing them to decide where and when such functions are performed and attempted to provide some (although limited) indication of the results, Google’s role in the process is not impartial. It is Google that chooses the possible link destinations with little ability for user alterations and as such, has received much criticism over the potential effects on competition.

It is for these reasons that publishers and authors are requesting the ability for their sites to be exempt from this functionality; however, any attempt to override an implementation specific user agent feature is a user hostile act which only serves to interfere with the user’s choices provided by the tool.

In my opinion, AutoLink is nothing more than a very advanced favelet (or bookmarklet) that may prove very useful to many end users in the long run; however much more control needs to be given to the user, particularly in regard to the service providers chosen for the link destinations. Beyond that, I only hope that someone will port this functionality as an extension to Firefox and content publishers and authors cease their attempts to control a user’s system.