@lindsay.at

DeWitt Clinton's screed on URL shorteners, especially directed at Twitter's usage thereof, is interesting, not only for the actual content (which is broadly true, and fairly sensible), but for the meta-message: Google is increasingly threatened by Twitter as the prime mover in the "Real-time web."

To me, this feels a bit like fear, uncertainty, and doubt spread about a competitor, attacking the competitor's actions while distracting us about Google's own actions. Most telling was this sequence about precedent:

As a thought experiment, imagine that your email provider suddenly started rewriting all of the URLs in your outgoing emails so they could track every link the recipients click on.

…

But since Twitter is the most popular, and arguably the most influential, of the new wave of micro-blogging systems, I sincerely worry that this is going to establish a precedent that everyone else will feel compelled to follow, since it is clearly an advantage to the network if they can get away with capturing this data. I ask, why wouldn't WordPress or Facebook or Tumblr do the same if they could?

We're supposed to be outraged by the privacy implications here, but the real outrage to Google is that it makes their job harder. Look at Gmail's privacy statement on what they do about you clicking links on email you receive:

When you use Gmail, Google's servers automatically record certain information about your use of Gmail. Similar to other web services, Google records information such as account activity (including storage usage, number of log-ins), data displayed or clicked on (including UI elements, ads, links); and other log information (including browser type, IP-address, date and time of access, cookie ID, and referrer URL).

Yes, Google's asserted rights are over your own use and not with others' use of the links you send. Ask an average user on the distinction, and I think they'd say it's different but not categorically so.

The other part that interested me is the "Safety and Transparency" part for Twitter's links. A major part of Twitter's justification for wrapping every URL (which I'm still personally dubious about) is to protect people from malicious links. Well, that sounds suspiciously like the role Google's stopbadware.org interstitial warning page plays, especially when Twitter doesn't have direct control of how the status message is displayed (it may be via a third-party application or SMS). Is this an argument against URL rewriting, or an argument against anyone else acting as a trustworthy intermediary?

I think Twitter's revelations on its monetisation and platform strategy earlier this year have Google genuinely worried that Twitter is turning into a trusted gateway into the web, and so it gives rise to pieces like DeWitt's, where Twitter is attacked for minimal differences in approach to taking on a threatening gatekeeper role. Google's problem, and DeWitt's myopia in offering solutions ("consider using an html payload"), is that Google is fundamentally of the web, and deals with web pages viewed through browsers. Twitter reaches beyond the web, being deeply embedded into mobile devices, and deals with much smaller units of interest than a web page.

Twitter's currently ablaze with talk about the recent change to how @replies are handled on twitter. (Actually, as of this writing, it's ablaze with a shocking number of "RT this if you disagree with Twitter's decision to hide replies to people you don't follow. #fixreplies" messages that cause a massive facepalm.)

While I agree with the change (because for most of my multiple use cases, that's how twitter works best), I feel for the people who didn't use Twitter this way. Their whole way of looking at the social web has been taken away by the elimination of an application preference. That's surely upsetting, and I can't hope to change their minds. But I can try to get them out of their heads and see how the change makes sense for a lot of Twitter scenarios.

I operate a bot, @recomme (which is currently in the shop for maintenance due to another API change that Twitter made with no warning — so my above empathy is real) that feeds on @ replies: send recomme a message, and it tweets back to you, also in the public stream. If you keep your tweet stream private, you must follow @recomme, and it must (auto-)follow you back. The same follow requirements happen if you want to send private messages and receive them back.

This seemed like a reasonable model when I designed it because to me, the sensible way of managing replies was to see only replies to those people you follow and have asserted that you found interesting. @recomme has nearly 4000 followers now. For those people who have kept their settings on receiving replies to people they're not following, that's a lot of eyeballs to reach, especially when anybody can trigger a message from the bot. Fortunately, this has not been noticeably abused by any of the thousands of users. Unfortunately, because of the existence of the all-replies setting, the bot receives nowhere near the number of tweets you would expect from the number of users, I suspect because the social cost of sending tweets to and from the bot are big.

Now, if Twitter stays with hiding replies to people you don't follow, then my vision of an "emergent social network" can happen. People can happily tweet the bot, knowing that people who don't care about music recommendations won't see those tweets or those replies. The people who do care about you and music recommendations do see those tweets, and I think that's a nifty way of opening conversations.

For my own tweets, I struggle with keeping the number of people I follow down, and I have a tough time refusing to read tweets. The "@ replies to people I'm following" setting was an effective filter. Actually, there's a core of twitter users who I know in real life, and I am more keen on seeing all their tweets. (And did so with a secondary account that saw all replies.)

But the real reason why I'm in favor of the recent change is because of new users. Techcrunch actually showed some insight after their typically incendiary headline:

Before tonight I never paid much attention to this train of thought - after all, on Twitter, I can just follow the people I care about and ignore those I don’t. But it’s clear that Twitter is concerned with appealing to a more mainstream audience, and if that takes making a very simple service even more simple, then by golly, that’s what they’re going to do.

Well, yes. Exactly. My pride as an early adopter is far outweighed by the desire to have more close friends and family come to Twitter.

What does a new user see after adding a few friends and "recommended users"? Some tweets, but a lot of decontextualized half-conversations. This is confusing and off-putting. I feel guilty some days when I use Twitter as an open IM/IRC channel, having long threads of conversation. For those who follow all of my tweets, I must seem incredibly boring, geeky, trivial and somewhat profane. That's accurate, but it's not what I like to remind people.

I think that seeing this chatty, focused, decontextualized side of Twitter is not a very gentle introduction for newcomers. It might be useful for comfortable users who have been around for a while but follow dozens of people (or for very prolific users who skim or otherwise filter their tweet stream but don't pay particular attention to any users in particular). But for introducing users to Twitter while fighting concerns that it is "trivial," it's a pretty important step to take.

Some ideas that were thrown at me were improving the experience:

Carr0t

I would prefer it on a per-user basis. So I could say ignore @replies from @stephenfry, but get them from @daagaak.

daagaak

It would be nice for API users, bots, etc to be able to specify if a tweet should act like that. I just dislike the list of discovery.

...in other words, make the control more fine-grained, whether it be push or pull. Well, changing the granularity would be welcome from my selfish point of view. I could zoom in on some users, and not on others. I can see the appeal of the push-control, too. I would have welcomed making @recomme more private, if that had been an option.

But the truth is, that's all too fiddly. It makes a degree of sense to a "power user" like me, but from a user standpoint, it's a disaster. Too many things to control. The point of the change was to simplify a hard to understand option, something that I stand behind on principle.

On the other hand, Anne said, "Never take options away from a user." Those are also wise words. My gut instinct to make as many people happy as possible would be to grandfather in the feature: keep the option for those who have changed from the default, make it disappear for all others. This creates two tiers of users, though, which is untenable in the long term, especially from the social aspect: these exceptional users have a view on Twitter that is fundamentally different from others, and are likely to use it differently. As such, I think the feature should be phased out. Leave it to clients to support an expanded view of the greater twitterverse.

There's a Twitter truism that's been floating around: "Anyone who tells you how Twitter should be used is wrong." I've tried not to fall afoul of that here, but it's a fine line.

I am a patient person, so it's only now that Twitter's perennial scaling problems are bothering me me enough to blog about them. That probably makes me the last person to do so, ever.

However, lately, it's hurt. It hurt most when I tried to implement a twitter bot at Mashed08. With API calls throttled down to 20 per hour, the best I could hope to do (via polling, and with IM shut out, that was the only obvious path) was to be a bot for one person making no more than one request every 10 minutes. So for the demo, the twitter connection was really baling wire and duct tape (or, ipython console and cut-and-paste into twitter's web form).

Last month, I read Tim Bray's Twitterbucks entry with interest. When I last checked in, nobody seemed to be interested in where the real scaling problems were, so the comment thread didn't come up with any real revelations.

Today, as I tried to reflect on why I use twitter, I came upon another potential solution: pay for what is the hardest to scale: disk access. When any high-volume application has to hit the spindles, it takes a massive performance hit. Twitter's recent outages seem to address that at least partially: paging backwards in your personal + friends timeline is scaled back, as are examining replies.

Seems to me that much of what Twitter covers well is the "now" and recent past. Going back in time on a merged timeline makes for increasingly expensive queries, and reaching further back in history goes beyond the memory caches. If Twitter didn't try to keep all its posts accessible, it could be a much more efficient messaging platform, always living in an amnesiac present. By having a web-accessible memory, with persistent tweets, it becomes a lot more difficult to predict where the database is going to be hit.

So take a look at Twitter as it stands right now. With the buttons that are disabled, which ones are the biggest pains? Single-user pagination? Friend pagination? Replies? Seems to me that the biggest omission is in having zero reply-page functionality, but the complex query (user > friends > friends' updates that can be seen > merged and sorted in time) database hit makes sense to limit. Why not cull functionality for all users such that it's either a complex query that hits memcached exclusively (the pages representing what's happening now and in the recent past), or a very trivial query that is allowed to hit the disks (a single, permalinked tweet or any user's front page of recent tweets). A twitter caught in the present, and exhibiting some memory when specifically prodded.

From there, you could charge for more archival access. I imagine this not as a monetisation move, or even one that could directly cover additional costs, but one that would allow serious users with serious needs self-select, not unlike what Flickr has done with their paid accounts. A paid user could access their archives as a continual stream of tweets on a blog-like page. They could access a more comprehensive memory of their friends' replies. They might even be given persistent past per-day or per-month archive pages.

I'll admit that I don't fully appreciate the particular scaling problems presented by heavy users like Scoble. Perhaps there are payment thresholds to pass once you follow 500 and 5000 users?

What do people think? I know I can't be the first to suggest it, but it's the first I've heard of it.

ROFLCon felt like a life-changing event. I don't know if the effects will really be that long-lasting, but it feels good for now. What really affected me was the massive outpouring of positive energy from nearly everyone: the enthusiasm of the organizers was infectious, and the attendees were touched with a sense of childlike wonder upon learning that the conference was really real. At first I was concerned about the dynamics of 600 introverts in the same room as 100 extroverts (largely from the marketing contingent), but it worked. People found their respective crews, and there were enough good will and shared reference points that misfits like me could wander from group to group with no resistance and lots of good will.

I was blown away by all the kind words said and blogged about the LOL panel. I actually couldn't explain the effusive praise from such worthies as David Weinberger. Alexis Ohanian, our moderator, certainly kept things moving and varied, and the panel seemed pretty well-balanced in their contributions. What really helped the panel, however, was the overwhelming goodwill from the audience. LOLCats were simply a fun, silly thing to talk about, because kittens make everything better.

This is relevant to my interests

What surprised me most about ROFLCon is how much I was supposed to be there. I thought it was right to attend completely separate from my professional duties at the university: LOLCODE has always been a separate thing, a lark ostensibly done in my spare time. For so long I've said my personal shtick was about tools for creativity, and the day job was more about multimedia content distribution, shading towards metadata, user-generated content, and vernacular creativity. I can be myopic sometimes. The biggest facepalm moment was learning that new pal Kenyatta was 'yatta' from the unmediated blog, from which I nabbed loads of articles for an old work blog. Of course, I didn't learn this until an hour after he left.

I loved how nearly every interaction could be memorable. I think the energy behind this was down to how young the participants were: starry-eyed youths were still in their teens, and the majority of the panelists who had done something tended to their twenties and thirties, with only a few incursions northward. This was in sharp contrast with the professional and academic conferences I'm used to: the young ones are comfortably into their early-to-mid twenties, and the accomplished ones are far older. And yet it seems I'm still most impressed with the youngest set at those same conferences.

I spoke with a fair few young people there. By the end of Saturday afternoon, I was pretty tired, emotional, and perhaps a bit full of myself. I ran into a young man named Dixon, describing himself as a Smosh fan. He "so wanted" to come to a school like MIT, and I told him of the relief I felt at MIT and other similar-tier universities recommitting to need-based financial aid. Once again, if you show up as a desirable student, they will make it possible to attend. And if you're there, there's no problem with who you are, regardless of color of skin: the MIT I know is one of the ultimate meritocracies. He seemed a bit relieved and... hopeful. I have hope for him, too.

MOAR

ROFLCon itself might be best termed a symposium: it was neither a conference in a traditional academic sense, nor was it like a convention, with the usual commercial bent (I wanted to get schwag from a number of creators, but there was no way of grabbing it.) Rather, it was a gathering of practitioners (meme creators), theorists (like the keynotes and Josh Green's session), and some engaged commercial interests (e.g., marketers) about the economy of ideas and internet micro-celebrity. It was an alchemical combination, not least because the practitioners were humble, introspective, and tended to be thoughtful about what led to their being there.

I would love to see another ROFLCon happen. I will do what I can to help make it so. I think there's more room for an academic track, not only in the social science of memes and microcelebrity, but on more technical subjects on the dissemination of ideas and traffic analysis, and even on practical technical matters like scaling to handle such traffic.

Can I imagine ROFLCon being a yearly thing? An institution? I'm a bit more dubious about that. What made ROFLCon so wonderful was that sense of childlike wonder from everyone, a hovering sense of disbelief just over everyone's shoulder that it actually happened. The positive vibes that grew from that disbelief permeated all the ecstatic interactions described above. Once the event gets taken for granted, I think it will be time to move on.

And if/when negativity or jadedness invades ROFLCon, it won't be pretty. This year, anonymous was in full force during the final two sessions. Props to them and their session about Scientology: I really think it's a brilliant social hack they hit upon in targeting the CoS. But when they're a masked mob causing havoc with no answerability... not so much. Not clever and not funny, and it really encroached on the good will that permeated the rest of ROFLCon. If ROFLCon can stay on the sunny side of the street, it has a bright future.

I wish Brian and Tristan and the whole echonest crew the best of success. Beyond dropping the Analysis API on the public last autumn, they've finally come out to the public with a real corporate website and a hint of what they do.

The real treat is their this is my jam web application. It's audio mosaicing made popular and fun. Best of all, it's let me realise something I've dreamt of doing for years upon years:

Bill Withers' Ain't No Sunshine has been my personal earworm for about 30 years, back to when I first heard it on AM radio. I rediscovered it late in college, early grad school, and then around 2001, I discovered just how many covers were out there, via a long-forgotten file-sharing protocol. It's immensely satisfying to have a terse mix of fifteen versions of the song with a couple minutes' effort.

It would be nice to be able to return to the mix and tweak it a little, without creating another one, but then there's a rough perfection to it already. It's also worth noting that the RSS created on each page is also an iTunes-enabled podcast, if you want a way to get the MP3s onto your computer. My feed has already been duly added to my everywhere feed.

If anyone out there cares, I've finally gotten off the $100/year hamster-wheel called .mac, and transferred the existing major pages living there into this site's CMS. Most notable are the Paradiddle and ConTeXt pages, both of which document work done over three years ago. I hope Google is able to index the page transitions, but in general, I seem to be pretty find-able.

Damn Joyent for continually being able to come up with products I want to buy. It's clear that they tap into a psychological type, and I fit that profile. I've written about the freedom offered by Joyent's pricing model for the "lifetime" plans, and all I can say is they've done it again: their virtual private server offering, the OpenSolaris-based "Accelerator," has a limited-customer offer for "lifetime" hosting.

Never mind that Joyent have repeatedly denied that they would (or even could) offer lifetime hosting on this utility-computing-style product. They did it. And halved the price for existing customers like me.

Never mind that while I had been intrigued and tempted by the Accelerators, I set the idea of signing up aside for two reasons: I have numerous containers at my disposal with my OpenSolaris servers at work, anyway, and the extra month's setup fee for an Accelerator didn't make it worth it for me. I bought it.

Absolutely brilliant of them: provide a product that I wouldn't pay $150 to try, but pay $500 more to "buy outright." I'm freed up from the pressure of having to take advantage of it right away. What surprises me most right now is that there is no news of the offer ending, yet. I would have sworn that there would be 200 takers within hours.

Today is the end of a personal era. Today is the day I let my long-held-in-reserve domain dandysalami.com expire. For a long time, I had intended the domain as a web home, and obtained it some six years ago as a quirky personal domain. (It's an anagram, get it?) I never did anything with it, and as my professional profile grew and the web matured, it seemed harder and harder to justify sending anyone to that domain with a straight face, or with that expressing anything about my identity. When I had the brainstorm about this domain last autumn, I went for it, and haven't looked back.

Still, I do feel a pang of regret, and the decision was made especially bittersweet when I received the following spam.

You are looking at a photo, made especially for you so that you can really see how your website or company name comes into its own on your company car.

To watch our [FOO]PLATES VIDEO follow this link: http://www.youtube.com/watch?v=NyFUsc_d_iw.

The product you can see is called [FOO]PLATES.

[Foo]plates are three dimensional letters, made of the same material as car manufacturers use for their brand name. The letters are chrome-plated so that they are resistant to any type of weather.
A standard [Foo]plate consists of 20 characters. A [foo]plate with 20 characters is approximately 40cms (15,7 inch) long. For more than 20 characters you can request a quotation via sales@[foo]-plates-b2b.co.uk. There are letters of 2.2 cms such as a, o, m, c etc. and of 3.2 cms high such as A, B, C, D, l, p, k etc. Additional characters are _ @ ( ) ? , € - ! The delivery time is between 1 to 3 weeks. [etc.]

I elided the company name because, well, clever though it is, it's still spam, and the last thing I want is to poison my google cred with that stuff.

Anyway, I thought that was a great use of an image processing script for something that was absolutely absurd. (Plus they get props for using Myriad as the font, and for roping YouTube to run their commercial.) Has anyone else run into these folks?

I've been watching Fake Steve Jobs for a while now, and now I'm realising that, true to his growing reputation as the Suck 2.0, writing in character and adopting a persona really can free one up, not only for iconoclastic pronouncements, but for making real insights. I was bowled over when I saw NBC and News Corp., sitting in a tree:

What we're doing today with things like Apple TV or TiVo is just applying Band-Aids to patch up a frigtarded system (linear TV programming) that made sense in the 1950s when bandwidth was limited. Our Apple TV only sidesteps the problem. It still forces you to download to your computer, then beam through a router to our TV box and then up into your TV. So great. Now you've got more pipes coming into your TV but this new pipe is kind of unreliable (wifi routers) and slow and clumsy.

The real fix is gonna happen when someone figures out the back end, aggregating good content (ie Seinfeld and I Love Lucy rather than Ask a Ninja) and then finds a way to get that straight into your TV without all these clumsy connections and multiple hops. But it's a battle. The linear model, as stupid as it is, still clings to life. Inertia is a powerful thing. But ultimately we'll win. Give us ten years. And yeah, this is why Apple is presenting itself to the Hollywood studios as a friend and ally, not a competitor. It's also why we didn't buy YouTube.

It's not necessarily the future, or the future as imagined by Apple, but it's definitely a (plausible) future...

There's a bit of cognitive dissonance seeing Jason Hoffman blogging on wordpress.com. Dr. "we admin servers so you don't have to", king of the DIY-ers, is going with a hosted solution. (Okay, sure, TextDrive is responsible for a fair portion of WP.com's infrastructure, but...)

Is that the natural way of things, now? People moving from hosted applications to hosted domains on servers and back to the hosted application space? Will I get sick of the control? Of fighting spam? Of responsibility for security? Hmm.

Yeah, early days. (Is it even really him? Dunno at this point.) But it's an interesting space to watch...