@lindsay.at

So, after weeks of dithering, talking with my colleagues and hearing their enthusiasm for the event provided me enough impetus to attend BBC's Mashed. It's HackDay 2008, basically, and it really felt like I missed something special last year. I've been getting my hack-fu together, and it felt like it would be nifty to go as a team.

But no, said colleagues dropped out almost immediately.

Mother-in-law came to the rescue, though, in offering to take Helena for the weekend, un-prompted. Great! (What? She's now expressing mixed feelings?…) So, I'm committed to going, even if I'm not going to approach it as the young man's game it appears to be. I have a hotel, I have a train, and I will attempt to treat myself right in the process. I want to hack, but I'll be fairly happy soaking up the atmosphere and giving out most of my remaining LOLCODE stickers, if people are interested.

For the rest: we shall see what we shall see.

After seeing the appropriate portion of the Macworld Expo keynote presentation by Steve Jobs, I have some thoughts about the revamped AppleTV.

First off, I'm pretty happy about it. I'm very pleased that the "version 2" is just a software upgrade. It was pretty clear that the original hardware was over-specced for what it did at the beginning, so I bought early with a lot of confidence that the hardware would last. (Subsequent teardown reports that suggested tiny margins on the hardware increased that confidence.)

It's interesting to note that the UK prices haven't changed. At £199 and £269, Apple is treating the UK AppleTV as a niche product. If you know you want it, you've probably already gotten it. As soon as movies are available internationally, then I would expect the price to go down a bit. You gotta have the blades ready to go, if you want to sell cheaper razors...

Looking at the demos available online, it's unclear where the "source" menu has gone. We have a 40GB unit, and when Rosemary or I are feeling in an aimless mood, we'll mount our 500GB iTunes library and browse that for inspiration. I worry somewhat that streaming from arbitrary sources might be compromised in the new software.

The store integration is very impressive, and very inviting. It's what's needed to make rentals work: highly visual, presenting a multitude of choices, and accommodating to impulse buys (or rentals). It looks like a model that all others should follow.

However, the AppleTV now appears to be little more than a portal for the iTunes Store. The menu system puts an extraordinary amount of attention on the Store, and pushes one's own content to the bottom. It seems odd that streaming content over the internet is given such priority after the first version: Apple's view always seemed to be, "Don't trust the internet's quality of service, but you can stream over the LAN." LAN-based content, as far as I can tell, seems to be hidden.

I always viewed Podcasts as a back door for (free) content onto the AppleTV. They are the "other easy way" to get content pushed (automatically) into the living room. Before, entering arbitrary URLs into iTunes (perhaps via a clickable itpc: link) was about equivalent to subscription via Apple's iTunes Podcast directory. Now, via the AppleTV, the iTunes store solidifies its position as an orifice to podcasts. It looks inviting, instantly gratifying, and well done, but it makes Apple more of a gatekeeper to free content.

The fact that high definition finally makes its appearance is exciting to me. It's not how I originally imagined it would be, but everything I've read suggests that SVC scalable video encoding isn't ready for prime time yet. It may make an appearance, once hi-def moves to a purchase model/off the AppleTV exclusively.

There has been some speculation on why hi-def is AppleTV only. Some think it might be due to piracy concerns by the studios, but I think there are technology reasons as well. For 5.1 surround sound, there is no reliable, universal way for Mac or PC users to enjoy content encoded that way. High-Definition video cannot be played out on any of the iPhone/iPod family, either, so simply placing that content into iTunes creates a confusing situation ("this content is not compatible with your iPod") for those much-beloved users. In other words, hi-def is AppleTV-only because the technology isn't ready to accommodate the other devices in the ecosystem. I've outlined the ways it could happen, eventually, but for now, a closed, black box solution is sufficient for content that will not have a lifespan beyond thirty days.

I'm really eager to see the updated software myself, but I do worry that Apple may have made itself too much of a gatekeeper to content in the rush to give people the movie rentals they wanted.

In the course of hanging out on the ZFS discussion list, I've developed a healthy paranoia for the safety of my data. Hard disk drives fail all too easily nowadays, and I know it's a matter of time before another major data-loss event, especially since my family's large media collections are being held on single external hard drives prized more for their attractive cases than for their reliability.

I found out a had a few hundred pounds left on my consulting account code at the university, and it's ideal for buying computer kit. I ran it past my Executive Producer (read: 'wife'), and she's also ready for redundant network-attached storage, having also looked into it for her job. I spent several days trying to figure out if I could build a solution myself based on OpenSolaris and ZFS, but ended up thwarted by form factor: I could find no cases that were compact as well as capacious enough for (at least) four hard drives.

So, I looked around, and it sounded like the Infrant ReadyNAS NV Plus is the unit to beat. As a UK supplier has a special on disks bundled with the unit, I put in an order right away.

But I wonder, what do I really want? I want flexibility and ease of administration. I want verifiable redundancy. I want ZFS. I also want something that will work seamlessly with my home network. I want something that understands Apple's protocols well. I want to be able to stop running iTunes at some point during the day, but still keep synching with my AppleTV. I want an integrated backup solution that works well with Apple's upcoming Time Machine feature in Leopard. I want something only Apple could provide.

The net is strewn with "Apple needs to provide this form factor or they won't get my custom" articles. I know that Apple's strength is in providing a simple, comprehensible product range that potentially serves a vast proportion of the populace. Still I wonder if an "iServe" could be coming. Consider:

• The AppleTV, iPhone, and the Airport Extreme (presumably running Darwin) are showing the way that Apple can expand their offerings while cashing in on the intellectual capital they've built up over the years of fine-tuning the full stack of Mac OS X. Add a little more sophistication with regards to storage to the Airport Extreme, and you have the basis of the device I'm envisaging.
• Apple has been building expertise and goodwill in the storage arena for years with the XServe RAID. It gets a fair amount of respect for that product from certain quarters. It won't be the first time Apple has tested a technology at the professional high end as a dress rehearsal for something in the consumer space.
• Apple has been very open about its efforts to port ZFS to Mac OS X/Darwin, and many have noted how ideal Time Machine would be as a GUI for ZFS's snapshot feature.
• The backup qualities of Time Machine only go as far as the reliability of the backup media, and external hard drives have had a sinking reputation for years. Building a reliable RAID NAS on ZFS would be a solution to a problem that people don't realise they have. You can now have faith in your backups, and serve all your home's or small workgroup's computers at once.
• Time Machine, as far as I understand it, seems to be rather storage-hungry (with snapshots, saving a copy of every version of every file that has existed on your computer from the day you install Leopard), and with the vast majority of Macs shipping with a single hard drive, that's a bit incongruous. Why should Apple sell you a software solution without providing the hardware or service to go along with it? This could be a sign for Apple to sell you something more.
• With iTunes moving into an increasing role as a media hub application, it makes bigger and bigger demands of the machine it runs on. With an AppleTV in the house, the ideal (as I use it) really seems to be 24-7 operation of the host computer, with iTunes always running. My Mac mini G4 has always been a 24-hour computer, but other people would like to send their iMacs to sleep once in a while. Seeing as iTunes serves multiple roles (media jukebox, media organizer, store front end, and a synchronizing/streaming server) at once, there's room for splitting off one of these functionalities into a lightweight server process that integrates with the iPod ecosystem more cleanly than the third-party daapd offered with RAID NAS products.
• There are NAS and RAIDed NAS products on the market already, and although they're gaining popularity, there are no dominant players. That's an opportunity for Apple to capture mind share with a new product. It's interesting that Netgear just bought Infrant, but Netgear still doesn't seem to be able to capture the public's imagination as Apple does.

RoughlyDrafted had a nice analysis of Apple's current NAS storage capabilities when it compared the new Airport Extreme with Windows Home Server. I think the Airport-as-NAS is just the thin end of the wedge for Apple, so here's what I propose:

• A 4- or 5-disk RAIDZ-capable array of disks
• AFP and SMB file sharing
• Printer sharing
• The server portions of iTunes
• Low power consumption
• Small form factor
• Simple administration through something resembling the Airport Assistant
• Possibly a gigabit ethernet interface (but certainly at least 100Mbit), for workgroups or direct connections
• Possibly USB and Firewire for direct connections

...and that's it. It could be a wireless (802.11 draft N) device, and could have router and wireless access point capabilities, but those would dilute the message on what this device is for: to serve and protect your files, reliably, quietly, and cheaply.

Everyone wonders when, with Apple. This is mere speculation: it may never happen. When could it happen, then? I would expect this sort of product to come in the wake of next year's Macworld Expo in January 2008, at the very earliest. I see the iServe as dependent on Leopard shipping, to help build the demand for the feature through Time Machine. It wants a major announcement to give it the time in the spotlight for the general populace to understand what it's for. That suggests Macworld Expo. As I've envisaged it, it's quite a boring, back-room product (which, we have to note, resembles the new Airport Extreme in 2007, which didn't even get a Macworld keynote mention), so an October release for the Christmas buying season would be inappropriate. I know that I could use this sort of product tomorrow, and would happily cancel my Infrant order to do so. That, I must acknowledge, is more wishful thinking than anything else in this article.

Any thoughts?

[I know I'm not the first to propose an iServe (though these couple links were found just after I finished the article). This is ultimately a revisiting of an idea that's been around for a while, with an idea on how Apple could make its mark on an emerging market category, and a survey of current developments with the Apple of 2007.]

I've been spending a fair bit of the last week puzzling through the various postings on ZFS performance. While Richard Elling's blog posts were informative, they didn't really tell me much about the workflow that interested me most: high-throughput multimedia streaming. I eventually took the question to the ZFS-discuss list, and got a lot of knowledgeable feedback. The essence of what I learned about various ZFS configurations, for my purposes (once I understood the superficial size/reliability/performance tradeoffs), gets boiled down to one choice:

Which is more important to your ZFS workflow:
random access or write performance?

I suppose this is old hat to people who are very familiar with RAID systems and/or ZFS, but it took some digging for me to find out. I'll set it out in words, in contrast with the pretty but dizzying graphs at relling's site. These guidelines obviously set aside any other bottlenecks, but as was consistently pointed out, media speed is usually the bounding factor with performance.

For mirrored configurations:

• Small, random reads scale linearly with the number of disks; writes scale linearly with the number of mirror sets.
• Sequential read throughput scales linearly with the number of disks; write throughput scales linearly with the number of mirror sets.

For parity (RAID-Z, RAID-Z2) configurations:

• Small, random I/O reads and writes scale linearly with the number of RAID sets.
• Sequential read and write throughput scales linearly with the number of data (non-parity) disks.

In other words, mirrors suffer on writes, collapsing to the number of mirrors, essentially. RAID-Z groups suffer most with random I/O, collapsing to the number of RAID groups, performance-wise, in those situations. A hypothetical table with two different configurations of 12 disks (four 3-way mirror sets vs. two RAID-Z2 sets) helps show the strong contrast:

	Random I/O		Sequential I/O
config	Read	Write	Read	Write
mirror: 4*3	12y	4y	12z	4z
RAIDZ2: 2*(4+2)	2y	2y	8z	8z

where y is the number of random, short IOPS and z is the sustained media throughput on the drives

My case is fairly clear: I want to both read and write multimedia streams fairly equally, so I favour RAID-Z groups. I don't need the same sort of long data life that others do, so I set RAID-Z2 aside for now.

In my particular case, I have 16 500GB SATAII drives to work with for the RAID. I am committed to one hot spare, so I'm down to 15 drives. Once I get my server, I need to know how much—and when—performance degrades when excessive numbers of streams are added and/or Random I/O requests are added to the mix.

For a long time, I had assumed I would use three sets of five-drive RAID sets. Looking at three-drive sets, I have to consider whether a ~17% drop in peak streaming performance is worth a 67% improvement in baseline small, random I/O (essentially the worst-case scenario).

How do I arrive at that? Five 2+1 RAIDZ groups have 10 data disks compared to the 12 in three 4+1 RAIDZ groups. If I go from 4+1 to 2+1 groups, I lose 2 disks worth of data storage and the equivalent amount of max streaming capacity, but gain two more RAIDZ groups for working on the seeking for random I/O. Actually, another table really makes the picture quite clear. I throw in a set of mirrored drives as further food for thought.

	Random I/O		Sequential I/O
config	Read	Write	Read	Write	Capacity
RAIDZ: 3*(4+1)	3y	3y	12z	12z	6.0TB
RAIDZ: 5*(2+1)	5y	5y	10z	10z	5.0TB
mirror: 7*2	14y	7y	14z	7z	3.5TB

If you've stumbled upon my Twitter feed, then you probably already know that the arrival and installation of my much-anticipated Apple TV was less than trivially easy. Our old television wanted nothing to do with the component output from the device (sound, no picture), so plans to obtain a new HDTV were moved up.

We ended up going to Costco for a 32-inch Samsung LCD, which has been a real pleasure so far. Once the AppleTV was hooked up, however, a new problem emerged. The network really was unkind to the AppleTV, and frequently dropped the connection between it and iTunes. No synchronisation was possible, and both iTunes and the AppleTV frequently got confused.

After some examination of the Apple discussion boards, it quickly became clear that the problem was my old Linksys wireless router. It limps along enough to provide ADSL connectivity and wireless access at home, but the ethernet ports on it are fried, and both ADSL and wireless have their bad days. The solution (reached after far too many trips between the ground floor and the attic) was to pull out an unused Airport Express, use the Mac mini as a bridge from the wireless connection to the router to its ethernet port, and set up the Airport Express as another wireless network to which the AppleTV connects. Ahem. So, put another way:
AppleTV --wlan-> Airport Express --e'net-> Mac mini --wlan-> Linksys --adsl-> Internet

I hope this is just a temporary measure, but it seems to have solved the problem for now. I now have a working AppleTV.

I've been watching Fake Steve Jobs for a while now, and now I'm realising that, true to his growing reputation as the Suck 2.0, writing in character and adopting a persona really can free one up, not only for iconoclastic pronouncements, but for making real insights. I was bowled over when I saw NBC and News Corp., sitting in a tree:

What we're doing today with things like Apple TV or TiVo is just applying Band-Aids to patch up a frigtarded system (linear TV programming) that made sense in the 1950s when bandwidth was limited. Our Apple TV only sidesteps the problem. It still forces you to download to your computer, then beam through a router to our TV box and then up into your TV. So great. Now you've got more pipes coming into your TV but this new pipe is kind of unreliable (wifi routers) and slow and clumsy.

The real fix is gonna happen when someone figures out the back end, aggregating good content (ie Seinfeld and I Love Lucy rather than Ask a Ninja) and then finds a way to get that straight into your TV without all these clumsy connections and multiple hops. But it's a battle. The linear model, as stupid as it is, still clings to life. Inertia is a powerful thing. But ultimately we'll win. Give us ten years. And yeah, this is why Apple is presenting itself to the Hollywood studios as a friend and ally, not a competitor. It's also why we didn't buy YouTube.

It's not necessarily the future, or the future as imagined by Apple, but it's definitely a (plausible) future...

I promised to go into further detail on what I mean between ownership and rental and the spaces in between. These are thoughts very much in progress, and with some aspects of it, my thinking on it changes daily.

I have "subscriptions" to certain multimedia/digital services for which I pay a flat monthly fee:

• ADSL: £25/mo for 20GByte download allowance,
• eMusic: $10/mo for 40 mp3 track downloads
• Amazon DVD Rental: £8/mo for four DVD rentals

With my internet access, I try to hit about 75% of my usage allowance in a given month, since usage caps were introduced at my ISP. Before then, I was paying more, getting less than half the throughput, and downloaded significantly less. I feel an internal pressure to get as much as possible out of this "limited" resource....