dekstop: weblog

I already gave a basic description of Pandora in my previous article, "Finally: An Alternative to Last.fm". Brief recapitulation: Pandora is a music streaming service that lets you control the kind of music that is played. You can define "stations" by bookmarking song titles or artist names, and the site then plays music that shares similar properties with your selected songs. In contrast to Last.fm, which is a social network, Pandora builds on a concept of rich metadata to find relationships between individual songs.

This time I had the opportunity to actually use the system, so this should make an article more focused on experience and less on speculation... I'll try my best. Thanks to Baher Al Hakim for the invite, and to Pandora's CTO Tom Conrad whose invite arrived in a kind mail just 1.5 hours after Baher's (I had contacted them both; yeah I was that needy to try out the service. I was actually surprised to hear from either of them, normally people don't respond to such requests from random people. Greetings from Berlin!)

Regarding the "Non-Social" in the title: I just thought that these days you have to point it out when it's not there. It's worth a mention that there still are people who try out non-derivative approaches; Pandora could just as well have implemented a system based on tags, friend lists, and other established methods, but they chose not to. (And by doing so they have potentially introduced a new paradigm; but a discussion of this would spread the already lengthy article unnecessarily.)

Executive summary: I'll be dropping Last.fm and will use Pandora instead. It's a really great service.

Initial Impressions

At first glance the interface is a revelation compared to Last.fm -- it basically consists of a UI to manage your list of "stations", and of player controls. That's it. No complex database browsing, no user pages, no community features. And I really liked that. Focus.

One of my first steps then was to test the database: Let's search for exotic music!

They have Bloc Party alright, but that should be regarded as a given.

Searching for Atom Heart returns a rather longish list of suggestions, all of which are song titles from Ray Charles, Grateful Dead, and many other artists that I don't want to mention here. Most of the song titles are variations around "Straight from the Heart". Uh. I can tolerate that they haven't included Uwe Schmidt, but please don't offer S Club as an alternative.

But that's just a minor flaw that doesn't spoil the overall appearance. Next search: Björk, which again is an easy one, but which serves as a basic test of their handling of non-English character sets. And it works, kinda: "Did you want Bjork?" Yes, please.

Next try: Pharcyde. Also quite popular in their time, but by now exotic enough to use as a test case, yet relevant enough to be a must-have. "Did you want 'Soul Flower (Feat. The Pharcyde)' by The Brand New He". The weird Flash interface fragmented the question, but I understand the attempt to communicate that you don't really have what I was searching for, and would like to offer a collaboration track instead. Fair enough. (Turns out the replacement is no substitute, to put it mildly.)

They have Coltrane, but the "similar" music that is served with it tends to circle around bar Jazz themes, which is one interpretation of what I was looking for, but not necessarily what I was expecting. The Sun Ra station likes to drift off into Crooner fluff and Big Band Jazz, which surprised me as well.

In the end, most of the queries I tried yielded results (they even have Muslimgauze, Michael Mayer et al). It might just not always be what you're expecting. But Last.fm is no different: these types of services seem to be really bad at serving specific expectations, but are really good at surprising you with stuff that you would have missed if you were just filling a playlist manually.

After browsing some more and making notes in the process I finally forgot about the stations and left the music running in the background, and although there were some, erm, interesting choices of music it usually didn't bother me. After you leave it playing for a while it can occasionally run out of control, but a reset is only a mouseclick away.

Like Last.fm, this really is an adequate substitute for radio.

Sidenote, and a Prediction

Why did they use Flash for the whole interface? I can't imagine that it simplifies the development process, and it's certainly no treat for the user, which becomes apparent when you need to navigate larger search result sets, or when you find that some of the most interesting features are hidden behind weird menus, or to be honest if you want to do anything besides using basic player controls.

At this point I wonder if it even makes sense to give a more concrete description of some of the interface flaws; the core problem is that they made up their own interface, and that the interface isn't always very consistent. It's working, but it's not great. But you knew that already from the start of this section, didn't you? Yeah, I wonder too when the time has finally come when people will stop inventing imperfect stuff in Flash that has already been perfectly solved in HTML.

Also they seem to be using Flash cookies to store stations. I set my Flash plugin to ask before it stores cookies on disk, and I deny every single cookie; the constant nagging becomes a little annoying after a while, but I'm stubborn like that. The site works anyway, and the stations are still there after you restart Safari. Can somebody please explain to me again: why do developers still use cookies for things besides login data?

These cookie requests seem to add up: if you leave the Pandora page unattended for a while and then come back, you are confronted with several successive requests from the Flash plugin, all asking if the page should be allowed to store its cookie. And these requests are presented in such a way that it's not really clear if you are confronted with a new dialog after you have clicked away the first, or if the interface simply isn't reacting to your clicks. This is a usability problem of Macromedia, not of the Pandora implementation, and most people won't bother enabling their cookie filter anyway; but still, I like to block unsolicited cookies, and there is no reason why cookie handling shouldn't be implemented with a little more moderation on the side of the Flash developers.

My prediction, regardless of the cookie issues: The Pandora team will learn it the hard way and gradually switch to HTML, and eventually only use a minimal Flash-based player applet. Cf. Flickr, who did learn this lesson by listening to their users. Let's just hope that this change happens sooner rather than later, and that Pandora won't start running amok refining workarounds to the interface problems Flash applications introduce, and lose money while doing the same mistakes others have already learned from.

See also the excellent writeup of reasons for Flickr's decision to reduce Flash usage at Ajaxian.com; and Scoble's claim that Flash interfaces don't suck for a contrasting opinion.

After the initial excitement has worn off, these day-to-day usability problems will become a major factor in the perceived value of the service. Last.fm had big problems with webserver load, which made it a lousy long-term user experience.

Intermission: Bug Reports

Of course the application is in an early stage, so there are bound to be bugs; to round up my impressions I'll present some more concrete examples of the issues I found.

Safari introduces a tab-related bug: if the Pandora player is in a background tab and a song finishes, the new song doesn't start until you bring the tab to the foreground again. After some quick research it seems likely that this is indeed a flaw in Safari's event handling mechanism, not a bug of the Pandora player. My workaround is to simply open the player in a window of its own, and leave it in the background. (The above link at Ajaxian states: "If 'Flash doesn't load in the background,' then I take it you are a Safari user. Talk to Apple about. It's the same for other data handled by plugins, including movies and music.")

I'm sure that Pandora's menu structure could be improved upon. There are three different submenus (with different look-and-feel, triggered in three different ways) that contain various functions; this grouping in three logical groups makes some sense if you know the product, but it's not inviting to new users. You have to e.g. remember that the "invite a friend" feature is part of the station-specific menu. And it has the effect that most non-basic features are hidden from normal view and require at least two clicks where Last.fm only requires one; these are even functions you might end up using regularly, such as the "I like this"/"I don't like this", and the phenomenal "Why did you play this song?" (more on that below).

Pandora, at least during the beta period, is an invite-only service. I don't yet understand the reasoning behind their implementation of the invitation process: you don't invite people to join the site, but to listen to a specific station, and only then can they join the site. The good side of this is that the receiver of an invite can immediately start listening to music, and this kind of instant gratification makes a good first impression (If he likes what he's hearing, at least). But it's a strange detour if all you want to do is let someone join the site.

Inner Workings

At the core, both Pandora and Last.fm operate on similarity metrics, and both offer the metaphor of "stations" to describe the notion of a point of reference in the recommendation system; but they use different methods to get there. Pandora's paradigm can be described as "rich metadata", in contrast to Last.fm's "complex queries".

Stations on Last.fm are based on user profiles: similarity between individual songs and artists is deduced from user listening behavior, which means that the content of a station changes over time as the listening behavior of the stream owner progresses. Anybody can benefit from this effect by listening to other people's stream, no further interaction required. This week a station might prefer Prefuse73, next week it's Steely Dan. That's the basic description; in reality they have implemented a complex network of social interaction processes that goes quite a bit further than that.

Pandora implements a distinctly different approach: all descriptive data is statically encoded by a limited group of schooled listeners, and your stations are basically bookmarks to database searches, where the query is an artist or song title, and the result set is a list of music that shares a similar genotype. This means it's easier to find definitive connections between songs (I'm reluctant to use the word "objective" in this context); but it only works if you put a lot of effort into your methodology.

It would be interesting to know if stations on Pandora can change as well, if they can have different result sets two months from now, even if you disregard new music that has been added to the database. Because that was always the coolest feature of Last.fm: the fluidity of the content.

It Entertains, No, It Educates!

Pandora's "Why did you play this song?" feature offers a hilariously entertaining insight into the system's inner workings. Hold on, scratch that: Pandora educates you! It makes you aware of how music works, on a basic yet articulate level, and it can teach you valuable instruments of communication about culture. This could become a tool for music teachers and their students.

Some examples:

• Based on what you've told us so far, we're playing this track because it features electric rock instrumentation, mild syncopation, a subtle use of vocal harmony, a vocal-centric aesthetic and minor key tonality. (Bloc Party)
• Based on what you've told us so far, we're playing this track because it features electronica influences, mild rhythmic syncopation, a vocal-centric aesthetic, repetitive melodic phrasing and a clear focus on recording studio production. (Björk)
• Based on what you've told us so far, we're playing this track because it features modern r&b stylings, demanding vocal performances, mild rhythmic syncopation, a vocal-centric aesthetic and string section beds. (Brand New Heavies)
• Based on what you've told us so far, we're playing this track because it features block piano chords, lead big band, angular melodies, interesting part writing and a groove oriented approach. (Sun Ra)
• Based on what you've told us so far, we're playing this track because it features electric rock instrumentation, mild rhythmic syncopation, mixed minor & major key tonality, mixed acoustic and electric instrumentation and many other similarities identified in the music genome project. (Yes)
• Based on what you've told us so far, we're playing this track because it features electric rock instrumentation, mild rhythmic syncopation, demanding instrumental part writing, a clear focus on recording studio production and an electric guitar solo. (Steely Dan)
• Based on what you've told us so far, we're playing this track because it features world music influences, avant-garde leanings, lead electric guitar, a groove oriented approach and subtle latin rhythms. (John Zorn)
• Based on what you've told us so far, we're playing this track because it features electronica roots, a unique form, a tight kick sound, synth hat and minimalist arrangements. (Byetone)

Impressive stuff. A bit like an "A new word every day" calendar. If I was a cynic I'd guess that progressive music journalists are already using the website as a basis for cut&paste review jobs; I know I would.

And what is this fascination in western popular music with "mild rhythmic syncopation"?

So...

You can't teach a machine taste. That's easy to forget with each new product announcement, but it's true now and it will always be true. It's not a bad thing, it's just good to remember this limitation when judging such a service. But even the primitive playlist randomizations of iTunes and musikCube have revolutionized my music consumption, and brought surprises in both directions that I would have never had with good old Compact Disk.

Both Last.fm and Pandora are definitely a step up from randomized playlists by adding levels of complexity wrapped up in an accessible interface, and there is no telling whether one will be the more successful approach in terms of "keeping up to the promises". Last.fm currently has more interesting content, at least for my taste; but they've had enough time to accumulate it. Both approaches establish a notion of "similarity" that only really works in specific contexts; but it's often the flaws of such approaches that make it an interesting listening experience.

Pandora still have to demonstrate how they can manage to serve a larger audience's taste and still remain cost effective. Last.fm clearly has the more scalable model to create and manage new content-specific metadata, but they make it up by forcing users to install a proprietary player software (which, as mentioned before, stopped me from using it any longer).

Aside from that, long-term success of either service will be more the result of business infrastructure than of the specific recommendation methods. It's probably like any other business, sadly: licensing schemes, marketing and presentation will be more decisive to success than the respective factor of technological invention, or the degree to which listening behavior can be accurately modeled and served, which to me is the more interesting aspect.

In closing I'd like to reinforce my advice to the Pandora developers: reduce the Flash interface. You will save yourselves a lot of work if you do it sooner rather than later. I know it's painful, but it's unavoidable.

Oh, and a request: bookmarks ("stations") are nice, but what I really want is a search field that is visible at all times.

Further Reading

• Tom Conrad, "Post BAR Camp Feedback on Pandora"
• juniorbonner, "Pandora's box of music"
• TechCrunch, "Dig into the Music Long Tail - Pandora"
• This is Berry, "Space age jewish tunes out of Pandora's box"
• Craig's Musings, "Pandora rocks!"

---