After five amazing years at Last.fm I decided to hand in my notice a few months ago, my last day was at the end of August. As a parting gift and sign of appreciation of the many things Last.fm has given me I produced a series of data visualisations of the scrobbles of all Last.fm staff, alumni, and community moderators I could find, and published it last week. In total the series encompasses 8.7 million scrobbles across ~180 graphs. The visualisation is a structured heatmap that is designed to reveal periodicities: years, months, day of week, hour of day. Storytelling …

» Full entry

Last weekend I was at the music hack day in London, organised by Dave Haynes and James Darling: a two-day event where software developers met up and wrote music-related software (or built hardware.) Instruments, a distributed content resolver, various SoundCloud tools, etc. Although the event attracted lots of interesting people from all over the planet (well, Europe) I ended up coding most of the weekend instead of talking. (On that note, I'm still amazed by the amount of time coding requires, even after you learned how to channel your ambitions more efficiently. Software development is still a painful process.) I …

» Full entry

Mark Downie on the Field-development list: More seriously, the trick I think with Processing is that it has been a wonderfully successful "stone soup". The IDE and its graphics architecture are left over from a different era and I can't see why it's language modifications are worth maintaining, but the vibrant community and library maker ecosystem is unlike anything else that's happening. But these two things are largely separable. The aim is to be able to connect with the latter while leaving the former behind. Quite a few people are realizing this — for example, lots of people are …

» Full entry

Over the past extended weekend I created Pool Radio, a tool that provides access to hopefully interesting Last.fm radio stations. See also the announcement in the Subscribers and their tag radio stations group forum, with some great comments by Nectar_Card. I'm aware that not a lot of people will find this site very useful, but people with an appreciation for the random and obscure can definitely benefit from it. Here are a couple of great user tag stations I've enjoyed over the last week: raw_u's etiopia tag radio (tag page), jirkanne's lllllllllllllll tag radio (tag page), JessiCoplin's scott storch tag …

» Full entry

Your use of the Software is limited to a fixed number of users and weblogs, as defined below and enumerated in Exhibit A of this agreement. [...] EXHIBIT A ========= Unlimited Free Version Term: Perpetual Number of Servers: 1 Number of Authors: Unlimited Number of Weblogs: Unlimited Support Level: none included or purchasable Updates and Upgrades: free Updates Maintenance Fee: None Am now seriously considering to update (from 2.64). Since I'm not planning to ever make any kind of money from this (including ads) all my previous reservations are moot. Edit: Hah. Turns out they're about to GPL the …

» Full entry

Get the JRuby Console, and then do stuff like this: irb(main):001:0> require 'java' => true irb(main):002:0> p = java.util.regex.Pattern.compile("/music/[^/]+/\\+images/[^/?]+?(?:\\?.*)?") => #:0x5efe34 @java_object=/music/[^/]+/\+images/[^/?]+?(?:\?.*)?> irb(main):003:0> p.matcher("/music/Christopher+Willits/+images/?").matches => false irb(main):004:0> p.matcher("/music/Christopher+Willits/+images?").matches => false irb(main):005:0> p.matcher("/music/Christopher+Willits/+images/").matches => false irb(main):006:0> p.matcher("/music/Christopher+Willits/+images/34523").matches => true Awesome! I've been waiting years for a convenient interactive Java shell -- and not to run the language, but to interactively test behaviour of library features under certain conditions. Don't want to write & compile a new class just because I want to check something out. …

» Full entry

As recently mentioned I was looking for useful hacks for the iPod Notes feature, and as Google didn't turn up much of interest I started writing one myself. The obvious first application: an offline feed reader. Turns out it's remarkably easy to do this. The Notes file format is basically text with some HTML markup, and even allows for links between individual documents. I.e., converting a feed into a series of notes and an index is pretty straightforward. The harder part is the syncing mechanism. My requirements were: syncing has to work on a default OS X installation, it …

» Full entry

A friend just told me this: Turns out the MP3s on your iPod are completely accessible, even without any dedicated software. They're simply in hidden folders. Writable hidden folders. …

» Full entry

Was browsing the Mozilla developer docs, curious about their documentation on JavaScript language updates over the different Firefox versions (JS is getting more and more Ruby-like). Realized that I need to brush up on my JS knowledge, maybe finally time to get that O'Reilly book. Found an "Introduction to the JavaScript shell" (they have a JS shell?!?), and this: cvs -d :pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot login cvs -d :pserver:anonymous@cvs-mirror.mozilla.org:/cvsroot co -l \ mozilla/js/src mozilla/js/src/config mozilla/js/src/editline mozilla/js/src/fdlibm cd mozilla/js/src make -f Makefile.ref Compiled it, ran it. print works. Cool! Now I just need to figure out what to do with it. …

» Full entry

Having \n as the only type of newline. TSV as ubiquitous data exchange file format -- it even has its own mime-type. NTP on every machine. UTF-8. It's the simple things... …

» Full entry

I'm start to look into more secure ways to store sensitive data, and Apple's encrypted DMG disk images seem like a good compromise between security and convenience. If you're worried about long-term storage and retrievability it of course has the disadvantage of being a proprietary format, which means you would need an OS X machine to decrypt those disk images. Not any more! In one of the interesting talks I missed during last year's 23C3 (while being busy doing other things) Jacob Appelbaum, Ralf-Philipp Weinmann and David Hulton presented their successful attempt to reverse-engineer the file format. They provide slides …

» Full entry

People seem to be migrating in drones from the former golden boy of the Rails community: the Typo blogging engine. Reasons vary (Sporkmonger's Bob Aman is put off by the lack of effective spam filters) -- but I can't say I'm surprised. Never installed Typo myself, but have been curious about the surprising amount of people who deem it acceptable to replace a solid solution (say, WP or MT) by something that exchanges stability for Ajax widgets. Apparently one year later the trend-setters can finally admit that flashiness is only cool when it doesn't require cleaning up every couple of …

» Full entry

Since I got my PowerBook early last year I'm on the search for decent alternatives to some of the applications I was using on the PC. Thanks to a flourishing development community I found replacements for the most important stuff (and more), but one thing that is still missing is a decent multi-track audio editor suitable for music production. I've been using Samplitude on the PC for the last couple of years, which after a brief learning curve turned out to be perfect for my needs -- it offers quite elegant means of editing audio, including non-destructive filters and timestretching …

» Full entry

I'm using a combination of fairly primitive methods to cope with blog spam. As this blog doesn't get too much comments the amount of manual work is relatively limited; main line of defense is an old-fashioned and relatively short blacklist. I'm notified of incoming comments, and in the rare event that a spam comment gets through I'll inspect it for new keywords. For a couple of months now it's become apparent that specific posts seem to attract more spam than others. I just thought that it may be great to have a statistic of this phenomenon -- so that I …

» Full entry

I've been suffering from a minor iTunes annoyance for a while now and finally decided to look into it: There is no 'clean' way to import an audio podcast file into your main library. Audio files that come from a podcast feed are treated differently to 'normal' audio files in a number of ways, and sometimes that's not what you want. The simplest solution in most cases is of course to re-download the file from a browser or the commandline -- but in my case that didn't work because the file was no longer online. And dragging it to the …

» Full entry

I decided on a whim to mirror "Network Forensics Evasion: How to Exit the Matrix" on my server, at least temporarily. This fairly elaborate text describes a number of technical (and some non-technical) means of hiding and obfuscating your "data trails". While this traditionally has mainly been a concern of crackers and dissidents, it's of increasing interest to the average consumer. I just started reading, so I can't say much about the quality of the document. The text comes with a disclaimer: I try to be as operating system agnostic as possible, providing information for Windows, Mac OS, and Linux. …

» Full entry

I found an older article on the Google Reader blog where they post a great bookmarklet: It displays feeds referenced by the current page and allows you to preview them in Google Reader. Convenient for their users, and a great building block for a more generic tool. I've modified their code in a number of ways -- the most obvious change is that the bookmarklet-generated links will point directly to the respective feed URLs, not to a Google Reader preview page. Then there are some cosmetic changes (I didn't like their choice of generated markup), but not much else. …

» Full entry

del.icio.us apparently has just added a feature that I've been wanting for a long time: It's now very easy to see the history of bookmarks for a specific URL without having to bookmark it yourself. Here's an example of such a bookmark history page: del.icio.us bookmarks for mailfeed.org. I regularly check these URL bookmark histories on del.icio.us, because it can answer all kinds of interesting questions, e.g.: How popular is this URL? Since when have people known about this? Who bookmarked this URL first? What are their comments? I imagine this caters to a small audience, but it's a …

» Full entry

I just started to write an aggregator in Ruby which will form the basis of a number of web applications, and a couple of minutes into the project I'm already excited about the expressiveness of Ruby and its standard library. So much so that I had to share the results of my first five minutes of coding. I decided that the aggregator I'm writing will take its feed URLs from an OPML document. A nice property of OPML is that it allows you to group feeds into a hierarchy of named elements, so that you can e.g. group some feeds …

» Full entry

Recently there have been a number of requests for new ideas in the aggregator market, and as I'm constantly dissatisfied with my feed consumption experience (no matter the tool) I have lots of opinions on the state of aggregator software -- and even some ideas for improvement. I'll save the grand overview for later; because some things are better shown than told I thought a good start would be to show sketches of what I'd like to see in the next generation of aggregators. Here's Sketch One, which is kind of an accumulation of concepts, and which describes the basis …

» Full entry

Just found out how to work around a major nuisance of GMail: the inability to select the low-tech HTML view as default interface. It's probably going to be old news for a lot of people, but it was new news to me, so I'll post it here for others to see. Background I've been using GMail as a secondary email provider for a while now, and while I like that its Javascript-based interface affords you speed improvements and other nice enhancements I can't get used to a major drawback: you lose the browser's history function, at least in Safari. Others …

» Full entry

Another 22C3-related Ruby script: I thought it would be neat to convert the list of Weblogs writing about 22C3 into an OPML file which can then easily be imported into your favorite aggregator. Ruby to the rescue. The script scrapes the Wiki page (or rather its export format version) and iterates over the table of blogs, adding a feed URL of each blog to the OPML outline. Look at the code for further information. Of course you can also simply get the exported OPML file linked below; but I expect that the Wiki page will change a lot even after …

» Full entry

Just before I left my house over the holidays I wanted to print out the schedule for 22C3, and found that I couldn't -- no matter which browser, OS and method I chose, the schedule table was unreadable. A couple of hours later I sat in a train and had some time to spare, so I hacked together a Ruby script that parses the iCal version of the schedule for 22C3 and creates an HTML page with a clean, readable timetable that you can also print ;) You can get the script below, and a version of the generated HTML …

» Full entry

FeedTools is an amazingly complete Ruby library by Bob Aman for accessing, parsing and generating feeds from within Ruby. While it is designed to work well with Rails applications, you can just as easily use it in your Ruby scripts: feed = FeedTools::Feed.open('http://dekstop.de/weblog/index.xml') puts feed.title feed.items.each { |item| puts item.title puts item.link } Requesting a feed every time you run your script is fine as long as you only parse your own feeds, but you should be a bit more polite as soon as you start requesting someone else's feed. Over time feeds can cost site owners a lot of …

» Full entry

I've written a Ruby script that extracts keywords from a MovableType-exported plaintext database, and while doing so wanted to include the Rails Inflector so that the script could merge singular and plural versions of the same word. It wasn't that obvious to me how to include the Inflector in Ruby scripts outside of Rails, and I searched for a while until I found the proper usage; so I'll document it here to save other Ruby newbies some time. In the end it boiled down to finding the proper require statements -- I'm not sure if this is the best way …

» Full entry

A friend has asked me for a way to read email newsletters from within his feed reader, and after some digging around I found it's straightforward enough to access POP3 mail from within Ruby, so I created mailfeed.rb. mailfeed.rb is a Ruby script that generates an RSS 2.0 feed from the content of an email inbox. This can e.g. be used to read email-based newsletters from within your feed reader. (See also: MailFeed for PHP, pymailfeed for Python.) It's really a pretty simple script, so there are a number of caveats -- e.g. there is no limit on the …

» Full entry

CollaborativeRank is an interesting service that builds on the del.icio.us database. They provide bookmark search, a ranking of popular bookmarks, and they attempt to find connections between the things people store in their del.icio.us account and their area of expertise. It's the last feature that I find the most interesting. While it disguises as a ranking of users, its main promise is that it could help you find experts on arbitrary fields. During the last couple of weeks I've been watching my rank, and while I wouldn't necessarily agree with its estimation of my expertise it's still interesting to watch. …

» Full entry

Just read in a comment by Esteban Kozak that SearchFox RSS uses both "attention and community data" when determining the value of an article, which means that some of the weird effects documented earlier might be a result of other people's behavior, as opposed to my own. To recapitulate: I'm trying to understand the algorithms behind SearchFox RSS's "Topics I Like" listing, and found that some terms are conspicuously high on the list where they don't really deserve to be (currently: "quake", "ning" -- see image below), and others that I care about more are nowhere to be found (currently: …

» Full entry

I don't get it. Around the time I wrote about SearchFox RSS's "Topics I Like" feature I adjusted some of my reading habits (notably minimizing the consumption of web two-point-oh hype, and subscribing to more non-tech-oriented feeds), and the list of "topics I like" hasn't really adjusted to that. Maybe I'm too impatient, but I was presented with about 1.200 articles since last Wednesday and the list of "topics I like" seems rarely changed. I've included a screenshot of my current dataset below; I've also appended the words to the original data set. Note how e.g. "Ning" and "Quake" are …

» Full entry

Chris Zumbrunn has news in the comments of my article "RFE: Server-Side Javascript?": There is a new JavaScript execution framework called Rhinola which looks like what I asked for: a framework that enables server-side JavaScript web development! A quick search leads to an enthusiastic article ("whoa! rhinola rocks!") on the haboglabobloggin' blog (which just went offline while I was writing this...), but the author mentions that the current incarnation of the software requires quite some Linux admin-fu to get it working; I assume this will change over the next year as the product matures. Rhinola is currently based on the …

» Full entry

For the last two weeks I've observed SearchFox RSS's list of "Topics I like" to both find out how it's working and to see if it accurately reflects my taste. See my earlier article "SearchFox Rocks. But Where Are the Web Services?" for a little context. Random observations: The keywords are indeed ordered by rank, the first keyword being the most highly ranked. You can deduce this by comparing the keyword movements at the start and end of the list over time: lots of movement at the end of the list. Easy come, easy go. The list actually reflects what …

» Full entry

SearchFox is really great. It's a web-based feed reader (currently in beta) that watches you reading feeds, and which uses this attention data to improve your reading experience. After you have used it for a while SearchFox develops an understanding of the things you care about, and presents these accordingly (feed articles are sorted by ranking, not time). How SearchFox works There are several ways to tell the application that you like a specific feed article: by reading the article (in SearchFox you are presented with a list of headlines and some metadata, and have to click a link to …

» Full entry

In June this year I started looking for ways to use JavaScript as a server-side scripting language to replace Perl, PHP and others, and documented my findings in the article "RFE: Server-Side Javascript?". Although all the pieces to bring JavaScript to the server are there, nobody has actually undergone the effort to implement it yet; but it seems there are more and more people interested in trying it, and I guess by this time next year we'll have some stuff to play around with. The reason I'm posting again is that I just found another related project: TrimJunction, which is …

» Full entry

Judging from the screenshots, iTunes 5 has put an end to Brushed Metal!! Haven't installed it yet myself, but I'll take it as a sign for good things to come in 10.4.3. I'm also curious what other changes have lead to the major version jump -- let's hope they have made some improvements to the podcasting workflow. Judging from the feature overview page the sources list can now have folders, which is a really useful addition. And what the hell does this mean: "now supports iPod syncing for Outlook and Outlook Express on Windows PCs"?? …

» Full entry

I already gave a basic description of Pandora in my previous article, "Finally: An Alternative to Last.fm". Brief recapitulation: Pandora is a music streaming service that lets you control the kind of music that is played. You can define "stations" by bookmarking song titles or artist names, and the site then plays music that shares similar properties with your selected songs. In contrast to Last.fm, which is a social network, Pandora builds on a concept of rich metadata to find relationships between individual songs. This time I had the opportunity to actually use the system, so this should make an …

» Full entry

Tom Conrad has apparently just presented "Pandora" at Bar Camp. Scoble quotes an email from Tom with a short description of the service: "Pandora is a "music discovery service" designed to help you find and enjoy music that you'll love. It works like this: you give us the name of an artist or song and we instantly create a "station" that plays songs that share musical characteristics with the artist/song you entered. From there you can fine-tune the station to your tastes by giving us feedback on the individual tracks we play. You can make up to 100 unique stations …

» Full entry

I'm on constant lookout for a better system to maintain this website -- currently it's a rather loose mixture of MovableType 2.6.x, custom PHP, and individual HTML pages, and it really shows. There is more inconsistency that I would like to have, and it is not easy to maintain -- every change to the layout requires modifying several template files, and modifying twice as many HTML files. Add to this the fact that some pages require different layouts (which sometimes involves custom style sheets). In short, it's a mess. I'd like to have an integrated solution, and it wouldn't hurt …

» Full entry

I've been using modified del.icio.us bookmarklets to feed my del.icio.us account, and thought I should share those. More experienced users will find nothing new here, but it's just as likely that it will interest some people. del.icio.us already have a list of bookmarklets for various browsers, and I started from there. I have made two additions to the default bookmarklets: a faster way to enter an extended bookmark description, and the ability to add a predefined tag. This is my generic "add to del.icio.us" bookmarklet, "add bookmark": javascript:location.href='http://del.icio.us/new/martind? url='+encodeURIComponent(location.href) +'&title='+encodeURIComponent(document.title) +'&extended='+encodeURIComponent(getSelection()) Note the code after "extended=" -- this tells the …

» Full entry

I'm currently testing the Safari CookieFilter script and have stumbled upon strange launchd behavior that I can't explain nor find a way to fix. Essentially launchd seems to be fairly liberal in how it interprets LaunchAgent triggers, and regularly seems to choke under conditions that I can't reproduce. The current release version of the CookieFilter LaunchAgent is designed to trigger the script every time there are changes to your Cookies.plist storage. But after running the LaunchAgent for a couple of days I found that it seems to stop responding to changes of this file. With some testing I found that …

» Full entry

I was looking for ways to automatically start my Safari CookieFilter script, because if I had to launch it manually every time I wanted to clean up Safari's cookies I would never really use it. On a Windows machine I would put the script into the Autostart folder so that it would be executed upon system startup, but I've found that I never really reboot my Mac, so that wouldn't work. Another option would be a script that invokes CookieFilter before it launches Safari, and to only start Safari via this script -- but this idea seemed to simple and …

» Full entry

Since Tiger I've been pretty satisfied with Safari, and there are fewer reasons to install third party plugins than ever. Yeah I could use Firefox's keyword search, but after having one too many browser crashes due to a SIMBL plugin running amok I figured I could just as well live without it. The only thing that has really been missing was a working Cookie whitelist. There is Cookies Eater, another SIMBL plugin; but that hasn't been updated for the current Safari version yet. And even if it had been, I'm not so sure that SIMBL is the best way to …

» Full entry

I've just started using TextMate. My initial assumption was that it is still early in the development process, so I planned to wait for a later release before I would decide if I should register a copy; but then I started using it for a couple of days and found several functions that blew me away. A tip of the day introduced the "^R" shortcut that executes the current line as shell script and inserts the output into your document; this is quite useful to me as I'm starting to use the shell and especially shell scripts for various text …

» Full entry

I'm in desperate need for a better backup strategy. By now it has become clear that our data storage media are utterly unstable; I've had lots and lots of misburned CDs, CDs that I can't read after years or even months, of after I change CD drives; I've had backups on DVDs that only can be read on specific computers, and at least four hard drive crashes (that I can remember) during the last 3-4 years. So I'm always interested in what other people have to tell about their experiences. Here's an interesting discussion on MetaFilter about Safely Storing Mp3 …

» Full entry

I just read another rant about the many weaknesses of PHP, and of course I'm in complete agreement. Over the last years I've come to use PHP more often than I would have liked to, simply due to the fact that it's pretty much the only scripting language that is available virtually everywhere; and in contrast to Perl I don't have to look up function calls and the language reference as much, simply because it is closer to what I'm normally using (namely the C languages, and Java). But the reason I'm posting this is not my urge to awaken …

» Full entry

Designed as a replacement for the Windows text editor Notepad, Florian Balmer's Notepad2 is a small, fast, simple and beautiful open source application. Along with an impressive set of basic text editing features it provides several useful tools and lots of neat little helpers that will improve virtually all text editing experiences, both for the programmer and the occasional user. Notepad2 has a nice interface with well-designed icons. It provides customizable syntax highlighting for many file types: all major programming languages (including perl and python), various file types relevant for web development, along with some rather exotic ones like XUL, …

» Full entry

I've just spent several hours looking for a free or open source hex editor, and it is surprising how many editors there are, and how few are worth checking out. I didn't think I would ask for much: a simple standard interface and the ability to open large files. I must have looked at more than 50 websites and installed nearly 20 applications, until I found HexEdit from Catch22 Productions. I have used the free version of ECS HexEdit for quite a while, but its nag screen and ancient interface started to bug me. xvi32 is a nice editor, and …

» Full entry

Alex Moffat has created a wonderful little tool to rapidly create UML sequence diagrams. The GPL licensed Java application, appropriately named "Sequence", creates diagrams from short sequence descriptions and allows you to export them to a PNG file. Sequence can't replace more sophisticated applications like TogetherJ or Visio, lacking e.g. support for asynchronous messages, but serves well as a small tool for quick sketches. You can literally design your first diagram seconds after downloading the Jar file. Example of a small sequence diagram: Loading a media file via the user interface. The MediaPlayer class notifies its caller of the properties …

» Full entry