Hacking Soundcloud: Creating an Interactive Track

#Bringing the message to the medium 🌐

Like most people in electronic/club music scenes, I spend a lot of time on Soundcloud. I was an early, hardcore adopter of the platform. In fact, probably within the first 3000 users, as shown by my ID in their system:

{
  "id": 2925,
  "kind": "user",
  "permalink": "rawfare",
  "username": "Rawfare",
  "last_modified": "2017/04/21 14:45:27 +0000",
  "uri": "https://api.soundcloud.com/users/2925",
  "permalink_url": "http://soundcloud.com/rawfare"
}

I'm what they call a "digital native" — always produced music on computers and started DJing that way, too. But given the history of the scene, I also felt the need to master “real DJ-ing” as well. One day I got a good offer for a pair of Technics 1210s and decided to take the plunge.

As I was playing with the turntables, I came to realize how natural it must have felt for the original wave of electronic musicians to have their work distributed and played on vinyl. They would manipulate the music with their hands, which in turn radically influenced the structure and rhythmic patterns of the tracks they were making. It was also similar to how they experienced the music they grew up with, with the same mode of listening and cultural relation. The direct sensory and emotional connection made music easier to appreciate — both for the maker and the audience.

I felt semblances of that, growing up mostly listening to copied cassettes(!) and files/mp3s (skipped CDs almost entirely — a ridiculous format). Today, this is different - the direct, personal connection is fully severed. Think about it. Almost all of our interaction with music is managed and mediated by middlemen — Spotify, Youtube, Soundcloud, Apple... They determine the full context within which the music appears. The artist is left controlling only the cover art, which totals up to only 5% to 10% of the visible screen space. Even if you embed a player in a website, the player will act against the artist. After a track finishes playing, a semi-random one will follow without asking. Or a nag window will pop up to nudge the user to visit the platform instead. In essence, both the artist and the listener are moved as far from the controls as possible, in order to tweak, optimize and steer their actions towards whatever makes the host platform appear successful.

The situation is incredibly bizarre — the devices in our pockets are more powerful than the computers that made the majority of electronic music. You could literally have the same music generated on the fly by your phone. And, it could be interactive in ways never before possible. It's the most mindblowing yet ignored and underutilized musical context ever.

I've been possessed by this realization, and started exploring the possibilities of taking back control. There are ways to make music more interactive, and artists should be messing directly with the format of how we publish our musical ideas today. It was vinyl, then CDs, and now it's the Internet itself.

I'm a music producer and performer first — I didn't give much shit about computers and programming, not more than it was necessary anyway. I see these things as tools of expression. But every musician should understand their instruments and the context of their work. Today, this context is the Internet, and we need to deal with it 😎

#The track 🐥

The track Pipo is a tribute to my crazy Amsterdam neighbor's parrot. My neighbor at the time was spending a lot of time at home, much of it screaming at her parrot Pipo at regular 15-20 minute intervals. In our flat, it became both a running joke and a genuine nuisance. I even considered adopting the bird. One day in 2014, the vocal performances inspired a simple rave-tinged track that just sort of materialized, and I've been sitting on it since.

In many ways, I found it to be a great fit for a fun experiment. I've increasingly felt that releasing tracks in a standard, “static” way doesn’t make much sense anymore. It doesn’t “work”. The shelf life of a typical release is getting ridiculously short — you get a week, maybe two of peak attention at best. Great works get buried under the avalanche of new content, racking up only a few hundred listens. I wanted to reflect this relationship somehow — by connecting the markers everyone seems to care about (play counts etc.) to the content of the track itself. In short, the idea was to make Pipo “alive”.

The real world Pipo commanded constant attention and sacrifice. Just like the neighbor's screaming, the track had to be as annoying as possible. As a lifelong fan and explorer of distortion, using that effect was an instictive fit^{1I'm well aware that distortion in electronic music feels strangely dated right now. It's a cringy hipster thing that got decontextualized. The original aesthetics grew out as an expression of the natural environment of the producers — using cheap, lo-fi equipment that nobody wanted back then, instead of the rare and expensive “vintage analog” collectors items traded by the me-too wannabes today. Amazing people made the noises beatiful and promoted it to the focal point of their work. Nevertheless, I think I deserve a free pass for this. Distortion is a huge part of the sound of my Rawfare project. I'm exploring a new sonic territory now, but I’m happy I’ve found a good, fresh angle to use it. (btw — I want to thank Adam Harper for driving this notion home for me. These lectures have completely changed how I think about music and sound design. It even made me start selling some of that analog gear I started to slide into😛)}. I put the final master of the track through some of my favorite tools and ended up with a handful of trashed-up versions.

With that, my idea was fully formed — upload a completely distorted track to Soundcloud, change it to progressively cleaner versions as more people listened to it, and gradually dial the distortion back if the weekly play counts are insufficient. This is the result in its very current state:

Update 2018: here's the player from my interactive music engine songsling.io, I've built it so you can also make projects like this.

#The Soundcloud Replacer 🤖

If you pay for a Pro account on Soundcloud, you'll have the possibility to replace the audio file uploaded for the track, without losing any of its statistics. This feature is a godsend in case you make a mistake that needs to be corrected, like a bad version of the recording. However, a more interesting use is to CHANGE the track entirely, depending on some feedback.

In order to do this, the replacement process had to be fully automated. My initial thought was to program it using the Soundcloud API, which is short for Application Programming Interface. This is a way to access applications without using the Graphical User Interface — the way most humans do, clicking and tapping on screens. The API allows you to use the raw data without the unnecessary baggage of a website or an app. Virtually every app has an API and it's typically used by other apps for scrutinizing data or embedding content. The interactions look like the short example at the beginning of this post. Soundcloud's API is not too bad — while you can use it to make programs to upload and delete tracks from connected accounts, it doesn't allow you to replace them.

Of course, this was not going to stop me. Luckily there's a way to make almost anything on the Internet bend to your will — using browser automation. Almost all software makers, and probably Soundcloud themselves as well, use browser automation to test parts of their application. If the code changes, automated tests are run against it to see if the changes broke any other parts of the application by accident. Because of this, browsers have all kinds of special hooks built into them that allows us to mimic anything that real people would do — clicking, typing, navigating, pressing buttons^{2You need a special version of the browser. The most common browser options are Selenium and PhantomJS. For this project, I've used Webdriver.io as a scripting framework to control them.}.

It's much like a string puppet, if you know how to use it. The details took me quite some time to figure out, but as you can see, the process works well (this is 100% automated, ghost-in-the-machine style stuff — I'm not touching anything):

You can recreate the exact steps you would take to replace the track yourself, by writing a script (literally, like for actors):

browser
// go to soundcloud.com
.url('https://soundcloud.com')
// chill for half a second
.pause(500)
// locate login button
.element('.loginButton')
// click on it
.click(‘.loginButton’)
// wait until login ('modal') window pops up
.waitForVisible('.modal__modal', 7000)
// double check if this is true
.isVisibleWithinViewport('.modal__modal')
// enter username...
.setValue('[name=username]', 'rawfare')
.click('[title=Continue]')
.pause(500)
// enter password — btw there's a much smarter way 
// to handle logins, this is for homework 😛 
.keys('*************')
// press the enter key
.keys('Return') 
.pause(1500)
// now go to the address of the track
.url(trackUrl)
// wait for more button to appear and then click
.waitForVisible('.sc-button-more', defaultTimeout)
.click('.sc-button-more')
// now wait for and click the edit button...
.waitForVisible('.sc-button-edit', defaultTimeout)
.click('.sc-button-edit')
// wait until the 'modal' edit window appears
.waitForVisible('.soundEditTabs', defaultTimeout) 
// upload a new cover image
// (there are two imageChooser elements on the page.
// make sure to select the one in the pop-up, not the other one.)
.chooseFile('.soundEditTabs .imageChooser__fileInput', imagePath)
// 🎉👌
// the meat of the script — choosing a new sound file to 
// replace the track with
.chooseFile('.chooseFiles__input', filePath)
// while it's uploading, we proceed to focus on 
// the description field and change the update time
.click('.baseFields__description textarea')
// make sure we're at the end of the description field  
.keys('ArrowDown').keys('End')
// in real life this has to be repeated 20x, 
// the length of current time format ...
.keys('Backspace')
// now simulate typing the current time, 
// to showcase when we changed the track (UTC timezone)
.keys(currentTime)
// different way to click on ‘Save changes’, and also a 
// powerful example of how to do almost anything you 
// could within a given page by running an arbitrary
// function in it's context 🤓
.execute(() => {
  document
  .querySelector('[title="Save changes"]')
  .click()
})
// wait until the little notification thing shows up saying the track was updated successfully
.waitForVisible('.gritter-with-image', 360000, 1500)
.getText('.gritter-with-image')
.then( (text) => {
    // check that it says it was 
    // updated successfullly
    expect(text)
    .toContain('updated successfully')
})

At this point I had automated the process of replacing the track, but I still needed to decide which distorted version to take. For this, I needed a good way to access listener feedback. I'm also obsessed with that topic since the turntable moment. All artists face the same environment today: we are slaves to metrics of "engagement" — listens, play counts, follows, likes, sign-ups and so on. These metrics have a significant real world effect, influencing media attention, bookings and sponsorship opportunities. There are several analytics tools that help you track them, as well as so-called "like gates". Like gates typically offer access to a free download in return for a follow or repost. To me, they all come off as sleazy marketing tools. They feel inherently embarrassing, probably for the reason that they're ill-suited for any creative interpretation.

This is why I've built my own tracking engine called Songsling.io over the course of 2016. It powers all my interactive musical artworks. My aim was to build something that can turn online projects into tamagotchis — virtual pets — that need to be fed by the visible feedback of your audience. All the online metrics I can measure are then patched back in to control the artworks themselves. I've used it for the first time to present The Bomb EP, which gradually unlocked its tracks as more people listened to them. It also powers Pipo.

Since I needed this engine to be working for me non-stop, I had to set up my own cloud servers. That takes a bit to figure out, but it’s incredibly cheap and 100% worth it just for the learning experience alone. I highly encourage you to give it a shot, if you're even slightly inclined^{3Here's a discount for DigitalOcean. Alternatively, this post has a a good comparison of cloud hosting providers.}. What you get is the possibility to build up your own army of automated robots (“shell scripts”) that do exactly what you tell them, any time — also while you sleep. Automation is coming to affect us in a big way, and you should learn some of it to improve your creative output.

The robot assistants set up for Pipo have roughly these tasks:

• Every 5-10 minutes, check for the latest playback statistics
• Calculate a project score from the growth in the plays, etc. based on their assigned weights (different multipliers — plays on Youtube or Soundcloud are worth more than Facebook)
• The project score is then stored and checked for by another automated bot every 15-30 minutes. The bot determines whether a change is necessary.
• If a threshold has been reached, it spins up the necessary virtual computers (“containers”^{4I'm orchestrating this using a tool called Docker.}). This happens in mere seconds. One container runs the string-puppeted browser, and the other one the automation script — the series of steps needed to replace the track on Soundcloud.
• After they’re done with their assigned tasks, the virtual computers shut down automatically and patiently sleep until the next occassion.

That's basically it. Thanks to all of the above, Pipo is always changing. I hope it inspires you to try something similar. Go and mess with the Internet. For more Pipo, you can also listen and grab the sound file on his dedicated web shrine. The visuals & 3D assets are the work of DLGNCE, an amazing 3D artist I was fortunate to collaborate with for this release. Our music video went online after the first 1000 plays: