How to find more artists on Last.fm

When you've used Last.fm for a while you'll no doubt have found some friends, people with similar tastes or interesting genres (tags). Some of the music inside those libraries might be really good, but if you want to find the ones that you don't know yet it's a lot of work to find those.

In this post I'm going to show you how to make list of artists including the  information if you've played them before and include links to listen to the artists. For this you'll need access to a web server that works with PHP, that can be your hosting provider or your own local WAMP-server. Make sure it supports PHP 5+ since we'll be using the function simplexml_load_file. I'll make a separate post how to install a local server (don't worry, it's really easy).

Luckily there are a couple of methods in the Last.fm API that can help with finding artists and their information. Note that you'll need your own free API key for this (you can get one here). The methods that I'm using here don't require authentication so it'll be a lot easier getting the info.

• user.getTopArtists - This methods fetches the list of artists that somebody has been playing. You can get a list for different time periods (including overall) and set the number of results per page. This is convenient for restricting the number of things the script has to check, if it's too much then it'll take too long to finish and the server will timeout. Setting it to a maximum of 25 works best for standard servers.
• tag.getTopArtists - Almost the same as the previous method but without the time periods and with a limit of 1000 artists.
• artist.getInfo - This one does most of the heavy lifting: for each of the artists that are listed by the previous two methods it checks it for their information and (since we also add a username) how many times we've played if.

There are more methods that provide lists of artists based on charts, location or groups but for now we'll only use these.

Making a custom Now Playing tab in MusicBee

I use MusicBee a lot these days to play music. There's an option to display some data about the currently playing artist+track (artist picture, name, biography and the amount of my plays). I tried using the Last.fm artist.getinfo-method to get an XML-view of the data but that didn't really work for me. It lacked readability and some of the info that I wanted like how many times someone else had listened to this artist and some other little tweaks.

MusicBee AutoDJ - Creating a custom radio

Last.fm radio has an option to build a standard radio based on a maximum of 3 tags, however these are used as an AND-station: play music that is Rock AND Hip-Hop AND Opera. This will usually result in very few tracks since there isn't a lot of music like that.

It is however possible to create a radio that is build with much more options since the Last.fm uses RQL (Radio Query Language) and it looks like a long series of letters. Thankfully you don't have to know (another) tech language since there is an excellent web-interface that takes care of this:

ComboFM

(http://www.combofm.de)
When you first enter the site you only need to enter your (or anybody else') Last.fm username to start building your own custom radio. You can add artists, tags, specific peoples libraries or their friends libraries, personal radio (Library, Mix Radio, Neighbours, Recommendations and (all) Friends), just click on any you want to add to your current Query or enter the name of one that isn't shown. You can delete items from it by clicking on the [X] and drag'n'drop items to get a different sequence.

When you have multiple items in your query, you get an extra option that makes this work even better than Last.fm's default: you can set the option to AND (the tracks needs to be both items), OR (the track can be either item) and NOT (the track can not be in the following item).

Some examples of when this is useful: say you want to play your recommendations but only the ones that are tagged with electronic, or the library of that friend with excellent taste in Blues but not the tracks that you already know.

Note however that this works only at track-level not artist-level: So a track from an artist tagged as dark ambient would play if it isn't tagged as dark ambient itself. (http://www.last.fm/forum/21716/_/616299/_/14770039)

When you've added+sorted your items you then copy the link for Last.fm Software to Notepad and copy everything after "lastfm://", next you go to "stream tracks from:" in the AutoDJ, change it to custom and paste it in "lastfm://". Press Start DJ

Some examples:
folk OR cajun OR bluegrass NOT classic rock NOT female vocalists
will give the following link:
lastfm://rql/dGFnOiJmb2xrIiBvciB0YWc6ImNhanVuIiBvciB0YWc6ImJsdWVncmFzcyIgbm90IHRhZzoiY2xhc3NpYyByb2NrIiBub3QgdGFnOiJmZW1hbGUgdm9jYWxpc3RzIiBvcHQ6ZGlzY292ZXJ5fHRydWU=

Tag vandalism

Since tags for artists are added by the listeners there will be some people who find it funny to tag artists like Justin Bieber with "speedmetal" or "death metal", very "funny" until you want to listen to a tag like that. A workaround for this is to make a radio where the tracks need to be tagged with more versions:
(tag:"death metal" and tag:"brutal death metal")
(tag:"hip-hop" and tag:"rap")

You can add these parenthesis by using an encoder/decoder like http://rumkin.com/tools/compression/base64.php (webinterface), just copy everything after "rql/", paste into the decoder and you see the normal version of your radio. Make the changes you want, press "Encode to base64" and paste the resulting string in its proper place.

List of possible items

Name   Value   Description
user           [username]   personal station
library   [username]   a users library
loved   [username]   loved tracks of user. After Nov 17 2010 this station is not available anymore!
rec      [username]   recommendations for a user
neigh   [username]   neighbour radio of user
adv      [username]   Play a user's mix radio.
simart   "[artist name]"   Similiar artists of an artist. Keep the quotes around.
tag      [tag]   Global tag radio
group   "Group Name" or [groupid]   Group radio
opt:rep|   [0-1](default: 0.5)   Specifies the track repetition, means how long it will take until the track is played again
opt:mainstr|   [0-1] (default: 0.5)   Type of tracks played(obscure to popular)
opt:discovery|   [true|false] (default: false)   Discovery mode on/off

Note: if the radio uses 240+ characters it only works with the AutoDJ, Radio Stations in MB can't use links of this length.

Sources

Source code and bug tracker: https://github.com/tburny/combofm
Blog post with the list of possible items: http://burnysblog.blogspot.nl/2010/04/combo-station-urls.html
Last.fm forum thread about ComboFM: http://www.last.fm/forum/21716/_/616299/1
interesting info in that thread:
Discovery-mode is outdated (http://www.last.fm/forum/21716/_/616299/_/18505455)

How to export your Last.fm-data (to Spotify)

Standard tools for exporting Loved Tracks
* Spotibot (only loved tracks)
* Playlistify

These two sites offer an easy way of finding loved tracks based on giving them a username. Playlistify has had some outages but normally they're really stable.
I've tested both sites to see what results they give and it turned out that Playlistify gave more results back than Spotibot (20 tracks more of my Loved-list).

Playlistify also has an option for adding the url of an xspf-file but no option to upload one, not that handy since you can only download the file from Last.fm but not link to it.
The downside of using Playlistify however is that they have sound effects while searching that are really annoying if you're listening to music. Maybe there's a setting somewhere on the site but I couldn't find it.

Both services have a problem with searching for tracks if the number of loved-tracks is fairly large. So far anything under a thousand tracks on Last.fm gives good results, but when I tried a user with 5064 tracks loved and another with 7455 both came back as either not a valid username or zero results. It turned out that because I had enabled AdBlockPlus both sites somehow couldn't trigger the export from Last.fm.
The strange thing is that after I had disabled ABP both sites could only "see" the first 700 tracks from large lists.

Ivyshere
A better solution for long lists is to not export the list in Last.fm as a xspf-file but to choose for TSV as the format in the drop-down menu (it's right underneath where it says XSPF). The export takes some time (tested it on a list with 7455 tracks) but after that you have your list in a format that can be read by Calc (part of the free OpenOffice-suite) or in Excel. I prefer to use Calc since I can't find the setting in Excel to import the data as UTF-8 and you definitely want that otherwise artist and track names get maimed when they have characters like ë, é or ô.

The file will contain one column with the track name, one with the artist and one with the Last.fm-URL for that track.

Open your spreadsheet application of choice and import the data in the file (usually something like "Data" and then "Import"). While importing tell the application that the data is deliminated (split) by a tab (make sure that any option for comma is not selected) and click import. After that you have all your tracks information in front of you, make sure you see the three columns. Now you have your data in Excel you need switch the order (tracks and then artists) to artists first, before copying.
Now select the first track name at the top left, scroll all the way down to the last artist names and shift-click that name. Copy those to the clipboard (ctrl+v).

Next, open the following url in your browser: http://www.ivyishere.org/ and click on "import your playlist now", go to the copy+paste section. Click the field and paste (ctrl+v) the list. You can adjust the results so they only return the original versions (the reason why anyone would want the covers is beyond me) and the option to only return tracks that are available in your country (meh). Next click on upload and the site looks through the list if it can recognize the format. You then get asked whether you want it to look for them in the available tracks on Spotify, click yes (obviously) and it goes to work. After some time you get to see a picture with underneath it a text with "Click to paste to clipboard", do so.

Next, go to the Spotify application, open a new playlist, go to Edit and click on Paste. Now you have playlist with all your loved-tracks in Spotify.

An alternative for ivyishere.com is http://spotmysongs.com/ but that one is a bit more work. I'll test it later to see if it gets more results.

Finetuning: I've solved most of the issues with importing Last.fm's tsv-files into OpenOffice (rename extension to .csv => open in Calc => set character set to UTF-8 => set both artist and track column to "text" => good import) I'll write it out tonight in a journal with some screenshots.

Finetuning 3: Ivy gives different results depending on the time of day|how busy Spotify's servers are.

Exporting your complete Playhistory
Go to http://cs.joensuu.fi/~pviktor/lastscrape/ and download + install LastScrape (windows only). Unzip the file and start the program, enter your username and let it scrape Last.fm for your playhistory. This will take a lot of time if you've played a lot of music. When it's done you can close the program since you're not going to upload the data to libre.fm. Important: I've tested this with six other accounts and with half of them (including mine T__T) the scraping got stuck somewhere.
Inside the folder/directory where LastScrape is unzipped you'll find a text-file called "listens.txt". Follow these next steps:
* Open the textfile and select everything that's in it.
* Open a spreadsheet program like Excel or Calc and choose for "paste special", in the dialogue-box choose for "paste as unicode". That way the data will be pasted in three separate columns.
* Now you can choose to let the data be as is but you can also have it searched by historical order. You can do this by selecting the third column and change the order (make sure you let it expand the range so the artists and tracks are also re-ordered).
* Next, select the artists and the tracks. Keep in mind that Spotify has a limit of 10000 tracks per playlist, so it's best to select them in batches of 10K.
* Go to http://ivishere.org, click on "import your playlist now" => "copy+paste-section" => paste the tracks there. Let that search for the tracks on Spotify.
* After a while it has found the tracks that are available on Spotify, click on the button and they're copied to the clipboard.
* Go to a new playlist in Spotify and choose paste in the menu bar (don't know why you can't do that directly in the list, but oh well)
* Rinse and repeat for the other tracks.

iTunes (Mac) - import a XSPF playlist
xspf2iTunesMac
This utility imports the contents of a XSPF playlist file into a new iTunes playlist (in other words: it acts as a track resolver).
 Usage: Load a XSPF playlist file and choose a name for a new iTunes playlist. Each XSPF playlist entry, which already exists in your iTunes library, will then be duplicated into the new created iTunes playlist.

Why I'm not a Last.fm subscriber (I promise: not an angry rant)

[Quick Executive summary: I have weird listening habits.]

Someone asked me recently why I'm not a subscriber even though I spend (way too) much time on it. I've seen the topic come by a lot on the Forum recently and thought it would be handy to place my thoughts in a place that's accessible by all.

Let me start of by telling that CrybKeeper gave me a month subscription recently so my opinion is founded on some experience when I say I never (for now) want to be a subscriber again ^_^ But I want to thank him again for the opportunity to at least try.

Recent Visitors
The biggest disadvantage for me when I would be a subscriber is that I then would have a Recent Visitors-box on my profile and it makes me totally paranoid: "why were they here? why didn't they say anything? why would they come here? where did they come from and why?" and a whole lot more.

The strange thing is that I can get stuff I do want to know, like "what's the average number of people that come by in a month, how many repeat visits, from what country, what OS?" already from statcounter.com (also very handy if you want to see the amount of traffic a group generates). Somehow I feel better about anonymous stats. Too bad it doesn't support long URLs so I would know how many people read this journal for instance (hint, hint Last.fm).

Sadly, none of these things are available through Last.fm, not even for artists.

Reply Tracker
Next, that box would replace my Reply Tracker-box that is the result of the best (IMHO) greasemonkey-script for Last.fm: I can set the number of replies, blacklist threads and how many times it updates (currently once every 60 seconds). Lightyears ahead of the Last.fm-page.

Radio vs On-request
Third, I hate radio...correction, I hate radios where I'm not the DJ. If I'm not 100% in control of what gets played I don't listen to it. None of that "you've only listened to this artist once" for me, in that regard I'm a cruel A&R-manager: I know when an artist is swine or good within 30 seconds and there are a lot more of them that I have to review.

Playlists
Fourth, the playlists are a total waste on Last.fm: maximum 200 tracks? Try 10000 tracks on Spotify with the ability to sort that playlist based by artist or track name, album, length of track, album name, date added or who added the track if it's a collaborative playlist...

Also, Last.fm? No collaborative playlists, or an option to sort those lists into (sub-)folders.

That Black spot
Fifth (but minor) point, some of the people I know on Last.fm frown on being/becoming a subscriber because of all the changes. Me subscribing is like saying that I disagree with their protesting against those changes. It's the wrong version of the Fairness Doctrine at work: if you don't hate Last.fm then you must be a filibuster. I'd like to support but I don't want the hassle that comes with that.

The strange thing is that I don't care about those changes at all but I do dislike the fact that I would be treated differently from people in other regions if I wanted to use those services. Last.fm can't help it but neither can I.

Banning artists to get a somewhat working radio
Sixth (and major) point, I hate banning artists all of the time in order to get a radio that works the way I want it: play only new artists and throw in some genres I've never listened to before in the mix. Somehow, dismissing from my recommendations is not a big issue with me for some reason.
When I tried the radio as a subscriber I hated that I had to whack it into submission before it became something of use, but only after I had banned 2K+ artists.

Even the Recsplorer became a hated thing when it started to play artists/tracks I had dismissed in the Recommendations and then only played genres I already knew. There's thousands (if not more) of genres and I want to discover them: tejano, fado, industrial, alt-country, contemporary classical and death country all came into my library because I started to look for new genres and the system now somewhat follows. Maybe if it started to use the similar tags-option and I could weight them it would be better. Here's hoping for some updates.

Self limitation
Seventh, I like to be able to judge how far you can get listening to music without paying for it for a fixed fee. It makes me be a better reviewer for my group.

Spotify
Eight, I already had a paid (unrestricted) account on Spotify, that costs more then a subscription on Last.fm. I would like to say because I want to support artists but it was mostly because of their annoying radio-ads. I have no need for mobile stuff so a Premium account would have been overkill.

The amount of obscure artists that are available there means that even when Tomahawk becomes fully functional to me (it has some issues right now), I would still use that as a source for music. There is no way I could find friends that obscure who would have those files.

Conclusion
I like the combination of a lot of music-sites in combination with Last.fm and I'm not exclusive about where I play tracks from. To me Last.fm is good for the stats, since there are no sites anymore that have that (looks at the remains of thesixtyone.com), the groups and the music community that comes from that.

gr.
Alain

PS. Sorry about that wall of text.

Google Custom Search

There are a lot of places where you can find the artist that you want to play, but going through each and every one of them is very tedious. You could use the standard Google search but going through page after page with results that don't actually give you just the music is like watching paint dry. Google does offer some extra code that you can use (operators) but if you use them too much then there's a risk that you get blocked from doing more searches, yipes.

However there is a workaround for this without using code and you get to decide what sites are the only ones you want the results from. This is really great since it can include sites that are normally displayed after page five in the results (like filetram or filestube). To get this you only have to make a Google Custom Search-engine, add the (list of) website(s) that you want to get the results from, choose the design and you have your own search-engine. If you're using Firefox with the Add to Search Bar-add on, you can right-click the search-field and add it as the standard search-engine.

Examples
This is one I made that includes (among others) Grooveshark, WeAreHunted, Bandcamp, Soundcloud (who switched from http to https?). I didn't include Youtube, MySpace, Faceborg or Torrent-sites since those really mess up the results.

Find that Artist!

This is a temporary one for testing out the use of mp3blogs, these usually don't show up that well in the results since most of them have a very low pagerank. There's a maximum of 5000 sources/domains that you can add per profile. I'm not sure yet if that is per user-account or per search engine.
Find That Artist! (on mp3blogs)

The reason that the blog search is temporary is that maintaining the list is a bit wonky in the Google interface. For instance: there is no option for mass deletion and with more then 20 it gets very tedious. I'm solving this by starting a Google-doc spreadsheet, add tabs for the larger genres that will include a row for the blog-fields and a row for the subgenre-fields. That way I can make extra notes or sort by sub genre.

Next example:
Hip-Hop It's based on 68 blogs (PM me if you want the list) and one blocked blocked URL (http://www.africanhiphop.com/forum/). The rreason for the block is that the search results contained a lot of chatter instead of actual music.

This test version is called Rock (http://www.google.com/cse/home?cx=014209395896937011977:utmcptld120) but it only contains blogs with punk, garage, mod, freakbeat. As a test I did a search on the first artist mentioned by the last blog added, and saw that it didn't show up in the results. It turns out that although Google (owner of Blogspot) is actually changing the domain for me from, in this case, http://garagerocks.blogspot.com to http://garagerocks.blogspot.nl it is unable to show me search results for the nl-version. This is probably because they want to avoid duplicate search results. Thankfully you can easily compensate for that by doing a search+replace on the "blogspot.nl" to "blogpost.com".

Project HiveMusic

This project was started to combine multiple places of content into one engine that will provide people with the ability to explore new music.

This page will contain the information needed for the project.

==Local==
Import the listening habits of listeners (including myself) on Last.fm to a local MYSQL-database, all including dated/time played.
    recent tracks
    loved tracks
    banned tracks

   
==Echonest==
For each create a taste profile on Echonest (A non-commercial API key can have 1,000 taste profiles associated with it. A taste profile can have up to 100,000 items in it.)
    upload the unique artist names
    their most popular track name
    loved/banned-status
    date first played artist
    Tags for the artists

There can be up to 20 custom attributes per item. Attribute names are short strings (maximum length of 32 characters). Attribute values are strings, numbers or lists of strings.


==Similar==


==Maintenance==
When a user is not listening to music have the engine import the tracks that were added after the last scan (if ($nowplaying == "true"){start scanning}). Artists that are new for the profile are uploaded to Echonest with the same format as a newly created profile.

==Optional APIs==
Blogger/Blogspot has the option to search for all blogs and posts by user. These could be called for a list of posts that include mp3-posts for a recommended artist.

==Documentation==
[url=http://developer.echonest.com/docs/v4/catalog.html#taste-profile-api-methods]Taste Profile API Methods[/url]
Blogger/Blogspot API https://developers.google.com/blogger/docs/1.0/developers_guide_php

==Project documentation==
http://project-hivemusic.blogspot.com