Replicating the 2013 Warmest 100

I’ve replicated the methods used by the Warmest 100 crew to check their results.

I managed to collect 15,588 total votes compared with their 17,800, so my list will be slightly less accurate, thought I’ve used a slightly more complex matching technique than what @flossinspace did, so that might help things a bit.

My Method

Firstly, I wrote a Python script to get a list of images (and some metadata, like creation time) tagged with “hottest100” from Instagram, using their API and the python-instagram library. I found 2,950 images. Then I used wget to download the images themselves, at standard resolution. I used the creation time to exclude any images created before voting opened, because a lot of them were for the 2012 list.

I wrote another script (in Python) to OCR the images using Tesseract, parallelised so it would run faster on my 8-core machine, using the multiprocessing library. This processed the images in about 5.5 minutes.

I then parsed the corpus of text through some data-cleansing scripts I wrote (in Python), to fix up some of the obvious mis-codings from the OCR process. Stuff like backquotes instead of apostrophes, L0fdy instead of Lordy, F1ume instead of Flume, that sort of thing. This gave me a file with 55,377 lines, many of which were blank, nonsense, or not votes.

Then I ran a matching and counting program I wrote in Python to tally up the results. I was looking for lines that were either identical, or a close enough match that they were obviously supposed to be identical. Basically this was compensating for the miscoding of the OCR process to find close matches. Lnrde, L0rde, 10rde, l5rde, etc. are all probably supposed to be Lorde, so they all get counted as the same thing. Otherwise I’d have to combine different matches manually, and that would be super-time consuming, not to mention boring.

My matching algorithm cascaded through matching methods from a fast dictionary exact match, through some simple partial matching methods, to the quite slow Nilsimsa locality sensitive hash used by @flossinspace. I tuned the algorithm to be conservative so I reduced the risk of over-counting things that weren’t actually the same, and I kept a tally of all the matches that were made to double check.

The matching program was also written to run in parallel on my multi-core machine using a simple map/reduce approach, which made a significant difference. The first non-parallel version of the program took 192 minutes to run, so a bit over 3 hours. In contrast, the parallel version completed in just over 18 minutes.

SPOILER WARNING

The list I got is below, so if you don’t want to see my results, don’t click through. You’ve been warned. I mean, honestly, what are you doing reading this post in the first place if you don’t want to see spoilers?

The final tallies concur, in some ways, with the Warmest 100, but there are some significant differences.

Now we wait for the big day to see how many we got right! Stats is fun!

My result concur with @flossinspace and the Warmest 100 guys for who will win, but my results differ from the Warmest 100 for number 3, 4 and 5, which surprised me. It’ll be fascinating to watch on the day and see which approach was more accurate.

I’ve including the vote tally so you can see how few votes separate many of the entries, and how many songs nearly made the top 100 but didn’t quite.

JPW Rank Votes Song
1 397 Vance Joy – Riptide
2 367 Arctic Monkeys – Do I Wanna Know?
3 305 Violent Soho – Covered In Chrome
4 288 Flume & Chet Faker – Drop The Game
5 220 Arctic Monkeys – Why’d You Only Call Me When You’re High?
6 210 James Blake – Retrograde
7 196 Rufus – Take Me
8 187 Lorde – Royals
9 182 Foals – My Number
10 180 Kanye West – Black Skinhead
11 173 London Grammar – Strong
12 165 Daft Punk – Get Lucky
13 164 Matt Corby – Resolution
14 158 Rufus – Desert Night
15 157 Thundamentals – Smiles Don’t Lie
16 147 Arctic Monkeys – Arabella
17 144 Touch Sensitive – Pizza Guy
18 139 Vampire Weekend – Step
19 137 Preatures, The – Is This How You Feel?
20 137 Haim – The Wire
21 129 Lorde – Team
22 128 Lana Del Rey – Young And Beautiful
23 120 Grouplove – Ways To Go
24 120 Childish Gambino – 3005
25 117 Chvrches – Gun
26 113 Bloc Party – Ratchet
27 110 Chvrches – Recover
28 109 Vampire Weekend – Diane Young
29 109 Lorde – Tennis Court
30 109 Haim – Falling
31 109 Cloud Control – Scar
32 102 Flight Facilities – Stand Still {ft. Micky Green}
33 99 Arcade Fire – Reflektor
34 98 Sticky Fingers – Australia Street
35 97 Amity Affliction, The – Born To Die
36 95 Goldroom – Embrace
37 91 Dustin Tebbutt – The Breach
38 90 London Grammar – Hey Now
39 88 Panama – Always
40 88 Kite String Tangle, The – Given The Chance
41 86 Max Frost – White Lies
42 84 Daft Punk – Lose Yourself To Dance
43 83 Remi – Sangria
44 82 Chet Faker – Melt {ft. Kilo Kish}
45 81 Kanye West – Bound 2
46 78 Robert Delong – Global Concepts
47 76 Kingswood – Ohio
48 75 Andy Bull – Keep On Running
49 72 Mikhael Paskalev – I Spy
50 71 Empire Of The Sun – Alive
51 70 Big Scary – Luck Now
52 66 Illy – Youngbloods {ft. Ahren Stringer}
53 66 Daft Punk – Instant Crush
54 65 St Lucia – Elevate
55 65 Ms Mr – Fantasy
56 64 Chvrches – Lies
57 63 Rudimental – Free {ft. Emeli Sande}
58 62 Disclosure – When A Fire Starts To Burn
59 61 National, The – Graceless
60 61 Disclosure – You & Me {flume Remix}
61 61 British India – Summer Forgive Me
62 60 Dune Rats – Red Light, Green Light
63 59 Vampire Weekend – Unbelievers
64 59 Cold War Kids – Miracle Mile
65 59 Arcade Fire – Afterlife
66 58 Cloud Control – Dojo Rising
67 58 Andy Bull – Baby I Am Nobody Now
68 57 Kwes – 36
69 57 Danny Brown – Dip
70 56 Safia – Listen To Soul, Listen To Blues
71 56 Kanye West – Blood On The Leaves
72 55 Jake Bugg – What Doesn’t Kill You
73 55 Haim – If I Could Change Your Mind
74 55 Daft Punk – Doin’ It Right
75 55 Bring Me The Horizon – Sleepwalking
76 53 Wombats, The – Your Body Is A Weapon
77 53 Boy & Bear – Harlequin Dream
78 52 Illy – Ausmusic Month Medley {like A Version}
79 52 Bring Me The Horizon – Shadow Moses
80 51 King Krule – Easy Easy
81 51 Fidlar – No Waves
82 50 Horrorshow – Dead Star Shine
83 50 Disclosure – F For You
84 50 Boy & Bear – Southern Sun
85 48 Queens Of The Stone Age – Kalopsia
86 48 Cloud Control – Promises
87 47 Jackie Onassis – Smoke Trails
88 46 Busta Rhymes – Thank You {ft. Q-tip, Kanye West & Lil Wayne}
89 45 San Cisco – Get Lucky {like A Version}
90 45 Rufus – Tonight
91 45 Daughter – Youth
92 45 Broods – Bridges
93 44 Josh Pyke – Leeward Side
94 44 Bliss N Eso – Act Your Age
95 43 Rudimental – Waiting All Night {ft. Ella Eyre}
96 42 Jungle Giants, The – Skin To Bone
97 41 Mr Little Jeans – Oh Sailor
98 41 John Newman – Love Me Again
99 40 Ta-ku – I Miss You
100 40 Pez – The Game
101 40 Dillon Francis – Without You {ft. Totally Enormous Extinct Dinosaurs}
102 40 Blink-182 – When I Was Young
103 40 Alunageorge – Attracting Flies
104 40 Allday – Claude Monet
105 39 John Butler Trio – Only One
106 39 Frightened Rabbit – The Woodpile
107 38 Violent Soho – In The Aisle
108 38 Clubfeet – Everything You Wanted
109 37 Northlane – Quantum Flux
110 37 Major Lazer – Jessica {ft. Ezra Koenig Of Vampire Weekend}
111 37 Kanye West – New Slaves
112 37 Gesaffelstein – Pursuit
Bookmark the permalink.

One Comment

  1. Pingback: Booting out the Warmest 100 | chrisjrn's site

Comments are closed