I’ve replicated the methods used by the Warmest 100 crew to check their results.
I managed to collect 15,588 total votes compared with their 17,800, so my list will be slightly less accurate, thought I’ve used a slightly more complex matching technique than what @flossinspace did, so that might help things a bit.
My Method
Firstly, I wrote a Python script to get a list of images (and some metadata, like creation time) tagged with “hottest100” from Instagram, using their API and the python-instagram library. I found 2,950 images. Then I used wget to download the images themselves, at standard resolution. I used the creation time to exclude any images created before voting opened, because a lot of them were for the 2012 list.
I wrote another script (in Python) to OCR the images using Tesseract, parallelised so it would run faster on my 8-core machine, using the multiprocessing library. This processed the images in about 5.5 minutes.
I then parsed the corpus of text through some data-cleansing scripts I wrote (in Python), to fix up some of the obvious mis-codings from the OCR process. Stuff like backquotes instead of apostrophes, L0fdy instead of Lordy, F1ume instead of Flume, that sort of thing. This gave me a file with 55,377 lines, many of which were blank, nonsense, or not votes.
Then I ran a matching and counting program I wrote in Python to tally up the results. I was looking for lines that were either identical, or a close enough match that they were obviously supposed to be identical. Basically this was compensating for the miscoding of the OCR process to find close matches. Lnrde, L0rde, 10rde, l5rde, etc. are all probably supposed to be Lorde, so they all get counted as the same thing. Otherwise I’d have to combine different matches manually, and that would be super-time consuming, not to mention boring.
My matching algorithm cascaded through matching methods from a fast dictionary exact match, through some simple partial matching methods, to the quite slow Nilsimsa locality sensitive hash used by @flossinspace. I tuned the algorithm to be conservative so I reduced the risk of over-counting things that weren’t actually the same, and I kept a tally of all the matches that were made to double check.
The matching program was also written to run in parallel on my multi-core machine using a simple map/reduce approach, which made a significant difference. The first non-parallel version of the program took 192 minutes to run, so a bit over 3 hours. In contrast, the parallel version completed in just over 18 minutes.
SPOILER WARNING
The list I got is below, so if you don’t want to see my results, don’t click through. You’ve been warned. I mean, honestly, what are you doing reading this post in the first place if you don’t want to see spoilers?
The final tallies concur, in some ways, with the Warmest 100, but there are some significant differences.
Now we wait for the big day to see how many we got right! Stats is fun!
My result concur with @flossinspace and the Warmest 100 guys for who will win, but my results differ from the Warmest 100 for number 3, 4 and 5, which surprised me. It’ll be fascinating to watch on the day and see which approach was more accurate.
I’ve including the vote tally so you can see how few votes separate many of the entries, and how many songs nearly made the top 100 but didn’t quite.
JPW Rank | Votes | Song |
1 | 397 | Vance Joy – Riptide |
2 | 367 | Arctic Monkeys – Do I Wanna Know? |
3 | 305 | Violent Soho – Covered In Chrome |
4 | 288 | Flume & Chet Faker – Drop The Game |
5 | 220 | Arctic Monkeys – Why’d You Only Call Me When You’re High? |
6 | 210 | James Blake – Retrograde |
7 | 196 | Rufus – Take Me |
8 | 187 | Lorde – Royals |
9 | 182 | Foals – My Number |
10 | 180 | Kanye West – Black Skinhead |
11 | 173 | London Grammar – Strong |
12 | 165 | Daft Punk – Get Lucky |
13 | 164 | Matt Corby – Resolution |
14 | 158 | Rufus – Desert Night |
15 | 157 | Thundamentals – Smiles Don’t Lie |
16 | 147 | Arctic Monkeys – Arabella |
17 | 144 | Touch Sensitive – Pizza Guy |
18 | 139 | Vampire Weekend – Step |
19 | 137 | Preatures, The – Is This How You Feel? |
20 | 137 | Haim – The Wire |
21 | 129 | Lorde – Team |
22 | 128 | Lana Del Rey – Young And Beautiful |
23 | 120 | Grouplove – Ways To Go |
24 | 120 | Childish Gambino – 3005 |
25 | 117 | Chvrches – Gun |
26 | 113 | Bloc Party – Ratchet |
27 | 110 | Chvrches – Recover |
28 | 109 | Vampire Weekend – Diane Young |
29 | 109 | Lorde – Tennis Court |
30 | 109 | Haim – Falling |
31 | 109 | Cloud Control – Scar |
32 | 102 | Flight Facilities – Stand Still {ft. Micky Green} |
33 | 99 | Arcade Fire – Reflektor |
34 | 98 | Sticky Fingers – Australia Street |
35 | 97 | Amity Affliction, The – Born To Die |
36 | 95 | Goldroom – Embrace |
37 | 91 | Dustin Tebbutt – The Breach |
38 | 90 | London Grammar – Hey Now |
39 | 88 | Panama – Always |
40 | 88 | Kite String Tangle, The – Given The Chance |
41 | 86 | Max Frost – White Lies |
42 | 84 | Daft Punk – Lose Yourself To Dance |
43 | 83 | Remi – Sangria |
44 | 82 | Chet Faker – Melt {ft. Kilo Kish} |
45 | 81 | Kanye West – Bound 2 |
46 | 78 | Robert Delong – Global Concepts |
47 | 76 | Kingswood – Ohio |
48 | 75 | Andy Bull – Keep On Running |
49 | 72 | Mikhael Paskalev – I Spy |
50 | 71 | Empire Of The Sun – Alive |
51 | 70 | Big Scary – Luck Now |
52 | 66 | Illy – Youngbloods {ft. Ahren Stringer} |
53 | 66 | Daft Punk – Instant Crush |
54 | 65 | St Lucia – Elevate |
55 | 65 | Ms Mr – Fantasy |
56 | 64 | Chvrches – Lies |
57 | 63 | Rudimental – Free {ft. Emeli Sande} |
58 | 62 | Disclosure – When A Fire Starts To Burn |
59 | 61 | National, The – Graceless |
60 | 61 | Disclosure – You & Me {flume Remix} |
61 | 61 | British India – Summer Forgive Me |
62 | 60 | Dune Rats – Red Light, Green Light |
63 | 59 | Vampire Weekend – Unbelievers |
64 | 59 | Cold War Kids – Miracle Mile |
65 | 59 | Arcade Fire – Afterlife |
66 | 58 | Cloud Control – Dojo Rising |
67 | 58 | Andy Bull – Baby I Am Nobody Now |
68 | 57 | Kwes – 36 |
69 | 57 | Danny Brown – Dip |
70 | 56 | Safia – Listen To Soul, Listen To Blues |
71 | 56 | Kanye West – Blood On The Leaves |
72 | 55 | Jake Bugg – What Doesn’t Kill You |
73 | 55 | Haim – If I Could Change Your Mind |
74 | 55 | Daft Punk – Doin’ It Right |
75 | 55 | Bring Me The Horizon – Sleepwalking |
76 | 53 | Wombats, The – Your Body Is A Weapon |
77 | 53 | Boy & Bear – Harlequin Dream |
78 | 52 | Illy – Ausmusic Month Medley {like A Version} |
79 | 52 | Bring Me The Horizon – Shadow Moses |
80 | 51 | King Krule – Easy Easy |
81 | 51 | Fidlar – No Waves |
82 | 50 | Horrorshow – Dead Star Shine |
83 | 50 | Disclosure – F For You |
84 | 50 | Boy & Bear – Southern Sun |
85 | 48 | Queens Of The Stone Age – Kalopsia |
86 | 48 | Cloud Control – Promises |
87 | 47 | Jackie Onassis – Smoke Trails |
88 | 46 | Busta Rhymes – Thank You {ft. Q-tip, Kanye West & Lil Wayne} |
89 | 45 | San Cisco – Get Lucky {like A Version} |
90 | 45 | Rufus – Tonight |
91 | 45 | Daughter – Youth |
92 | 45 | Broods – Bridges |
93 | 44 | Josh Pyke – Leeward Side |
94 | 44 | Bliss N Eso – Act Your Age |
95 | 43 | Rudimental – Waiting All Night {ft. Ella Eyre} |
96 | 42 | Jungle Giants, The – Skin To Bone |
97 | 41 | Mr Little Jeans – Oh Sailor |
98 | 41 | John Newman – Love Me Again |
99 | 40 | Ta-ku – I Miss You |
100 | 40 | Pez – The Game |
101 | 40 | Dillon Francis – Without You {ft. Totally Enormous Extinct Dinosaurs} |
102 | 40 | Blink-182 – When I Was Young |
103 | 40 | Alunageorge – Attracting Flies |
104 | 40 | Allday – Claude Monet |
105 | 39 | John Butler Trio – Only One |
106 | 39 | Frightened Rabbit – The Woodpile |
107 | 38 | Violent Soho – In The Aisle |
108 | 38 | Clubfeet – Everything You Wanted |
109 | 37 | Northlane – Quantum Flux |
110 | 37 | Major Lazer – Jessica {ft. Ezra Koenig Of Vampire Weekend} |
111 | 37 | Kanye West – New Slaves |
112 | 37 | Gesaffelstein – Pursuit |
Pingback: Booting out the Warmest 100 | chrisjrn's site