status

How selection bias will skew your activity mailing results

Here’s an interesting thing that I came across when evaluating a mailing for a client that I need to share.

Fallacy01

The observation

So what we’re looking at here is the revenue for a given group of customers from 06/2014 to 12/2014. (Note that the data presented here is completely fictional recreated in a spreadsheet, but follows patterns similar to the original data. See below for more details.)

This group of customers had been selected due to its inactivity as defined by the revenue in 06/2014 being smaller then a certain threshold (again, the actual selection was a lot more refined that is described in this model).

The mailing was sent out on 01/07/2014 to the group of customers. So we’re looking at a total revenue of $46,837 in 06/2014 and a subsequent jump in revenue to $271,950 in 07/2014.

At first sight, that seems to be great news – the mailing worked, revenue increased drastically, everything fine.

However, when I looked at preceding months, the following pattern emerged:

Fallacy02b

So our group of customers had a high revenue 01/2014 – 05/2014.  Then in 06/2014 – the month that was used to determine whether or not a customer was active – the revenue suddenly drops by about 80%, only to be back at the original level the following months.

Now this seems rather odd. In fact, it looks a lot like there’s something wrong with the analysis.

But as it turns out, it’s actually perfectly correct. What we’re observing is due to what I call the selection bias.

(more…)

11
Nov 2015
POSTED BY Sebastian
POSTED IN

Other

DISCUSSION No Comments
status

How to tell if your results are significant – a practical guide

Marketers frequently face a situation like this: In a survey it is found that 57% of women prefer product A, while 60% of men prefer product B.

In this article I will show how marketers – using only simple statistical analysis tools available in Microsoft Excel – can quickly and easily decide whether or not they can draw meaningful conclusions from such a result, or whether they may be making fatal mistakes by interpreting random noise as valid data.

stats mofo

Marketers frequently face a situation like this: In a survey it is found that 57% of women prefer product A, while 60% of men prefer product B.

Some marketers will just go “Great, statistics prove that women prefer product A, and men prefer product B. We’ll market product A to women then and product B to men.”.

But is this really always a valid conclusion? Couldn’t it also be that the difference is purely coincidental? After all, we haven’t asked all people, but only a subset of people: those participating in our survey. So maybe if we took another sample, and asked different people, the results would be different? May well be!

Statistics to the rescue!

As is often the case, statistics can provide a solution. Before delving into the details, let’s look at another, more formalized example. Dice!

Suppose we take two dice, and we want to know if one of them is loaded, i.e., we want to know if one of the dice yields better values than the other. Let’s start by throwing them 10 times each. This is what the results may look like:

Example 10 dice

Well. What do we get? Let’s look at the mean value for each die. As a reminder, if the two dice were fair dice, there would be an equal likelihood of one in six for each number to turn up. More formally, the expected value would be 3.5 (=1/6*1+1/6*2+…+1/6*6).

So what do we get for our dice? For die 1, the mean is (4+4+4+…+3)/10 = 4.40, the mean for die 2 is (5+2+1+…+1)/10 = 3.20.

So is die 1 better than die 2? Well the average is higher, of course, but as you will intuitively suspect, 10 throws is quite a small number of throws to draw any meaningful conclusions.

(more…)

25
Nov 2014
POSTED BY Sebastian
POSTED IN

Other

DISCUSSION No Comments
status

Why the new Spotify UI makes no sense. A rant.

Spotify

A while back I’ve been whining about the lack of a decent music collection management Spotify. In the past months, Spotify has worked quite a bit on their UI and have changed their interface a lot. Sometimes for the better, sometimes for the worse. Bottom line however: the music collection management is still freaking useless.

Disclaimer: the following contains a lot of SHOUTING and CURSING. Sorry about that. But I’m really really passionate about my music player, and just hate to see that Spotify, which I otherwise LOVE, can’t seem to get their sh*t together.

So here we go: First, there’s the SONGS tab:

(more…)

27
Sep 2014
POSTED BY Sebastian
POSTED IN

Other

DISCUSSION No Comments
video
13
Mar 2014
POSTED BY Sebastian
POSTED IN

Other

DISCUSSION No Comments
status

Last.fm partners up with Spotify to offer on demand plays

Spotify Last.fmGreat news! Last.fm partners up with Spotify to again offer on-demand plays:

We’ve teamed up with Spotify to bring their entire catalogue, on demand, to the world’s leading music recommendation service.

About two years ago, Last.fm had dropped their on-demand plays, to “focus on the features that make Last.fm unique — scrobbling, personalised radio, and being the online home for your music taste”, or, put differently, CBS wanted to cut the costs so they decided to save the royalties.

Well I’m glad they’ve finally come up with this solution! The integration works pretty well, too: if you hit the play button on last.fm, Spotify is opened in the background to play the track. Smooth. What’s great, too, ist that on a “Top Tracks” list, such as the one below, the entire list is put in a Spotify-playlist.

PlayOnLastFM

08
Feb 2014
POSTED BY Sebastian
POSTED IN

Other

DISCUSSION No Comments
status

Spotiamp – it really whips the llama’s ass.

Found this little gem today:

Spotiamp

Spotiamp is a little program that lets you login to your Spotify account and play your playlists and more.

Apart from being nostalgic, it actually is a great player! The minimalistic design is timeless of course, but it is also a very fast player. The download is less than 500kb, it loads up in no time, and hardly uses any ressources. Plus, there’s any equalizer (which is dearly missed in Spotify).

In other words – it really whips the llama’s ass!

 

09
Jan 2014
POSTED BY Sebastian
POSTED IN

Other

DISCUSSION No Comments
image

Spotify free mobile arrives on iOS

Good news for all free users – Spotify free is now available for iOS as well:

Spotify free mobile App store

09
Jan 2014
POSTED BY Sebastian
POSTED IN

Other

DISCUSSION No Comments
status

Spotify Discovery now live

Spotify Discovery, announced back in December (and mentioned here) is now live for the web player:

“The Discover page is available from today on our brand new web player which is now available to all users. Just head over spotify.com on your computer to try it out. We’ve also started to gradually roll out the Discover page to users on our desktop and mobile apps. When we’re ready to update your account, you’ll receive a notification.” (source)

01
Jun 2013
POSTED BY Sebastian
POSTED IN

Other

DISCUSSION 1 Comment
status

Spotify Follow now live

Spotify just posted that Follow,  announced back in December is now live.

Now on Spotify, you can follow all the people who turn you on to the music you care about. Find out what friends and artists are listening to in real time, and check out the music that matters to the trendsetters in your life.

27
Mar 2013
POSTED BY Sebastian
POSTED IN

Other

DISCUSSION No Comments
status

Spotify introduces Discovery

The Spotify press event of Dec 6, 2012 is full of great feature announcements. Most importantly, Spotify has introduced their new Discovery page as well as their new Follow functionality.

As mentioned earlier, knowing what your friends listen to isn’t all that interesting. Much less does it help you discover new music. As Daniel Ek of Spotify puts it:

“Spotify today is great when you know what music you want to listen to – but not so great when you don’t.”

Check out the video of the press release (Spotify Discovery is mentioned about 16min into the vid), or read on for the highlights:

According to Ek, the biggest issue for Spotify users today is: “How can you help me figure out what I’m gonna listen to?”

The traditional approach to answering this question for online recommendations today is to present to the user a list of 500 items saying “Because you like this you might also like that.”

Spotify, on the other hand, wants to “make discovery even more seemless and intuitive” by making it “truely human” and “personal”:

“That’s not really how a friend would approach the problem. They would know what you like, and they would recommend you a few items instead – but with a ton of context.”

Hence Spotify wants to “give Discovery on Spotify the context that’s been missing”.

Whereas previously recommendations basically were just lists of songs, artist or albums (or just “cover art” as Ek puts it),

“Now in Spotify recommendations come with context for why they fit my tastes.”

The new Discovery page

He continues to present the new Discovery function on the web app:

WebPlayerDiscovery

The Discovery tab basically gives recommendations for artists or album based on your music taste.  However, there’s an extra layer of context. For example, it will give small artist biography for recommended artists. Or, it will tell you why an artist is suggested for you  (“You listen to Deadmau5. Check out Daft Punk.”). It will tell you when one of the artists you like releases a new album (“You might like this new release by Muse”), or reminds you of your old favorites (“Do you remember this song?”) – even based on your personal data such as your birthday.

Moreover, it integrates information such as upcoming concerts (from Songkick), reviews (from Pitchfork), or news about artists that I follow.

Basically, they take all the content that’s available via the Spotify app platform, and feed it into Spotify Discovery.

In summary, the Discovery page adds context and combines personalisation and recommendations to give users a more helpful and natural user experience.

Seems like quite a powerful tool to me if it’s done right. Looking forward to trying it out on my own!

Spotify Follow

Ek continues to argue that the best context that users can possibly get is a recommendation from a real person that you trust.

“Social has always been a very big part of what we do here at Spotify, but up until now finding people who can introduce you to music you cared about has been pretty hard.”

Even though “you had access to all your friends on Facebook”, but “there’s really only a handful of these guys that are amazing sources of music”. Hence Spotify has introduces a new follow functionality, which allows you to follow artists, music journalists, or companys in the music space. It works pretty much like facebook: once you follow someone, stuff they post will show up in your newsfeed.

The beauty of this really is that it a allows – like on Facebook – artist to communicate with their fans directly, be it sharing playlists or announcing new releases.

“Now artists can talk back and they engage their fans right where they already are, right when they are ready to try new music.”

Pretty cool stuff coming up I’d say!

13
Feb 2013
POSTED BY Sebastian
POSTED IN

Other

DISCUSSION 5 Comments