Like a lot of other commenters here, I think the OP didn't run an mobile app install ad which is meant exactly for this scenario - which would show up only on mobile inside newsfeed.
OP - would love to help out and dig in further. Can you email me (I'm sriramk@fb.com)?
I'm not sure what makes the OP think that you can run a conclusive experiment with a $20 ad budget… you're not going to get any conclusive numbers out of that. With click-through-rate and install-rate both typically in the low single digits, you're likely to get, if you're lucky, a handful of install for a thousand impression. In order to have a sample size big enough to make conclusive statements like this, you'd need at least 500 installs - ie. an ad budget of $1,000 for each network.
Disclaimer: I work at Chartboost, now the largest mobile gaming ad network.
I wish more people would point this out everytime someone makes a claim that "x ads don't work" and I read the article and the punchline is they spent $5 over 2 months.
That may be true, but I did get a noticeable increase in install rate with the AdMob campaign, with the same budget.
With a bigger budget, I might have been able to get more of a boost from Facebook, but I'm an indie developer and I really don't have the time or budget to spend hours and hundreds of $ tweaking a campaign.
I think you're missing my point. I wasn't trying to say that the ROI would be better on Facebook with a bigger budget (though, I guess anything is greater than 0%). I'm saying that your budget is too small to make the numbers you're seeing anything else than statistically insignificant luck.
Imagine taking a coin toss and trying to figure out whether heads or tails came up more often, but only tossing the coin three times. Imagine you get HHT, and conclude that head is a better bet than tails.
This is essentially what you're doing, by concluding that one option is better than the other, based on a single-digit install sample size.
Now, I'm not saying you should be spending massive ad budgets, and spending a lot of time tweaking your campaigns, I'm merely pointing out that you don't have enough raw data to make conclusive judgements about the effectiveness of these two networks.
Being a major player in this industry, I can tell you that the numbers I'm seeing seem to indicate that Facebook (and Chartboost) are in fact much more effective ways to buy installs for mobile games than AdMob, with a properly set up campaign. Feel free to contact me directly if you'd like more insight regarding this.
Why did you buy desktop ads to try to get mobile installs? Of course you're going to get a ton of impressions and hardly any actual installs: no one browses Facebook's desktop site via their android phone, and people on desktop aren't looking to install an app.
Also, your Google ad is significantly better looking than your Facebook ad, and is also much more likely to be seen while people are on their mobile phones.
Not to mention the complete lack of conversion tracking. He's measuring success of the mobile app ad by monitoring game activity WoW which is laughably unscientific.
Facebook supports conversion pixels as well, so conversions through his signup flow could have been measured. Instead we just get a vague disappointment in WoW signups
Developer here: you're right that it's totally unscientific. I never really set out to do a "scientific" comparison, because being an indie developer, I really don't have the time or budget to spend hundreds of dollars to find out if something works or not.
So I spent around the same time setting up the Facebook campaign as the AdMob one (about an hour each, including the time to make the graphic), and for the time/$ spend, the AdMob campaign trounced the FB one.
In terms of conversion tracking, it was quite unscientific, true. But basically, I can track the number of new players of the game per day. Normally I get a pretty steady flow just from the Play Store. With the FB campaign running, there was no significant uptick in signups, but with the AdMob one running, the uptick in signups was immediately clear.
I didn't actually know it's possible to target mobile only with FB campaigns, and so I'll definitely be giving that a go.
Anyway, my point with the post was just that for someone on a tight budget, with not a lot of mental bandwidth to spare for tracking ad conversions, AdMob was just way better value for me.
Conversion tracking is definitely the norm in the app-install world.
However what the OP did is sort of life-time-value (LTV) analysis which in the long run might give a better indicator on return-on-ad-spend. Post-install tracking is available now via various 3rd party tracking solutions.
It does involve more up-front work on an app developer - i.e. find all the important metrics in the app that is relevant (sign-ups, various activities like virtual good purchases etc...), and track these metrics over time.
There are quite a few flaws in the OP which most of the comments here have captured, and provided remedies to fix them, so I won't repeat them here. I look forward to the follow up here.
Disclaimer: an ex-Admob engineer + currently working in a mobile ad startup.
It's hard to tell from the included screenshots, but both based on the dimensions of the Facebook Ad (it looks like a "Right Hand Rail" ad and not one that run in the newsfeed) and the fact that a Mobile App Install statistic was not provided, I think its fairly likely that the Facebook Ad in question might not have been the Mobile App Install [1] format. That type of ad can be targeted only to people who are consuming Facebook via the Facebook client for your desired Mobile OS (in this case Android) and the call to action will be direct install. It will also indicate precisely how many install occurred via the Mobile App Install conversion stat. The add in question would have been a link to the play store via the web -- even if the user is not on a mobile device.
If I am correct (I can't be sure given the article), the results are not surprising. A properly run Mobile App Install Campaign can yield sub $2 installs depending on the app being advertised, but that assumes requires some experience running online ad campaigns.
I think that's uncharitable. What I, as a non-expert take away is that online ads are complicated and counterintuitive and that even seemingly conservative strategies ("buy from Facebook") can fail badly. The appropriate tale treatment for problems like that is blog posts like this one and the discussion they produce.
Front page of HN is 10k views last time I got it, definitely a big win for him.
That said, the results are hardly surprising. I showed AdMob ads in a game once and they paid almost nothing compared to other networks. They have a huge glut of apps showing ads for almost nothing.
So I'm the developer here. I've been posted on HN before and while HN posts land lots of eyeballs on my website, there's been no noticeable uptick in game installs at all :)
From experience spending extremely large amounts of money: If done right Facebook will win every time. Use targeted mobile app install campaigns, Facebook's bread-and-butter.
Your main problem is going to be that the game is sci-fi themed. They are very hard to marketing. If any perspective developers are reading this, don't make sci-fi themed mobile games.
I'm not surprised that facebook ad is a complete waste of money. First, lots of people block ads in facebook newsfeed. Second, who would click on ad of a game when people are busy posting baby's pictures?
There was no conversion tracking for the Facebook ads? If I ran this campaign I would be mainly interested in how much each conversion is worth to me and what each one cost.
tl;dr: Comparing conversion rates (a metric which is usually in range of 1-3%) using a sample size of n = 60 clicks, by just looking at the total number of registered users.
Even if you had amazingly high conversion rates, with sample of that size, the total number of conversions could be just about anything. Simple variance.
Well done on releasing the game with good reviews though. That's a great achievement. Just spend a bit of time just thinking about advertising :)
Like a lot of other commenters here, I think the OP didn't run an mobile app install ad which is meant exactly for this scenario - which would show up only on mobile inside newsfeed.
OP - would love to help out and dig in further. Can you email me (I'm sriramk@fb.com)?