Skillz Inc. (NYSE:SKLZ) CEO Q1 2021 Earnings Conference Call May 4, 2021 5:00 PM ET
Stefan Gerhard - VP of Finance
Andrew Paradise - CEO
Casey Chafkin - Chief Revenue Officer
Scott Henry - CFO
Conference Call Participants
Jason Bazinet - Citi
Brian Fitzgerald - Wells Fargo
Michael Graham - Canaccord
Drew Crum - Stifel
Welcome to Skillz First Quarter 2021 Earnings Conference Call. At this time, all participants are in listen-only mode. A question-and-answer session will follow the formal presentation. Hosting today's call, we have Andrew Paradise, Chief Executive Officer, Casey Chafkin, Chief Revenue Officer, and Scott Henry, Chief Financial Officer of the company.
We hope you had a chance to read our press release and stockholder letter that we published earlier today, both of which are available on our Investor Relations website. We'll have brief introductory remarks and then dive into your questions.
Some of the management's comments today will be forward-looking statements within the meaning of the Federal Securities laws. Forward-looking statements, which are usually identified by the use of such words as will, should, or other similar phrases are subject to numerous risks and uncertainties that could cause actual results to differ materially from what we expect.
Therefore, you should exercise caution interpreting and relying on them. We refer you to the Company's SEC filings for more detailed discussion of the risks that could impact future operating results, and financial conditions.
During the call, management will discuss non-GAAP measures, which we believe can be useful in evaluating the Company's operating performance. These measures should not be considered in isolation, or as a substitute for our financial results prepared in accordance with GAAP. A reconciliation of these measures to the most directly comparable GAAP measure is available in our first quarter 2021 earnings release.
And with that, I'll turn the call over to Andrew.
Thank you, Stefan. Good afternoon, everyone. And thank you all for joining us today to discuss our first quarter 2021 results. We're incredibly proud of the strong results that we delivered throughout this quarter, and we're pleased to share some highlights.
But before digging into the details, I wanted to take a moment to thank the team that makes this all possible, especially our Skillzians, working to drive the future of interactive entertainment, and our developer partners, who are committed to bringing their unique craft to the world.
In Q1, we delivered strong performance across the business, achieving our 21st consecutive quarter of revenue growth. We generated 92% of revenue growth year-over-year, driven in large part by 81% growth in paying monthly active users.
Paying MAU describes our subscriber base, and is perhaps the most important metric to our business. Paying MAU is driven by growth in new paying users, which is a function of both the number of new installs or downloads, as well as the install to deposit rate. While new installs overall remain consistent with the prior year period, the quality of traffic and our ability to convert installs to payers increased substantially.
While monetizing non-paying users is an important feature opportunity, Skillz currently generates nearly all of its revenue from paying players. So, this nuance is crucial to understanding our growth strategy. We optimize our marketing spend to acquire paying users, not installs. This is very different from other mobile gaming companies that monetize via advertising, and just want to acquire the largest number of eyeballs possible.
Of course, we pay for each impression regardless of whether or not they convert to being a payer. So if we could optimize our marketing spend to the point where we only acquired paying users, we would do that. Throughout this quarter, we drove growth broadly across games and platforms, really hitting on all cylinders.
As we define the future of interactive entertainment, we continue to make game changing investments across our business to capture the hundreds of billions of dollars in global market opportunity ahead.
In terms of genre expansion, we're particularly excited about continuing to expand our platform from our first successful categories of games to all game categories, and are now seeing early validation of our multi-year investment in synchronous player technology. This next feature set will expand our content library to include games, ranging from first person shooters to real time strategy. We've opened up our synchronous technology to a small number of developer partners to test what we've built. We're collecting feedback from this initial group to refine the developer experience, to build product performance prior to launching this technology to our entire community later this year. We're encouraged by the early results. This is an area where we expect to increase investment in order to support the new types of games content, that our community will want to engage with.
Ultimately, we envision being able to support real time multiplayer content genres, such as Battle Royale. We believe synchronous game modes are an important part of the future of the industry, and a potential step function of growth for our business.
Another opportunity ahead of us, we're going to deepen and expand our partnership with brands. We see a future where every brand has a companion game that can be used as another way to connect with current fans and acquire new ones. We offer a trusted and mutually beneficial environment, where our user experience is enriching these brands, and these brands are able to more fully engage their audiences through our interactive content.
In fact, just last week, we officially launched and started accepting submissions for the NFL and developer challenge. There are multiple ways where we can partner with brands that align with their goals and power engagement their audience. We believe that not only this results in a better user experience for end users, but it also drives brand sponsored prizes, traffic, or both to our platform.
In terms of Android, we've been building out our Android footprint since 2018. But there's a huge opportunity in the Android market, and it's not monolithic. In Q1, we expanded our Android footprint growing revenue from Android users twice as fast as iOS.
On the international front, we remain on track to launch in India by the end of this year, with our patented and proprietary technology, which offers players a fair and trust experience, serving as a core competitive advantage that we bring to the opportunity. In preparation for our India launch, we're investing in localizing the product, including payments. This is critical to penetrating the most affluent gamers in India, we're taking a systematic approach to the process, so that can be replicated as we expand to other international markets in 2022, and beyond.
Lastly, we have a unique strategic asset with our deep first party player data. As we discussed during our Q4 results, user acquisition costs are still comparatively high versus prior years. Despite these near-term headwinds, we expect to maintain a long-term strategic advantage and user acquisition. Thanks to a broad appeal of our content across demographics, and our deep first-party player data.
This unique data set becomes even more valuable based on that announced changes to iOS 14.5 they've gone into effect, which result in the loss of data for ad networks. We believe there will be a strategic opportunity for Skillz to leverage our unique data set in the coming quarters.
Looking forward, our continued momentum that these many growth opportunities gives us the confidence to announce that we're raising 2021 annual revenue guidance to $375 million. That will represent a 63% year-over-year growth.
We're excited about the many growth opportunities ahead of us. We have a strong balance sheet with $613 million of cash. We have no debt as of the end of Q1. Hopefully, you've had a chance to read our stockholder letter, which contains more details on our Q1 performance.
But for now, thank you for being a part of our journey.
The line is live.
Pardon me, Davita.
Okay. Great. Go ahead, Andrew.
Before we begin the Q&A portion of the call, we have some news to share. Scott, our CFO, who joined us last year has announced his plans to retire. I think secretly, pretty sure Scott's retiring so he can devote more of his time to perfecting his skills games, particularly Pool Payday, which he's remarkably good at.
In all seriousness, we're really grateful to Scott for doing exactly what he set out to do, when he came on board. It was to use his experience to help take Skillz public. He will remain our CFO through June, and then he'll stand in an advisory capacity to ensure a smooth transition and continuity of operations, during our new CFOs onboarding.
That said, we're really pleased to announce Scott's successor. We've hired veteran finance and Investor Relations exec, Ian Lee to be our new CFO. He'll be joining us on June 21, coming from Airbnb, where he served as their Head of Investor Relations. Ian was on the core deal team for Airbnb's successful 2020 IPO. He's also led Investor Relations as a Senior Finance Executive at other world class technology firms, including Atlassian and Trulia, both of which he helped take public, as well as holding IR leadership roles at Hewlett Packard and Omnicom Group. We're looking forward to introducing Ian to all of you in the coming months.
In the meantime, Scott, who always tell us some time together, I wanted to say thank you for all of your efforts and contribution to Skillz.
With that, let me open the line for questions.
A - Stefan Gerhard
Great. Thank you, Andrew. So first question will come from Jason Bazinet.
Thanks. Can you hear me?
Yes, we can hear you. And let me just see if I can -- yep, we can hear you fine.
Okay, great. So I've so many questions that I've gotten from investors, but maybe I'll just start with one or two and then we can circle back. But, I guess the main question is that there have been some reports from bears that have focused on sort of slowing MAU growth and slowing installs. And I just wonder if you guys could sort of address that. And then tell us, you know, how those trends are faring and whether you think it's the right metric to focus on?
And the second question that I have from investors is, when you guys talk about paying MAUs and those trends, what portion of those paying MAUs are using sort of just exclusively bonus cash as opposed to maybe money that they put in the ecosystem themselves? Thanks.
Thanks for the question, Jason. We have our Chief Revenue Officer on the call, my colleague Casey Chafkin, who is responsible for distribution of the network. Casey, do you want to take that question about MAU?
Absolutely, Andrew. And Jason, thanks for the question. For a business that generates revenue from consumer payments, the most relevant KPI that everyone should be looking at is paying MAU or MAU. And this represents the number of consumers who are paying each month. And that metric was up 81% year-over-year, as Andrew mentioned.
It wouldn't make sense to evaluate a business like Netflix, using site visitors rather than subscribers. And that's really the same way that we think about it. But it doesn't make sense to look at Skillz using downloads or MAU, instead of paying MAU.
MAU, simply put, just isn't a metric that correlates with our business performance. So we reported it when we went public, because we hoped it would help people better understand our business. But, as it turns out, reporting MAU seems to have created more confusion than it has clarity.
I'll hit your second question, Jason, that…
Do you want to hit, or Casey should I jump in for the second. Second question is pretty easy to answer, which paying MAU is all depositors.
Understood. Thank you.
Stefan, we're going to get to the next question.
Yes. Next question is from Brian Fitzgerald. Go ahead, Brian.
Yes. I got you guys. So, thanks. We want to ask about the bridge between GMV and revenue, the puts and takes there in terms of the promotional work you do there? And how you kind of manage it? Are you expanding at a time of expanding promotional work right now? How do you manage that? How does the maturity of the game developers in the portfolio impact? How much promotional work you're doing? And then, talk about how you deploy that promotional activity relative to gamer activity, relative to gaming wins to ameliorate or manage any exposure there?
Maybe a little bit related to that, that first question. But we've got some inbounds on how much work [ph] is being done? And how are we sure that there's not a live go match to that? Sorry, for the long winded question.
No, no, it's okay. Thank you very much to the question, Brian. And maybe I can reframe slightly, and call it the elephant in the room. There's a short report out there alleging that we're not accounting for revenue correctly. And, whether or not that's accurate, and maybe more broadly on rev rec.
So maybe I can call my colleague, Scott, our CFO to kind of talk a little bit about this, and then we'll follow on.
Yeah, sure. Thanks, Andrew, and thank you for the question, Brian. Our revenue recognition policy is based on the same GAAP principles, as other publicly traded platform marketplace companies, that exhibit similar characteristics to Skillz, that accounting is driven by a company's business model, as I think we all know. And our customer is the game developer, right, not the end user.
So as you know, we've got a two sided marketplace business model, connecting game developers with players. And that's different from the traditional game publisher business model, where upselling games to players with a player, where the end user is actually the customer. None of this is new information, as this is all in our in our filings that we've made with the SEC, and that have been audited by Ernst and Young.
And maybe I can jump in here. The short reports calculated our net deposits minus developer profit share as between 19% and 47% of our GAAP revenue in 2019 and 2020. This is simply not true. The actual number is 1.5x to three times higher than the claims number. And you can see this in our SEC filings. You can calculate it. The actual number of 60%. And this is just an obvious error in the calculation in those reports, whether that error is intentional or not, well, it's anyone's guess.
Thank you for the question, Brian. Hopefully, that sort of answers. Let's go the next questions, Stefan.
Great. Next question will come from Michael Graham. Michael, can you hear us?
Yes. Thanks for taking the question. I wanted to ask one about [technical difficulty] I know that you continue to see really good progress with the amount of paying MAU. And that's got to be favorable for you to [technical difficulty] and just give us what unit economics are currently? And how you see them trending?
That's a great question, Michael. Let me throw that one over to Casey, our Chief Revenue Officer.
Happy to speak to that, Michael. As I think many of us have seen CPIs are up across the industry, and Skillz certainly hasn't been an exception to that. As Andrew talked about, both in the shareholder letter that we put out, as well as in the last quarter's earnings results, our LTVs have remained consistently strong. But, our revenue growth has actually outpaced the construction of our marketing infrastructure. And so this quarter, we completed the first step in addressing that, by building out our marketing leadership bench. We expect acquisition cost to come down in the back-half of 2021, as the investments we're making now begin to yield results.
All that said, our average three year LTV to UAC, for 2018 to 2020, the last three years was 3.8x, as reported at the end of 2020. We've actually never had a cohort stop paying. They never die. That's why we call them internally, our forever cohorts.
In fact, they realized five year LTV and I'll say that again, because I think it's important, realized five year LTV is 60% higher than our realized three year LTV. And so, that's obviously not factored into that 3.8x number. Maybe last -- I've seen some investors referencing credit card panel data as the means to approximate our LTV. We don't share our LTV for competitive reasons. But we do both. We do own both the dataset that's being referenced, as well as our historical actuals.
And I guess I just wanted to say, if you're using that, proceed with caution.
Thanks for the question, Michael. All right. Let's go the next one.
Great. I'll call on Drew Crum. Hi, Drew. Can you hear us?
I think he's muted.
Let's go to Andrew [Indiscernible]. Hi, Andrew, can you hear us?
I think you have to unmute him?
Nope. He has to unmute himself. Okay. While we go back to Michael Graham. Michael, did you have a follow-up question?
Well, I do now. I [Indiscernible] to ask a follow-up. I just, I know game genre diversification is an important topic for your model. So, can you give us an update sort of on what's going on there?
Yeah, I think that's a great question, Michael. So when we launched in 2014, we had over 100 games that I'd say most people never heard of. When we had our first hit, it actually accounted for literally, almost every penny of revenue. So seven years later, that game has grown considerably. It's still on the platform. But we have still [technical difficulty] concentration, whatever title as the head of the moment we tend to have revenue concentration in that title.
So in Q1, games on our top developers 42% of revenue. And I don't think it would be possible for me to predict which game will be biggest this year, like until you almost certainly it will be a double digit percentage revenue again. And I think that's really because of the hallmark of successful media businesses is such that users flocked to the best content. Skillz is no exception to that rule, really. We I would say embraced that and really looked at it as we want to have the best content surface easily to our users to ensure that we're getting people a highly engaging, highly entertaining product.
As we continue to build out the platform and really think about the innovation that we're building amongst this vibrant community of developers, I think it's certain that we're going to see new, bigger and more meaningful hits. And then beyond that, I'd say, our platform, it's very sticky for the developer. We have had less than a handful of developers ever leave the platform or about 90,000, of GMV. And it's, I think, really, because we are providing a superior monetization format for this type of art.
When you think about the next hit game, two years ago, we wouldn't have talked about Blackout Bingo!, but we saw based on the data signature of Blackout Bingo!, when it came on platform, that it had potential to be a top game on the platform. And we have this deep portfolio of incredible developers who are building high quality content. And while we can't speak to other people's future business results, I'd say, it's fair to say that we feel good about both the content pipeline and the revenue gains we’ve issued.
Okay. Thank you. Next question is from Casey [technical difficulty].
So two questions. I had one from an investor just asking about the revenue guidance raise that you guys did. I think this investor was expecting more given what's given the India launch and synchronous game technology that you guys are rolling out. So can you just maybe provide a little bit of color about how much of India and how much of the synchronous gameplay is sort of embedded in that in that revenue guide raise that you took?
And then the second one is, given this focus on sort of GAAP revenue versus sort of cash deposits, can you guys give net cash deposited at the end of the year and at the end of the quarter, so investors can sort of try and do their own counts? Or is that something that we'll have to wait to the K comes out or some other SEC filing? Thanks.
Sure. Let me hit the first one first. Thank you for the question, by the way, Jason. So the first question is whether or not revenue guidance includes international and sync, and the answer is no. Our revenue guidance raise does not include either. So, we've talked about, we as a company forecasts things that we understand, and that we can meet or beat expectations. We don't forecast new content, we don't forecast new geographies. Sync would include new content and the new geographies would include India.
On our second question on providing more cash deposit guidance, going forward, I think we're always struggling with the balance of disclosing as much as possible to enable the shareholders to understand the business, but also protecting the business and trying to really balance those two things.
When we look at our business on the net deposit basis after developer profit share, that would be another way of potentially looking at our gross profit, but that'd be a non-GAAP approach. So we've avoided that. The GAAP approach used by others is a gross profit approach, and it's considered a more accurate representation of the type of business we’re in, particularly for Skillz let's say GAAP gross profit provides a better indication of long-term profitability.
And it really helps people understand that we expect a material portion of the additional marketing dollars will get substituted with cash deposits over time. And because our gross profit margins are a lot higher than other companies, I think the way that GAAP lays out for a business, it makes more sense than looking at the positive basis.
Anything Scott or Casey wants to add?
Nothing to add there.
Next question comes from Brian Fitzgerald.
Thanks, guys. We wanted to maybe dig in a little bit on IDFA impacts has been popping up on everybody's quarterly calls, and maybe more specifically, how exposed do you guys feel you are to -- i.e. maybe not impacting you guys, but impacting audience networks that you guys use for gamer acquisition. And can you give us some color as to it was three or so a handful, half a handful of audience networks you use to acquire games and how you think about the distribution there?
Sure, that's a great question, Brian. IDFA actually landed last week. Let me turn it over the Casey, our Chief Revenue Officer to talk a little bit more about it. Casey is on mute.
Thanks, Andrew. As Andrew mentioned, for those of you who don't know IDFA took effect just about a week ago. And we haven't thus far seen any disruption from that policy change. Our teams did a really great job working with our partners to make sure that we were completely prepared for that.
And I think that as we watch IDFA filter through of course, it's still early, but as we watch the adoption of iOS 14, filtered through the system, we should expect to see any impact over a period of months, rather than weeks or days.
That said, I actually think IDFA represents an opportunity for us more so than a threat. We, as a platform that is both broad in content and appeals to a large demographic, we have a strategic advantage in our user acquisition against other companies that are relying on niche audiences.
The second thing that's worth pointing out is that, we have incredibly deep first-party player data as a result of the nature of our service, location services, stored user accounts, payment credentials. And these are all things that we have for our players, and most people don't. And so as Andrew mentioned in his opening remarks, we believe there's actually a strategic opportunity for us to leverage that data set in the coming quarters.
Alright. Who's up next?
Great. Next question comes from Andrew [Indiscernible]. Can you hear us?
Yeah, I can hear you guys. Hope you can hear me? Sorry, I joined a little late. [technical difficulty] different calls at the same time. So my apologies. Just a couple of questions. A nice job on raising [technical difficulty]. Could you talk about a couple of trends for us to, I guess, do a better job of modeling our [technical difficulty] model.
One, I know, I don't think you're going to disclose the cash deposit trends, but is there a range that [technical difficulty] numbers that you want [technical difficulty]. And how you see the cohorts playing out over the rest of the year?
And then, on the flip side, UA spend, marketing spend going forward, is it pedal the metal recipe? Or how should we think about that? Thank you.
Sure. Good questions. So let me hit the first one, which is just talking about our cohorts and help with the cohorts. So okay, a little bit of background just to make sure [technical difficulty] here. Okay, there we go. Great.
So, first one is just talking about cohorts. Let me talk a little bit about that. We've actually never seen a cohort actually die and stop paying. The users have been trained, and really to play and pay every day for seven days a week for years climbing now.
We've talked earlier on this call about the leverage of LTV over CAC over a three year period, which last is supposed to stay at 0.8x. We've actually generated 60% incremental beyond that on a five year LTV. But, having said that, we've actually never reached finality on a cohort in terms of LTV.
Casey, do you want to talk a little bit about marketing spend which is the second question?
Yep, happy to. And I think, when we forecast our marketing spend from a seasonality perspective, we don't see tremendous seasonality in our business. Our business is a daily hobby and a daily habit for our consumers throughout the year. We do see some seasonality in terms of the best times to acquire new users. And so, if you look at both the last two years, we've reduced our marketing spend from Q1 to Q2, and then have ramped it up again throughout the back-half of the year. But not Q1 is always our biggest marketing expenditure from a historical perspective. And so that's how we plan for the year going ahead.
Great. Can you go to the next question?
Great. Next question is from Drew Crum. Drew, can you hear us? I think Drew is trying to unmute.
You hear me okay?
Okay. All right. Sorry about that. [technical difficulty] So I guess, [technical difficulty] MAUs understanding the importance of the paying MAUs, how important of a metric is that to your developer community? Do you need more MAUs in order in to reduce the revenue concentration business?
So, let me break that apart into two questions. One is, first, how important is paying MAU to our developers? And then second is, do we need to increase paying MAU to reduce revenue concentration?
Casey, you want to jump in? I feel like you are…
I'm asking about MAUs, how important is the monthly active user to the developer community? It's the top of the funnel. And do you need more players on the platform to reduce the concentration?
Yeah, I'm happy to hit this again. Because, it's something that I feel very passionate about in terms of how we optimize the business. And the top things that our developers care about is the revenue that they're generating on the platform. MAU may be a vanity metric. But as I mentioned before, it's not actually correlated to our business performance.
In fact, the percent the portion of our MAU that are non-paying users actually cost us money, because we provide servers for them, we provide customer support for them. And so our non-paying MAU while they represent an opportunity from a revenue perspective for us in the future, are not a contributor to revenue today. And so, we don't optimize the business for it, no one on my team is optimizing the business for growing MAU.
What we're trying to do is optimize the number of people who are paying on our business, the equivalent of subscribers. And our developers are looking at what percentage of the traffic inside of their games is paying, because a higher percentage of their users paying is better for them. And this goes without saying it's because that maximizes their revenue, which is the top thing that they care about. And so we continue to optimize for that metric. And until we capitalize on the opportunity to monetize our non-paying MAU, we won't focus on growing it.
If I could add actually as you've seen the expansion from Q4 percentage paying, so we have a higher and higher percentage of paying MAU, which hopefully you understand the reason that's happening is because that's what we're focused on, is getting more and more of the MAU to pay.
You may you may ask the next question, but logically, which would be how far do we think this can go. We've actually commissioned three different studies, some of which are public, some of which are not, but I point you to the Frank and Magid study, which is the last one created by a -- this is a primary research firm for video games, that shows that actually, the willingness to pay for consumers is theoretically 40% of audience.
So we would expect to continue to expand from 17% payer upwards over the quarters to come. And we certainly are working hard on that, as we think. While we think and we know from talking to our developer community that that is what is most important.
But, for what it's worth, if someone wanted to value our business based on MAU, there are a number of ways that we can go out and acquire an abundance of low cost traffic. We can drive installs, we can chart a game if you'd like that doesn't actually result in the highest ROI. What you're looking for, again, is paying MAU. And so if somehow we -- and again, this is an opportunity for us, but if investors wanted to give us an unfair multiple, because by basing -- nu valuing our business off of MAU, I could imagine an argument for doing that.
Great. Thank you. Next question is from Andrew [Indiscernible].
Hi. I just wanted to follow-up on the sales and marketing spend. Thanks for that color so far. And then I got a follow-up after that for Andrew, I think. If we think about launching several new games this fall, or as you launch more games, should we think about marketing spend concentrated more on UA or on virtual currencies? How should that trend, I guess through the back-half of the year?
And then my follow-up question for Andrew, I see some [technical difficulty] in the background there. How's that game progressing on the platform? How's that game progressing? And then just broadly, when we could expect some of the games related to the NFL license? Thank you.
Sure. So, maybe I'll hit the Tetris and NFL piece first. The NFL developer challenge was actually opened about a week ago, and we announced that. We've seen a really significant response from the developer community, as to give you an actual number of developers, as we want to maximize how many are listening on this call and deciding to compete in that competition. But we expect it to be a healthy level of competition to win a partnership with NFL, and we're certainly seeing that.
So that's kind of one on NFL was moving right along. It's the challenge. So the signup portion of that challenge is still open for another month. So hopefully, if there are any developers listening, they'd love to jump in there and sign up. It's an opportunity I think, to win not just a branding deal, but also the marketing muscle the NFL, which is a multi-100 million fan reach. Truly phenomenal. So, that's on the NFL.
Separately on Tetris, Tetris is in soft launch in the platform. We've not done press yet with network, the creator of the latest version of Tetris. This is Tetris Clash that I think you're referring to. You can go and check out the title. We don't really comment on individual developers titles. I know that as revenue concentrates in the developer or title, we've questions from shareholders and investors about that company, but that's their business, where our job is to enable a better business for our partners.
So that's what I'll tell you about Tetris and where it is, and what is next in terms of that game scaling out.
I would add, actually, it is important for you to know, Tetris is not in the forecast for our business. So, when we're looking at revenue for 2021, when we're talking about raising guidance, that actually has a forecast of Tetris, contributing zero revenue to the company.
Thanks for the color, Andrew.
Absolutely. So, I don't think and I was just confirming, I don't believe that we provide guidance on our full year marketing spend. But you can see some details in our result process that we go through from an optimization perspective, with respect to engagement marketing.
So, we mentioned that with respect to user acquisition marketing, Q1 is frequently one of the best times to purchase digital media impressions. So we typically invest more in Q1 acquisition than we do in Q2. And this typically results in fewer new installs in the second quarter of each year relative to the first. And it's part of our consistent focus on maximizing return on marketing investment.
And you'll hear me say that a lot, because that's absolutely the most important part of everything that we focus on, is maximizing the ROI of each dollar.
With respect to engagement marketing, you can see that engagement marketing in Q1 is lower as a percentage of revenue than it was in Q4. We talked in Q4 about some of the experiments that we were running, focused on converting users into paying users. And the optimization that we engaged in Q1 was part of the process of maximizing the return on those investments. Whenever we launch investments, we're looking for the smallest sample size necessary to understand what the business impact of a change is going to be.
But we're constantly experimenting with an eye towards ROI, and our seventh value, which is frugality. And so we're going to continue running that optimization throughout the back-half of the year, as we look at the most effective ways to deploy both our acquisition dollars and our engagement marketing spend across the ecosystem.
Thank you. Appreciate it.
Absolutely. Great. Who’s next?
[Operator Instructions] Okay, great. Well, if there are no further questions, Andrew, I'll turn it over you to wrap it up.
No further questions, already. Are we sure? You want to go going once going twice, three times. Okay. Well, thank you everyone very much for joining our Q1, 2021 earnings call. We're really excited to be meeting with you again in quarter to talk about Q2. Good afternoon.