Analytics update Feb 2022
Analytics & transparency are key components of Vanga. We understand that you care about your customer experience and in order to trust the algorithms are doing a good job - you need to monitor their performance.
Because Post-purchase offers happen inside the sandbox of the Shopify checkout, so far we've been limited to what we can do with our tracking capabilities. For the past month, we've been working on a major analytics upgrade in order to have much more reliable data. Now that Vanga is out of beta, it's crucial that our stats are true.
By doing so, we've found some major issues with our reporting, which are now fixed - but the analytics you see will change from the 25th of January going forward:
Improved accuracy of the number of shown offers
The most frequent issue that merchants are reporting, is that Vanga won't show post-purchase offers to all of their customers. While this might have multiple reasons ( for example ineligible payment methods ), we haven't been tracking a portion of the customers who see a post-purchase offer and close the page without taking any action. We now have much more visibility on the actions that your customers take when they see an upsell offer and as a result, you will see an increase in the number of shown offers.
Improved accuracy of offer conversion rates
With better accuracy on the number of shown offers, comes better upsell offer conversion rates. As you'll see more offers being shown, you'll see a lower conversion rate from now on. Keep in mind your conversion rate hasn't decreased - the old metrics were just higher than they should've been.
Improved accuracy of manual offers revenue
Manual offers that have set a special upsell discount haven't been taking the discount into consideration when showing upsell revenue. We've now fixed that and from now on manual offer revenue will take discounts into consideration, however, if you use discounts you will see a lower upsell revenue from now on. Keep in mind your revenue hasn't decreased - the old metrics were just higher than they should've been.
Improved accuracy of upsell revenue when adding taxes
Taxes calculated at checkout haven't been taken into consideration when showing upsell revenue. We've now fixed that and from now on taxes calculated at checkout will be added to your upsell revenue stats, however, if you calculate taxes at checkout you will see a higher upsell revenue from now on. Keep in mind your revenue hasn't increased - the old metrics were just lower than they should've been.
In conclusion
We understand this is really bad news. Making data-informed decisions is heavily dependent on the accuracy of the data, therefore, having such major deviations is unacceptable. The reality is we f'd up - we knew there's room for improvement, however, we couldn't fix it sooner. What I can promise you is that those data deviations were not ill-intended and we're determined to be as accurate and as transparent as possible. We'll keep working on adding more stats and visibility on customer UX and upsell performance.
Currently, we're confident our stats are accurate and you can trust them to make decisions. We'll keep monitoring for issues. If something feels off - please let us know ASAP - we'll look into it.