🤔 Challenge: How can a project management tool increase its speed of experimentation without relying on engineering help?
đź’ˇ Solution: Layering Chameleon onto their existing tech-stack to validate hypotheses quickly and run experiments
🏆 Outcome: An 8% increase in activation rates and 3X the speed of experimentation
Shortcut is project management software with over 20,000 active organizations using their tool. Founded in 2016, Shortcut has over 100 employees based around the world – and a fast-moving Product team at the heart of its growth. As Shortcut grew, its Product team had big ideas, strong hypotheses, and a ton of energy to match. But one thing was slowing them down in particular: being able to independently test those growth initiatives…Â
Building an experimental growth stack
Meet Dana Calderone, Senior Growth PM at Shortcut. She joined the Product team in 2020, shortly after Shortcut’s $25 million Series B Funding – and things were going well, as Dana puts it:
We had laid the groundwork for a pretty great product and we had a good signup and onboarding experience.
The Shortcut Product team had a solid tech stack already. They were using Amplitude for product analytics, Looker as their Business Intelligence (BI) tool, and Pendo for in-app messaging and further product analytics. And while Shortcut’s analytics stack continued to support them as they matured and grew, one piece of the puzzle was missing for them. With all this product data, they had built up a backlog of experiments to run – but were unable to independently test them without engineering help.Â
So the team began to look for a tool that would complement their existing stack and enable them to test hypotheses quickly. And Chameleon fit the bill with its feature offering of native A/B testing, advanced element selection, and two-way integration with Amplitude (data source and data destination). Shortcut chose to keep Pendo for product analytics but transitioned to Chameleon for in-product communication.
Chameleon enabled us to move quickly. Being able to go in and easily create new experiences in the product was a big deal for us. I can now go from one experiment a month to three experiments a month, or more!
Boosting user activation with Microsurveys
With Chameleon’s Growth plan under their belt, the Shortcut team dug into their experiment backlog right away. One of the first areas they examined was their signup flow. Something strange was happening with their signup conversion rates; half of their users were getting stuck at the first step, filling in their company name, and Dana’s team didn’t know why.
So they asked those stuck users what was going on. Shortcut quickly launched a one-question, drop-down Chameleon Microsurvey asking why users weren’t moving forward, with the following answer options:
I don’t have a company name
I’m not sure about a company name just yet
I’m trying to join my teammates in Shortcut
I’m not ready to move forward
Other (with room for comments)
This Microsurvey with a timed delay was triggered by on-page elements, to collect data from the right audience. Users who were moving through the signup flow with ease would not see the survey, so it wouldn’t intrude on the overall user experience.Â
This is why I really liked Chameleon, it would have taken forever to build something like this in-house.
When Dana got the data back, she discovered there were a large number of people who were already part of a larger organization or even using Shortcut for a side project. Asking for their company name was adding friction to the account creation process as users were looking to join colleagues, or simply didn’t have a company name!
With this known point of friction behind them, Shortcut went ahead and ran a winning A/B test to change “Company Name” to “Workspace”. The result?
We increased the conversion rate by 8% which is huge as that’s the very first step users take with Shortcut – all thanks to the data we were able to leverage with Chameleon
đź“š Read more about this experiment on the Shortcut blog đź“š
Increasing activation with A/B testing
Most Shortcut customers are making the switch from another product – the biggest hurdle for them is the fear of changing tools. But switching doesn’t have to be hard! Shortcut removed the common aches and pains of tool switching by creating an “import data” functionality. However, a chunk of users still weren’t using it, and those users were bouncing back out of the product, never to return.Â
Dana and her team worked on their hypothesis:
If we can reduce that mental hurdle of switching tools, and show users that importing is easy, we can reduce dropoff rates
Using a Chameleon modal shown to new users, Shortcut was able to showcase how easy it is to import data from another tool. Their Head of User Education put together a short video that is embedded within the modal. They then launched an A/B test with Chameleon to measure the impact of users seeing this modal.
Using Chameleon’s Amplitude integration, the Shortcut team can deeply analyze how this Experience affects users throughout their activation journey.
We do everything we can to both activate users in their early moments and guide them through the process. That's where Chameleon comes in, we use it wherever we can to leverage the in-product experience.
​​👀 Discover how Chameleon’s Amplitude integration works ​​👀
A shortcut to base-wide announcements
Shortcut hasn’t always been “Shortcut”. Pre-2021, they were known as Clubhouse, which caused a surge in new users looking to join a live roundtable, not manage their projects!
With a change as monumental as a new company name, you need to confidently know that your active users are aware of what’s happening. Email works, but people can unsubscribe, or skim over it so easily. But with an in-product announcement targeted to all users, you can confidently know that anyone using your product will see this, no matter whether they go on to read more – they’ll know that a change has been made.
If it's something that I need to ensure that customers read and understand, having it in the product feels like a better guarantee than only having an email for that.
Using a Chameleon Modal targeted to all users, Dana and her team were able to confidently tell their users that Clubhouse was now Shortcut – reducing confusion and offering further reading on the name change by directing users to their blog announcement.
We would have been in a far worse position if we didn’t have a tool like Chameleon to communicate changes to the right people, at the right moments
While a name change is a once-in-a-company-lifetime event, Dana continues to use Chameleon for feature announcements and marketing events. Shortcut uses Chameleon Segments to deeply target the right audience, at the right time. Long-term users see trigger-based product announcements, new users see upsell opportunities. Â
Next steps with Chameleon
Increased activation, deeper product adoption, successful feature releases – and more to come! As Shortcut continues to use Chameleon, Dana is now encouraging the Design team to take advantage of Chameleon to collect feedback on existing and upcoming features.
We’ll keep using Chameleon, wherever we can, to leverage the in-product experience