An Honest Review of Pendo Analytics

Many SaaS businesses rate Pendo Analytics among the best in the industry.

There’s certainly a lot of buzz about their analytics in the onboarding world at the moment. We previously discussed them in our blog post about Pendo here.

The question is: do the analytics features offered by Pendo live up to the hype?

We had a play with some of their tools to save you the trouble of testing them out yourself.

TL;DR

• You can review Pendo’s analytics in their free trial, which displays various different graphs on the basis of dummy data.
• To replace the dummy data with real numbers from your SaaS company, you’ll need to install a JS snippet.
• Paths show you all the actions taken by users before or after a specific event.
• Funnels show you how many users completed each stage of a specific funnel that you pre-determined. Userpilot’s Goals feature is comparable, but shows a higher number of goals at once and requires fewer clicks to do so.
• Reports allow you to track page views, clicks, events, or guide views related to whichever feature you want. Compared to Pendo’s other analytics features, this feels like vanity data.
• Retention shows you the percentage of customers who stick around relative to the first time they used your product. The graph is visually impressive but lacks a qualitative explanation about why users are retained.
• NPS is not available on Pendo’s free trial; it requires you to pay. Userpilot includes NPS for free.
• In general, Userpilot is much more concerned that you interact with your customers directly via NPS and qualitative microsurveys than Pendo is.

First impressions of how Pendo measures user behavior

If you sign up for a trial with Pendo, you are asked whether you initially want to explore Behavior Analytics, Dashboards, or Guides.

Each of these three options has its own dedicated product tour. The product tours are not especially interactive, as they just take you from tooltip to tooltip, but they do show what the product does.

While Behavioral Analytics is self-explanatory, it’s worth quickly outlining exactly what Pendo means by their jargon for the other two options:

1) Dashboards is somewhat similar to Google Analytics, in that it gives you access to visitor browsing data like weekly visitors and unique visitors per hour.

2) Guides are Pendo’s version of experience flows, so groups of UI patterns work together to achieve one onboarding objective.

The value of dummy data

If you’re wondering where Pendo gets the user data from that it displays in the free trial, the answer is that this is so-called “dummy data.”

In other words, it’s not real customer data, but rather made-up, placeholder data that shows the user how Pendo’s analytics suite would work if they created a real account.

Using dummy data like this is a smart move and something that we’ve written about here at Userpilot in the past.

The alternative is to greet new users with a blank page, which is about as unappealing as starting an art exam in school with a blank piece of paper. It takes a lot of activation energy to overcome the initial friction of starting to draw something.

How to input real data into Pendo Analytics

If you want to replace the dummy data with real data from your own SaaS company, you’ll need to connect Pendo to your website.

You can do this by installing a JS script, in the same way that you would install Userpilot.

This is a one-time process that occurs through a HTTPS API.

Once the script is installed, Pendo will automatically collect and analyze user data from your web properties and/ or digital products.

Now let’s look at the four main analytics features that are available to you once you’ve connected Pendo’s product analytics to your website.

Pendo Analytics Feature #1: Paths

Paths show you every possible sequence of events before or after a specific user action.

That’s a bit abstract, so let me use a concrete example to show you what I mean.

Here is a screenshot from the Pendo demo of all the actions taken after users reach the “Accounts Overview” page. Bear in mind that all the data is dummy data still.

You can see that:

• 30% of users added a new account. It seems likely that this would be the dummy SaaS company’s preferred outcome in this instance.
• 26% of customers went to the Dashboard next, and 16% went to Contacts. This suggests that these customers were still somewhat engaged, but didn’t immediately take a step forward towards becoming active users.
• 15% were untagged. This suggests that these users selected a next step that hadn’t been pre-defined in Pendo by the dummy product marketing team.
• 13% are tagged as “No next step.” Most likely, these users just bounced from the website.

These types of insights into user engagement are useful because they help product teams identify the most common patterns in user behavior and act accordingly to improve onboarding systems.

In the event that 60% of users were bouncing, for example, one could speak to the design department to see if the UI could be made a bit more attractive.

Pendo Analytics Feature #2: Funnels

Funnels are in some way the inverse of Paths.

Whereas Paths show you all possible routes chosen by users before or after a given event, Funnels allow you to track the progress of users towards a specified sequence of steps.

Funnels will also show the number of users who dropped off at each stage so that your product team can step in and intervene accordingly.

Again, a visual example is helpful here so that you can see what I mean:

In this instance, 85% of users chose NOT to click Accounts Overview after landing on the Dashboard.

After that, 88% of those who DID land on the Dashboard chose not to add a new account.

Pendo’s Funnels versus Userpilot’s Goals

Userpilot has a similar feature to this called “Goals.” As the name suggests, it’s more goal-orientated than Pendo’s Funnels. Here’s what it looks like:

It’s notable that Userpilot lets you see all of the Goals you want to track for your business at the same time, as well as displaying the most important Goals at the top of the screen.

From the Overview section of the screenshot above, you can see that Userpilot is also still communicating where people drop off in your funnel.

It’s just doing that in a very essentialist way, where it only highlights the most important funnel, and still gives you the possibility to analyze other Goals in one click if you need to.

Compared to Pendo, where you need to specify which Funnels report you want to run and enter the desired parameters manually, Userpilot shows you more data more quickly.

So even though Pendo’s data visualization is quite flashy, it loses out compared to Userpilot in terms of operational efficiency.

Pendo Analytics Feature #3: Reports

There are four types of Reports that Pendo offers:

1. Page Views
2. Feature Clicks
3. Track Events
4. Guide Views

The analytics software will graph out whichever of these four feature adoption metrics you choose for whichever feature you select.

Any Reports you create are saved under “Saved Reports.” This is a valuable repository of legacy data.

In this example, I chose the “Add new account” feature from the dummy data, and this is what was graphed out for “Feature clicks”:

As with many of the analytics features Pendo offers, this graph is easy on the eye and straightforward to read.

Vanity data?

It’s somewhat helpful to see how users interact with the product in these reports.

Yet I suspect that Reports is more about vanity data than any of the other 3 analytics features that Pendo offers.

As the name suggests, the product feedback from “Reports” is quite generic, and it’s not immediately obvious how to use it.

This is very different from Userpilot’s Goals, where the data is directly tied to the customer’s self-defined Jobs to be Done.

Pendo Analytics Feature #4: Retention

This is my favorite Pendo analytics feature that I’ve found so far.

Retention shows how many of your customers return to your app over time.

It achieves this by highlighting the percentage of visitors or accounts who return to your app relative to the first time they used it.

Google Analytics has a similar feature, and one has the strong sense that this was a design inspiration for the Pendo team.

I really like the visual result here. It looks very elegant and is easy to read.

Quantitative analytics vs qualitative analytics

One thing that’s notably missing from the product insights and analytics Pendo provides is WHY customers are being retained or not retained.

Put another way, Pendo’s analytics are only quantitative. They only give you the numbers that show the change in retention rate over time.

By contrast, Userpilot gives you the option to create qualitative microsurveys.

Source: Postfity

Microsurveys don’t have to be complicated; one or two questions is plenty.

You can use these to follow up with users to find out WHY they are behaving in the way that they are.

And of course, you can still use NPS scoring to get quantitative analytics from Userpilot as well. This brings me to my next point.

NPS and User Feedback are NOT available in Pendo’s free plan

I spent some time looking for NPS analytics on Pendo’s free trial, but was disappointed to see that sentiment analysis is only available to paying customers.

This is different from Userpilot, which makes all features available to all plan types, regardless of how much people pay.

Pendo’s proprietary metric

Upon further digging, it appears that Pendo places great emphasis on a different quantitative metric to NPS, namely Product Engagement Score, or PES for short.

According to Pendo’s blog:

“The product engagement score (PES) is a metric that provides a singular view into how users are interacting with your product. It is a composite score, made up of three elements:

• Adoption: The percentage of users or accounts using key features in your product, represented by any or all of your Core Events
• Stickiness: The 30 day rolling average of the percentage of monthly active users who return either daily (DAU/MAU) or weekly (WAU/MAU)
• Growth: User or account growth rate for the chosen time period, compounded to the annual growth rate”

The value of direct customer contact

What’s conspicuously absent from this is any sense that the SaaS businesses that use Pendo should ever directly ask their own customers how they feel about using their product.

This is a problem, and actually seems a bit cowardly. It’s almost as if Pendo is scared of asking SaaS companies to engage with customers directly.

By contrast, Userpilot allows you to both solicit quantitative feedback from customers via NPS, and then follow up on that using qualitative microsurveys.

Pendo only allows you to ask for direct feedback if you pay them, and their commitment to PES suggests that they’re more interested in completely automating the analytics process than they are in enabling an honest feedback conversation between SaaS companies and their customers.

Call me old-fashioned, but I think that even in 2021, talking to your customers directly is something that most product managers should aspire to do once in a while.

Conclusion

Having read this article, you should now have a good understanding of the range of different analytics options that Pendo offers.

And what it offers is certainly respectable. I especially like the way that Paths show you where users go after viewing a certain feature, and the retention rate graph is visually impressive. Top marks to the designers in both cases.

Nevertheless, one feels like Pendo is a bit more concerned with pretty graphs for their own sake, rather than providing data that is directly tied to the goals of product marketing teams.

Compared to other onboarding software such as Userpilot, Pendo lacks qualitative analytics, and ways to speak to customers directly. Where such options do exist, they require you to pay, whereas Userpilot offers them as part of a free trial.

So on the basis of this evidence, it seems like Userpilot’s analytics might be the better choice here.

Do you agree? If so, you can test out Userpilot’s analytics by starting a free trial today.