New User Engagement

This is meant as an addendum to the post I wrote yesterday about measuring engagement on Hive and trying to determine if low engagement is a factor in user retention.

In that post I examined engagement by measuring the average number of direct replies to all comments and posts, excluding those from the top 50 accounts which feature high degree of automation. I used Granger Analysis to determine if the level of engagement on Hive could predict user activity in the future, which would be expected if it were a factor in user retention. There were no statistically significant results to be found.

A limitation of that approach was that it looked at engagement overall, but by far the most critical time for user retention is when a user is brand new. 24% of newly active accounts have made their last post within 3 days of creating their account. My concern with that approach was newly active user engagement being lost among the larger number of established users who also receive more engagement.

So today I will reexamine engagement but looking only at brand new user engagement. The following chart shows the average number of direct comment replies per day, to users who have created their account no more than 2 days before the comment is made.

Unlike yesterday's data, this metric turns out to be stationary so we can proceed to Granger Analysis immediately.

Below are the results of Granger Tests, statistically significant results are highlighted.

Conclusion

When we choose to focus specifically on brand new users only, there is some evidence that the level of engagement they receive has predictive value on user activity from 5 to 6 days out. The impact appears to be limited, but it nonetheless implies that the level of engagement that new users receive is a contributing factor in Hive's user retention problem.