Time to 1st response and csat. A box and whisker graph case.

19Mar
by Raúl Antón Cuadrado

Time to First response is a very popular metric when it comes to support and there is a powerful reason dor that. It has a direct impact on customer satisfaction that can be measured.

Main Outcomes

  • Time to first response is longer than what we perceive.
  • There is a correlation between time to first response and CSAT score we get, so we can assume a correlation between client satisfaction and time to first response.

What’s time to first response and why is this important?

First we need to define TTFR. In my case the metric shows how long it takes us to get back to a customer’s message (and this could be applied to chat or email questions/requests). Time to First Response measures the time that a customer waits until they understand we are “with them”.

Fancy a graphic definition extracted from a support chat?

Jamie Edwards, COO and Co-founder of Kayako says that “First reply time is more important than overall reply times because it’s an acknowledgment to the customer that their issue is being looked into.” and Tim Woo, Marketing Director for Framed Data is also pretty clear “Speed is king for the first reply. Contrary to what we would assume, most customers prefer a quick but “ineffective” response over a calculated, delayed answer“.

Both of the above opinions can be discussed. But we can accept that the sooner we acknowledge a user question, the better impression we cause, the better state of mind we create to start and the more we will be able to help and create satisfaction.

The problem. We need to calculate how fast we are sending the first message to our users in support chat.

Let’s imagine we have a table containing all the messages on our chats, with a FK, the chat_session, we can get a list of chat sessions each one with 3 values: when it starts, when it finishes and when the customer gets the first response from our operator.

   select
        chat_messages.chat_session,
        min(timestamp)  as session_start_dt,
        max(timestamp)  as session_end_dt,
        min(
            case
                when ( msg_from_operator = true ) THEN timestamp
                else null
            end
        ) as session_first_response_dt
    
    from chat_messages
    group by chat_messages.chat_session
    having min(from_unixtime(cast(timestamp / 1000 as bigint))) >= '2020-03-01 00:00:00'

Where the ‘having min’ clause assures we are not interrogating the DB for a too long period.

Solution. Pasting results to Excel to get some indicators.

I would start by asking chat operators about their perceived time to first response. An anonymous poll with 2 questions would be enough…

  • How long do you think it takes for a customer to get an answer on average?
  • How long do you think it would be acceptable to get an answer when you reach out to customer care for another company?
  • Bonus if you decompose the previous question into 2 (Time for a regular support service, time to consider it top-notch)

This could be extremely useful to compare perceived results with real values. I asked this question to about 20 support agents and I got this result:

By pasting results from the first query to Excel, we can easily get average time to 1st response (I bet it is higher than you expected), the median, percentiles… With the information exported to Excel…

  • You can also analyze extremely high times to first response to see if there is a pattern that can be analyzed.
  • Subsequently, you could find some unanswered chats. My recommendation here would be to launch a strategy to get back to these users you did not answer when they tried to contact you. (In my case I have a Looker Daily report with the list of chats, sent to several persons)
  • You can also create some cool, nice and insightful graphs…

Again, a picture is worth a thousand words.

Result. Correlation between customer satisfaction (CSAT score) and time to first response could be visible with the right graph ( Boxplot ).

There is a correlation between customer satisfaction expressed in the bot score and the time to first response.

Note bot score is called this way because it is represented by bot, red, blue or green depending on the customer answer to a feedback poll.

The 3 famous bots of the botscore

In this case, the problem is to find a graph showing the correlation. It is not a trivial decision. I’ll save you the details: the Boxplot is the right choice as we can compare time to first response quartiles for each bot answer and display the 3 boxes altogether:

This graph would basically mean that you get more or less the same distribution of green/blue/red bots when you answer fast, but from a given elapsed time you have way more possibilities to get a red bot. 

This has a clear effect on the Average Bot Score depending on the Time To First Response, as the following graph also highlights.

Final Notes

We don’t evaluate our support responsiveness is by the first response. First response time is just a small part of the overall experience.

Leave a Reply