ChatGPT for Customer Service: Automating Responses to Reviews

Client Profile

Our client is one of the leading property management companies in North America, operating manufactured home communities, RV resorts, and campgrounds. Structured as a REIT (real estate investment trust), the listed company has communities and resorts across the U.S. and British Columbia with hundreds of thousands of sites.

Managing online reputation is vital for businesses today for a range of reasons – including shaping consumer perception, building trust, gaining visibility and a competitive advantage, handling crises effectively, attracting talented employees, and fostering growth and partnerships. Given the potential of social media to make a single video or post viral in seconds, it is imperative for businesses to engage with customers online, resolve problems, respond to customer inquiries, and gather feedback for improvement.

For a property management company like our client, by actively participating in the online conversation surrounding its properties, the company can address customer queries or complaints quickly, enhance customer satisfaction, gain insights into guest preferences, and strengthen their position in the market.

But responding to online reviews can be a laborious and time-consuming process. Our client receives dozens of online reviews in a day, in the form of star ratings and comments, across platforms such as Google reviews and TripAdvisor, as well as on its social media accounts.

As is usually the case, customers’ reviews cover multiple aspects of their stay and convey positive and negative sentiments using different emotions and tones including sarcasm, happiness, concern, etc. This, coupled with the varying levels of English proficiency across their customer base, made it difficult for the client to respond to reviews correctly and in a timely manner. With the client’s marketing team handling responses manually, even responding to a star rating with no comments took 1-2 minutes.

Our client wanted to be more connected and enhance customer interactions by replying to all reviews and comments, while also improving the accuracy and time taken to respond. However, they didn’t want their replies to sound like the standard automated responses that businesses usually give on such forums.

Auxis has worked with this client since 2019, and is now a trusted advisor to them, so they turned to us when they decided to implement robotic process automation (RPA) and Artificial Intelligence (AI) to respond to online reviews.

Auxis’ initial approach was targeted at responding to star ratings received on Google Reviews, which had no comments, for which we created an automation that worked off a list of templates containing pre-generated responses for responding to the reviews.

The next, much more difficult task was automating responses to reviews with comments. The challenges there were many:

• If someone was complaining in a review, a generic response would be unhelpful or could make things worse.
• There was skepticism on the client side on whether RPA could quantify and correctly identify emotions and tone of voice such as annoyance, sarcasm, irony, etc. in reviews, leading to worries about vague/wrong responses in such cases.
• Interpreting reviews which were a mix of negative and positive comments would be a challenge to solve using traditional sentiment analysis models.
• There were also concerns about the ability to achieve standardization of outputs from such a model.

Auxis’ key solution steps included:

Integrating NLP tools for sentiment analysis

To tackle these challenges, we used IBM Watson Natural Language Processing (NLP) and Azure Cognitive Service for Language, both of which provide natural language processing functions and features for understanding and analyzing text.

We deployed these for customer sentiment analysis, to enable our automation model to select the most appropriate response from a list of predetermined responses. However, the responses were still too generic.

Enter ChatGPT to generate human-like responses with a personal touch

The Auxis support team then decided to leverage ChatGPT for customer service. This AI chatbot, which has gained instant popularity since its launch, is built on OpenAI’s GPT-3 family of large language models and has been fine-tuned using both supervised and reinforcement learning techniques. As tools using advances in generative AI work their way into businesses and society, they could drive a 7%, or almost $7 trillion, increase in global GDP, according to research from Goldman Sachs.

While the technology has proven itself highly capable of generating human-like responses, ChatGPT and similar deep learning models are often perceived as “black boxes,” that is, they produce outputs without providing clear explanations or insights into their internal workings. This was a challenge the Auxis team took on when building our solution.

We initially did a PoC using ChatGPT, testing four different GPT-3 models that can understand and generate natural language – Ada, Babbage, Curie, and Davinci.

We did configurations using Python and began the iterative Prompt Engineering process for ChatGPT to generate more human-like responses tailored to the client’s requirements. Later, we moved to the GPT-3.5-turbo model, which is the most advanced language model to date and provides greater robustness to the solution.

We did continuous and ongoing adjustments and finetuning of algorithms to provide guardrails for the solution, such as a list of considerations that ChatGPT uses to create responses, such as removing references to employees or specific places that were mentioned in the review, avoiding agreeing with negative reviews or committing to anything, and not using words such as “free”, “low cost”, “no cost”, “refund”, “discount”, etc. for generating responses.

The solution is also trained to flag any reviews which include “forbidden” or explicit words and scenarios as exceptions. These are routed to human respondents along with the reason for the review being flagged, all the information needed for them to respond to it, and the suggested response generated by the bot.

This has resulted in the solution generating personalized and relevant responses with a very high level of accuracy.

Our configurations also work in other languages, helping expand the capability of using ChatGPT to answer reviews in languages apart from English.

Secure protocols for data, privacy concerns

Auxis also ensured that data and privacy concerns are factored into our response generation model, sending generated responses through different filters to ensure that the business and data privacy conditions are met. This was implemented using secure protocols in the API call and on the UiPath software.

We integrated our solution with Revinate, a guest communication solutions platform for the hospitality industry, for publishing the generated response on the correct review platform.

The PoC solution was ready for testing in 2 weeks and was merged with the existing response automation in one week. Excited about the results from this initial PoC, the client gave their go ahead, and the solution went through the User Acceptance Testing phase to final implementation on Google Reviews within a few weeks. The solution also went into production for TripAdvisor a few months later, with minor modifications. From process discovery and analysis through go-live, Auxis delivered the solution for both platforms in just 2 months.

Our solution integrating ChatGPT with RPA has transformed the way the client handles the laborious, and
sometimes, complicated task of answering customer reviews. It has given them the ability to respond to
Google reviews in near real time, leading to faster resolution of customer complaints and greater
satisfaction, and opened a pathway for us to onboard more of the client’s social media and online portals
to this solution.