Management Summary

Project summary

In 2018, Swisscom Enterprise has introduced the Artificial Intelligence (AI) in customer service by launching a Mailbot solution. The customer agents are now starting to rely on machine learning algorithms to analyze and classify customers e-mails automatically. The objective of the Mailbot project is to reduce the Contact Centers workload, to improve the efficiency and the transparency in the service request management by:

• Automating the email handling/triage and dispatching of the customer requests 
• Providing transparency in the steering of requests with the collection of new indicators and the implementation of new dashboards
• Providing transparency to customers on submitted requests and fulfillment status
• Reducing the effort for status emails / calls with customers 

The goal of this Master Thesis is to measure and analyze the performance and impact of the new Mailbot on Swisscom Enterprise to demonstrate with evidences:

• The achievements and performance achieved
• The impact on the service request management, the customer care service and more generally on the internal organization

Background

A deep assessment conducted at Swisscom Enterprise revealed that the several online portals and entry points were not optimum in terms of customer experience and satisfaction. In consequence, customers were engaging directly with the Contact Center by phone or email which significantly increased the workload of the Contact Centers agents over the years.  

Swisscom Enterprise was looking to introduce AI technologies in its operation and searching for use case with direct business value. An investigation was performed to understand the challenges in the automation of Email triage and see how AI could be applied to improve efficiency in the Customer support. This is how the Mailbot project was initiated. 

Two short term business cases were identified, and the objectives set:

• SPS:
  • Purpose: Automate the catch and dispatch of information requests received from customers on info.enterprise@swisscom.com which were currently outsourced to Swiss Post Solution and manually processed by their agents. 
  • Objective: 
    • Process emails, open Service Requests (SR) and dispatch automatically 80% of them. 
    • Terminate the outsourcing contract with SPS by the 31^st of December 2018.

• ECC Mobile: 
  • Purpose: Automate the catch and dispatch of mobile services requests to reduce the workload experienced by agents due to the current customer experience with multiple online portals.
  • Objective: Process emails, open, prioritize and dispatch automatically 50% of SR. 

Swisscom Enterprise Customer Care was also suffering from the lack of transparency and indicators in the service requests management. The end-to-end performance monitoring of requests management was not possible with the existing workflows, data and tools. SR were not systematically opened, and no automatic confirmation were sent to customers. The business indicators were focused on the productivity of the agents (effective time spent of SR / presence time) and status of SRs (number of SR opened, number of SR closed, number of SR per domain). This was a clear call to provide transparency and build new KPIs to better steer the service request management. 

The call for transparency and the expectations from the management were supporting the need to demonstrate the benefits of the Mailbot project through a performance and impact analysis.  

Process

This research work lasted 14 months in total and the following methodology and steps were followed.

The process designed was simple and generic enough to be compatible in other research context.  The first step has been to build a global understanding of the research context, the current situation at Swisscom Enterprise and the stakeholders’ expectations.  Based on those findings, a series of workshops and meetings have been conducted with key stakeholders to select the research methods and design the approaches for impact and performance measurement:

• Quantitative approach for performance
• The sources of data (Swisscom data warehouse and the Mailbot database) have been identified
• Business specific and machine learning KPIs have been defined
• The measurement solution was implemented by Swisscom Data scientists using the Elasticsearch/Kibana stack, a suite of open-source analytics engine and tools
• The KPI have been computed every day and then aggregated for analysis on a per week basis
• Qualitative approach for impact
• An online survey has been conducted with agents from all customer care teams and members of three execution groups to evaluate the impact of the Mailbot on the daily work and the customer satisfaction.
• Personal interviews with key stakeholders have been organized at the end of the research to review the initial expectations, validate the achievements and gather the lessons learned. 

Findings and conclusions

The performance and impact analysis have demonstrated the following: 

• SPS:  

The Mailbot was launched for SPS on the 16th of April 2018 and it is a success in all aspects (business objectives, performance, impact). The target objectives of 80% of the total volume of emails automatically processed by the Mailbot has been reached in 9 months. This achievement has allowed the termination of the SPS contract on the 31st of December 2018 according to the planning. 

The Mailbot solution has proven to work very well as the context was “ideal”: Reduced customer expectations, low differentiation needed between requests categories, availability of labeled data at the beginning and no specific organizational and workflow change. 

• ECC Mobile:  

The Mailbot was launched for ECC Mobile on the 3^rd of September 2018. The evaluation is positive but more nuanced than SPS. The Mailbot has proven to be successful in providing the transparency to better monitor and steer the management of requests for mobile services. This is already a key business value which has improved the efficiency in managing the requests priorities. After 6 months of operation, the current performance with 22% of emails automatically processed (target of 50%) do not allow yet to demonstrate the level of automation and efficiency improvements expected. 

The context of ECC Mobile has proven to be much more challenging than SPS: High customer and reactivity expectations, high differentiation needed between requests categories (each nuance count), limited emails volume per category, no labeled data at the beginning, important and still ongoing organizational and workflow changes. 

The focus of the project team is now to be better at requests classification and prioritization for ECC Mobile. To tackle the current limitations, the following actions have been defined:

• Investigate the decrease of performance in accuracy and use case detection since the addition of new sources of emails
• Industrialize the continuous improvement of the machine learning algorithm: feedback loop and classification model 
• Compute and check again the confusion matrix with the business to merge or remove some classes and simplify the requests catalogue
• Inform and further train the agents on the SR classification to verify that they are systematically classifying SRs according to the Mailbot rules. 
   
• Overall:

The surveys and personal interviews have proven that the Mailbot project had a positive impact but require some improvements. The results showed it had no negative impact and disruptive effects on the customers. The automatic confirmation sent to customers seems to even improve their satisfaction. 

This Master thesis has supported the definition of a generic approach in six steps with recommendations to Swisscom Enterprise to continue with this transformation program and manage other AI initiatives. Swisscom Enterprise has gained valuable experience and maturity to move forward in the implementation of the Omnichannel Engagement Center and the introduction of Voicebot and Chatbot. While the focus remains those internal initiatives, Swisscom Enterprise can now capitalize on this investment to see new business opportunities. 

Finally, and based on ECC mobile experience, an AI initiative should not be seen as a technological project but rather as a business solution. Even if a good understanding of AI capabilities is a prerequisite, it is by sharing a common business vision with trained and knowledgeable teams that Swisscom Enterprise will succeed in this exciting journey. This will be the driver for new business innovation and opportunities powered by AI in the coming years.