Skip to main content

Overcoming advanced analytics obstacles

Insurance Consulting and Technology
Insurer Solutions

By Brian Mittleberg and Jason Rodriguez | August 5, 2020

Insurers did not make the progress they anticipated in advanced analytics. This article sheds light on the obstacles to implementation.


Our 2019/2020 P&C Insurance Advanced Analytics Survey (released in Q1, 2020) reveals a familiar story of insurers organizing their data sources in preparation for transformational analytics across pricing, underwriting and claims. One of the more obvious features of the 2019/2020 survey results is their striking resemblance to the 2017 results. In fact, we provided the 2019 and 2017 results in this year’s summary explicitly to chart the progress of our insurers. The unfortunate news is in the two years between the surveys, insurers did not make the progress they anticipated in advanced analytics (Figure 1). Based on our work with clients over the past two years, it is clear that the lack of progress is due to more than just a simple underestimation of effort. Many of our clients ran into significant obstacles that they didn’t foresee back in 2017. Those obstacles include unexpected complications in accessing nontraditional data sources, a lack of sufficiently skilled modelers and a lack of infrastructure to rapidly deploy their models.

Figure 1. Top applications insurers plan to use two years from now for AI and machine learning
Actual for 2017 Expected for 2019 (in 2017) Actual for 2019 Expected for 2021
Build risk models for better decision making 13% 44% 26% 60%
Reduce time spent by humans 11% 49% 22% 60%
Better understand risk drivers 21% 44% 20% 56%
Identify cases that pose higher risk 11% 46% 14% 50%
Augment human-performed underwriting 7% 37% 7% 47%
Identify patterns of fraudulent claims 9% 39% 17% 47%
Identify bottlenecks in claim processes/Process claims more efficiently 3% 30% 7% 43%

Nontraditional data source access

Looking at the survey results, insurers not only expected to employ more advanced techniques by 2019 but also expected to tap into nontraditional data sources. These sources include unstructured text data and web clickstreams (Figure 2), which have a depth of information that often warrants the use of advanced analytics (Figure 3). This makes progress on advanced analytics and nontraditional data sources intertwined, since our clients often expect to use advanced analytics techniques to analyze these nontraditional sources. When working with nontraditional data sources, our clients sometimes struggle to access the data, and the data are often of a much lower quality than traditional sources used for financial reporting. Given that these nontraditional sources are not currently relied upon for analysis, the information is typically not captured in a way that facilitates analysis, is inconsistent or is incomplete. Unfortunately, our clients sometimes do not learn this until it is too late, and it can take years to accumulate the necessary data to perform some analysis (e.g., profitability analysis on an insurance product).

Personal lines

Actual for 2017 Expected for 2019 (in 2017) Actual for 2019 Expected for 2021
Unstructured internal claim information 34% 66% 38% 69%
Unstructured internal underwriting information 24% 50% 18% 67%
Web/Clickstream/Phone/Email customer interactions 34% 47% 35% 62%
Social media 26% 53% 38% 58%
Images 11% 32% 24% 53%
Usage-based insurance (UBI)/telematics 26% 55% 24% 51%

Commercial lines

Figure 2. Nontraditional data sources insurers plan to use two years from now
Actual for 2017 Expected for 2019 (in 2017) Actual for 2019 Expected for 2021
Unstructured internal claim information 41% 91% 53% 81%
Unstructured internal underwriting information 25% 63% 16% 66%
Social media 19% 44% 16% 53%
Other unstructured internal customer information 9% 56% 9% 44%
Web/Clickstream/Phone/Email customer interactions 13% 44% 13% 38%
Web/Clickstream/Phone/Email agent interactions 13% 34% 13% 38%

This figure shows the advanced analytics techniques that insurers use. One-way analyses ranks highest at 81%, followed by generalized linear models at 79%.
Figure 3. Advanced analytics techniques that insurers use


How we can help

While an ad hoc approach is common, insurers are always better off with a solid data strategy. We help insurers to identify, characterize and prioritize their various data sources to ensure that the efforts to improve data access result in better analytics and greater business impact. We have the knowledge of both data and impactful analytics to help carriers identify useful data and understand the data’s value, which can often result in a major acceleration in the process of moving from data infrastructure improvement to impacts on business results.

Skills gaps

Advanced analytics projects can be complex and require uncommon skill sets. A good team has a balance of business subject matter expertise, IT knowledge and advanced analytics skills. Finding the right resources to fill those roles has become increasingly difficult as the demand for analytics increases across all industries. With limited resources to meet a carrier’s ever-growing ambitions in analytics, insurers often create matrixed organizations with an analytics center of excellence that allows analytics teams to serve business operations across the enterprise. This arrangement can be challenging for both the team to balance its workload and for management to get a clear picture of project plans and progress. Further, prioritizing projects that impact disparate areas of the business (i.e., that impact results in different ways) can be difficult, leaving open the possibility that the most impactful projects are overlooked. These disconnects can lead to misunderstandings about what can be reasonably accomplished and potentially lead to an overestimation of progress or business impact.

How we can help

We help organizations create project road maps that are grounded in advanced analytics realities to address these resource challenges. The first step is to take stock of ongoing and proposed projects to ensure that their current trajectory is consistent with future-state ambitions of the organization. We then work with the management team to convert this catalog of projects into a prioritized project road map. In contemplating the road map, management will have a focused discussion about each proposed project in terms of its proposed impact, effort and immediacy. This helps to set realistic expectations for the future and a clear understanding of what is needed to achieve the desired future state.

After the road map is established, we can help by supporting the top priority projects by both performing the work and advising on industry best practices. This support can be critical to ensuring that organizations set off on the correct path and avoid critical errors and expensive rework that cascade into delays for subsequent projects on the road map.


Perhaps the most disappointing difficulties relate to implementation. Most analysts have experienced the following scenario: After receiving some long-awaited data and working excitedly through the analysis, the project is communicated to business stakeholders in claims or underwriting only to learn that either the IT team is not prepared to deploy the analytical solution, or the claim or underwriting professionals are not prepared to use it. Both problems commonly occur, and these are worst-case scenarios: The insurer has now squandered both time and resources on a dead-end project. This outcome can be avoided through proactive communication and adoption of the right technology.

First, stakeholder engagement is a key ingredient for project success. It entails both alignment of objectives at the project onset and continual dialog as the project progresses. Analytics teams should share data summaries, observations and model descriptions in exchange for business workflow insight, important contextual considerations and general usability feedback.

Issues with IT implementation also have solutions, but rather than getting IT more engaged in the process, the best solution can be less engagement. At many technology companies outside of the insurance industry, the analytics team owns the deployment process; the IT team just provides the requirements for implementation and maintenance. For the analytics team to do this, they need to either use scoring engine software that allows them to automatically deploy their analytics into a production-ready solution or create their own scoring service (often deployed in the cloud). By far the easiest solution is adopting scoring engine software to simplify and standardize the technical deployment process.

How can we help

Our technology solutions are designed to directly solve challenges with model and analytics deployment. Radar Base and Emblem allow the analytics team to build analytical tools that span the analytical projects from data manipulation and model building to scenario testing. Once the analytical products are finalized, the final solutions (including generalized linear models and gradient boosting machines) and constructs (such as rates and underwriting rules) can be instantaneous deployed using Radar Live. This greatly simplifies deployment and reduces IT work required to implement the solution. Radar Live deployments can either be in a local on-premises IT environment or in the cloud.


Insurers have fallen short of their own expectations for advanced analytics. This might not be surprising to anyone given the many potential ways an organization’s plans can get derailed. Most organizations find it difficult to really understand their capabilities — especially in emerging areas where the requirements are not completely understood. The good news is that the most common problems have solutions in insurance-specific software products and expert advisory services. Insurers just need to understand the key obstacles blocking their way and be willing to adopt a broad range of solutions to get their plans back on track.

Senior Associate, Insurance Consulting and Technology

Data Science Lead, Insurance Consulting and Technology

Contact Us