Research Hub > How Big Data Becomes Smart Data for Financial Institutions
White Paper

How Big Data Becomes Smart Data for Financial Institutions

Adopting AI to support analytics positions firms as leaders in the coming decade.
  • by Mike Chapple
  • |

Financial institutions are awash in data. From credit card transactions to payroll deposits, insurance claims to property records, the data warehouses of banks, insurance companies and other financial institutions are filled with petabytes of information that can provide business value. The challenge for financial institutions is transforming the Big Data that they already have into smart data that actually delivers business value.

The Big Data already stored in financial data warehouses shares three characteristics, often referred to as the “Three V’s of Big Data.” These are:

  • Volume: The sheer quantity of data poses significant challenges to financial institutions. Recent advances in storage technology make it possible to affordably maintain large quantities of information that would have been cost-prohibitive in the past.
  • Velocity: Data arrives at a dizzying pace. As just one example, more than 150,000 credit card transactions take place each minute in the United States. Big Data solutions must be able to rapidly ingest this data and store it for future analysis.
  • Variety: While many financial records come in highly structured formats, most data that exists today is unstructured and exists as free-form text, images, audio and video. Big Data solutions that focus only on structured data may miss out on major opportunities.

Cutting-edge financial institutions are in the midst of a transition. While the past decade was about moving from traditional in-person financial services to a “digital first” strategy, the next decade will be “AI first.” Financial institutions that spend time now building out their analytics capabilities will find themselves positioned as leaders in this new operating environment.


The number of loan agreements processed annually by AI systems at JPMorgan Chase.

Source: Financial Times, “AI big help for business negotiating red tape, survey finds,” Sept. 10, 2017

The Evolution of Data Analysis for Finance: From Big Data to Smart Data

Financial institutions have long embraced the power of computing to find needles of insight in haystacks of financial records. Credit reporting agencies pioneered the routine use of modeling by summarizing consumer credit histories with a single credit score. Credit card processors routinely comb through customer transactions in search of fraudulent activity. Insurance companies carefully scrutinize risk information, claims histories and financial records to accurately price products. These applications of analytics are nothing new. For decades, financial institutions depended on high-performance computing to crunch the data, often in the wee hours of the night, so they could deliver business insight with the morning’s first cup of coffee.

The overnight reporting model worked well for many years, but it simply isn’t up to the challenge of competing in the modern business environment. Financial institutions need faster access to data to guide real-time decision-making. Loan officers can’t make decisions based on yesterday’s data, and they can’t wait until tomorrow for the data to update. 

Analytics systems must also be able to handle the crushing volume and velocity of modern financial data. Systems that lack the capacity to store and analyze this granular data simply won’t rise to the bar set by competitors. Financial institutions now demand the ability to run many applications in parallel on the same stream of data, reacting in milliseconds to jump on opportunities before a competitor moves first.

Artificial intelligence solutions provide the technology platform that allows financial institutions to rise to this challenge. Designed to operate in Big Data environments, AI provides decision-making models that rapidly evolve in reaction to the changing financial environment. These models harness the power of Big Data by leveraging algorithms that quickly sweep through the massive amounts of information that are part of each record and pinpoint the data features that are most relevant to the decision-making process. AI focuses on teasing the story out of the data and making decisions informed by the data alone.

Institutions that leverage AI find themselves able to automate decision-making in unprecedented ways. AI technology that enters an organization as a tool designed to inform traditional human decision-making processes often moves into a new capacity when leaders witness its success. Instead of informing human decision-making, AI becomes the agent of digital transformation, allowing automation to overtake those processes and make them both faster and more powerful. Financial institutions that embrace this automation often improve their reaction time, reduce their margins and increase their profitability. 

To learn more about AI solutions, download the white paper "Smart Data for Finance."