Generative AI and Large Language Models (LLMs) are advancing so quickly that it has led to overreaching marketing claims about what they can do, including solving very hard ML problems like energy optimization, sales forecasting, predicting failure of an asset, etc.
While technical practitioners may recognize such claims are often misleading, many business leaders, seeing aggressive marketing from large companies, may take them at face value. So for executives, I want to answer two key questions in this post:
- Where do Large Language Models fit within everything else your business uses for analytics and decision making?
- Given what LLMs can and can’t do right now, how can you best take advantage of their strengths?
I’ll cover some basics first, but if you are already well-versed with these, feel free to skip ahead to the framework section further below.
Understanding Enterprise Data
To understand where LLMs fit in, it helps to look at how enterprises currently manage data. The Digital Age has created both opportunities, like leveraging data for insights, as well as challenges like analyzing unprecedented volumes. Let’s quickly recap some key concepts of Enterprise Data to establish the foundation for where LLMs can help: Types of Data and Analytics Maturity Model.
Differentiating Structured and Unstructured Data
Enterprise data mainly falls into two types: structured and unstructured. The kind of data you have determines the tools and methods needed to get value from it. The following table outlines the differences between these two types, helping to clarify how each is used in data analysis.
The Four Levels of Analytics Maturity
The Analytics Maturity Model helps organizations maximize the value derived from data in a methodical manner. It has 4 levels: Descriptive, Diagnostic, Predictive, and Prescriptive. Sometimes a fifth level, Cognitive, is used.
Organizations evolve through these levels as their analytics capabilities mature over time — from reactive reporting to predictive planning to AI-powered strategic advice.
The key is to leverage the most advanced analytics suitable for any business situation. While descriptive analytics works for rear-view reporting, predictive analytics enables smarter future planning. Over time, organizations scale their analytics capabilities vertically across all levels.
Strengths and Challenges of Large Language Models
While the umbrella term ‘LLMs’ refers broadly to all large language models, there are different types suited to particular use cases. Choosing the right LLM depends on your specific use case. It can be technical, so referring to previous blog posts might help.
- Foundational models: These form the core and are broadly trained on vast datasets covering all knowledge areas. They can perform various tasks like understanding emotions, summarizing text, and more. Ex.: Llama2
- Chat Models: These are fine-tuned on top of foundational models for meaningful conversations. They’re used in chatbots and virtual assistants. Ex.: Llama Chat
- Code Models: Specialized in understanding and generating code, such as translating between programming languages or automating coding tasks. Ex.: GitHub CoPilot.
(Passing over others in the interest of brevity.)
What unites all LLMs is profound text comprehension, having learned from vast volumes of diverse writings. This allows them to deeply learn the patterns of human-written language and develop a strong comprehension ability. This enables remarkable skills like:
- Analyzing sentiment in passages
- Summarizing documents
- Answering text-based questions
While text-based training equips language models with language understanding, it doesn’t grant them inherent numeric prediction abilities. They grasp structured data superficially, missing crucial mathematical relationships. Without an innate understanding of numeric dynamics, they recognize patterns but can’t reliably project them into the future. Their interpretations offer insightful narratives but lack statistical validation and confidence intervals for reliable forecasting. In essence, this rules out language models from making structured data predictions in enterprises.
Additionally, from a technical perspective, LLMs grapple with:
- Handling Structured Data: Most business-critical data is stored in databases. While LLMs can create chat interfaces, there are challenges in using them for accurate database queries, including potential errors in SQL formulations to extract correct data.
- Inconsistent Responses: Enterprises require consistent outputs, but LLMs provide probabilistic responses, making it hard to guarantee consistency, especially in chat interfaces.
Now that we understand the strengths and limitations of LLMs, enterprises can maximize their value by focusing on their descriptive analytics strengths while remaining realistic about their limitations. The following is a simplified Analytics Maturity Model, focusing on descriptive and predictive analytics.
A framework to introduce LLMs to advance the Analytics Maturity Model.
The biggest value in deploying LLMs is to enhance access to past events and qualitative insights. Thinking they will improve statistical forecasts or algorithmic recommendations is unrealistic currently. By correctly leveraging LLMs for descriptive analytics, enterprises can maximize productivity and democratization. But, predictive capabilities will still rely on traditional Machine Learning solutions.
While text-based training equips language models with language understanding, it doesn’t grant them inherent numeric prediction abilities
Getting Instant Insights on Past Events
LLMs transform accessing what happened through conversational
Continue reading where it was originally published: https://www.actalyst.ai.
