Artificial Intelligence has been a field in constant evolution since its inception over 75 years ago. With innovation at its core, we’re witnessing the emergence of Gestalt AI — a cold start, few-shot learning, unsupervised data processing AI, capable of extrapolatory answers to real world problems.
The Paradigm Shift to Small Data
There is a clear distinction between the Big Data techniques and the Small Data approach. Big Data has been the cornerstone of AI development for years, focusing on solving complex, poorly understood problems by sifting through massive datasets. The goal was to find patterns and insights within the chaos, a task that required considerable computational power and resources.
Small Data techniques shine where problems are well understood, centered on a bounded domain—a specific context or set of conditions—rather than the entire functional domain of a problem. This targeted approach allows for the creation of models “on demand” and tailored to the immediate needs and data associated with a particular query.
Creating a Small Data model involves a different technique compared to its Big Data counterpart. They require less data to tune their parameters because they don’t attempt to generalize over a broad functional domain. Instead, they focus on predicting outcomes in a narrowly defined context, leveraging a smaller, context-specific data set.
Local Momentary Models: The New Frontier
Small Data Models, or what we refer to as Local Momentary Models, approach to problem-solving is precise and tailored unlike Big Data Large Models that make many assumptions about the data and the internal architecture’s ability to represent various perspectives.
Let’s illustrate this concept: Take the shared experience of smelling freshly baked cookies. Sugar is the shared datum and the activation function as the smell that triggers a response. We’re drawn to the kitchen by the smell, but the underlying motivations and experiences are distinct. Sugar is a molecular form of information, when we taste sugar the taste buds send an analog electrical signal to our brain conveying the expression of “sweet” as a symbolic abstraction.
The advantages of Small Data Models are manifold. By focusing on a specific and well-understood domain, these models can be more efficient, requiring less data and computational resources to produce accurate predictions. They are also more adaptable, as they can be rapidly developed and deployed “on demand” to suit particular situations or queries.
As AI continues to evolve, Small Data Models are set to play a crucial role in the future of machine learning. By offering tailored, efficient, and precise solutions to well-understood problems, they represent a significant shift in how we think about data and AI development. As we embrace this new approach, we find that, as Winston Churchill said, “Out of intense complexities, intense simplicities emerge.”