Small language models (SLMs) have captured enterprise interest due to their ability to support advanced AI applications while requiring less effort to build and operate than large language models (LLMs). SLMs are particularly useful for applications where computational resources are limited (e.g., mobile devices and edge computing). Furthermore, their ability to be fine-tuned for domain-specific tasks makes them prized for applications like virtual assistants, chatbots, and decision support (for customer service, help desks, HR, and other scenarios).

SLMs Defined

SLMs are designed to perform natural language processing (NLP) tasks efficiently while using fewer computational resources compared to LLMs. They are built on the same foundational architecture as LLMs, typically using transformer-based neural networks; however, SLMs employ various techniques to reduce the size of their architectures.

SLMs are termed “small” because they have significantly fewer parameters — ranging from a few million to a few billion — compared to LLMs like OpenAI’s ChatGPT or Microsoft Copilot, which can have hundreds of billions or even trillions of parameters.

Parameters play a key role in a model’s ability to understand language and improve its performance. In effect, they are numerical values that help determine how the model processes and generates text. You might think of them as settings or weights that act to guide how the model interprets input and predicts output by adjusting the strength of the relationships between words or concepts. In large models like ChatGPT or Microsoft Copilot, billions of parameters may work together to create coherent, contextually appropriate responses to inputs.

Despite having fewer parameters and smaller, more compact architectures, SLMs provide many of the key NLP capabilities of their LLM cousins, including:

• Text generation — predicting the next word or sequence of words based on a given input, using patterns and context learned from large amounts of text data; this enables the model to generate coherent, contextually appropriate responses or content.

• Text classification — categorizing text into predefined groups (e.g., for sentiment analysis, spam detection, fraud detection, and sorting customer feedback).

• Text summarization — producing concise summaries of longer textual inputs.

• Language translation — accurately converting text from one language to another.

• Named entity recognition — identifying key entities like names, dates, and locations in text.

• Tokenization and part-of-speech tagging — for breaking text into meaningful units and analyzing grammatical structures.

Benefits of Using SLMs

SLMs are built with fewer parameters and employ less complex neural net architectures than LLMs. This allows for faster training, improved accuracy (for focused domains), reduced energy consumption, and deployment on devices with limited resources, all of which make SLMs attractive for building enterprise applications.

Faster Training for Domain-Specific Applications

SLMs are prized for their ability to be customized and fine-tuned to support more narrowly focused applications, such as customer service or financial/medical document analysis. Their smaller size and focused scope allow them to be trained and customized with smaller datasets (specific to their intended function), which helps reduce the computational and operational costs associated with developing and deploying them.

Basically, SLMs do not require the massive datasets required for training LLMs, which are intended to support a broader range of use cases. Such specialization makes SLMs more efficient (and less resource-intensive) for narrowly focused use cases, offering a major advantage by alleviating the overhead involved in developing and deploying a broader and general-purpose LLM like ChatGPT. Moreover, because it is faster to train smaller models than larger ones, the development process is accelerated, leading to faster deployment and testing of new applications.

Improved Accuracy

Because their training is focused on specific tasks, SLMs can provide more accurate responses and information within the domain-specific areas they are trained for. Their specialized nature allows for additional fine-tuning that can even outperform LLMs in domain-specific applications.

Efficiency & Cost-Effectiveness

SLMs require less memory and computational power than LLMs, allowing them to run in low-power environments, including on smartphones and edge computing applications, without requiring the use of cloud computing platforms. This makes SLMs ideal for developing intelligent, AI applications where latency can be an issue (e.g., real-time language translation systems on mobile devices). Not having to ship data to the cloud is also beneficial for applications where data privacy is necessary (e.g., financial, health/fitness, location information). Lower computational requirements also translate to reduced energy consumption and operational costs. This makes SLMs more environmentally friendly and cost-effective to develop and run than LLMs.

Efficiency is also important for enterprise use cases. SLMs are well-suited for customer service, enabling real-time automated responses to support inquiries and powering applications that rely on real-time data analysis.

Considerations & Potential Drawbacks

Organizations should weigh a number of possible drawbacks when considering adopting SLMs, including limited comprehension, challenges in cross-domain adaptability, and ethical considerations.

Limited Comprehension

SLMs can be limited in their ability to handle complex, context-heavy tasks compared to LLMs. For example, because of their smaller size and fewer parameters, SLMs may struggle with understanding context in lengthy or intricate text inputs. This could result in less accurate or less coherent responses, particularly with applications that require deep comprehension or reasoning.

Challenges in Cross-Domain Adaptability

SLMs can have problems generalizing across diverse domains. As mentioned previously, organizations typically need to customize their SLM implementations to support specific tasks, domains, and industries. Although this fosters high accuracy within those areas, it could potentially limit their adaptability to other use cases. Additionally, the smaller scale of SLMs could lead to accuracy issues should they encounter rare or unusual vocabulary and linguistic patterns.

Ethical Considerations

Like all language models, SLMs may inadvertently perpetuate biases present in their training data. While their smaller size often makes them easier to audit and fine-tune, the risk of bias remains an ongoing concern.

Applications & Domains

Organizations are implementing SLMs across various applications and domains, including:

• Customer support — chatbots and virtual assistants for handling customer inquiries and providing support. For example, intent recognition can be used to understand the purpose behind a customer’s query, such as categorizing it as a request, complaint, or product inquiry. Virtual assistants can automate product advice and troubleshooting for customers.

• Education — personalized learning tools tailored to student needs (e.g., individual study plans and flashcards), automated grading systems, and content summarization.

• Finance — customer service automation and fraud detection (e.g., financial document analysis to identify deceptive or fraudulent text within financial documents or communications).

• Healthcare — assisting with medical documentation, summarizing patient records, supporting telemedicine, and providing automated recording and transcript generation of patient-doctor interactions, both online and for in-person appointments.

• Retail — product recommendations, sentiment analysis of product reviews and customer inquiries/complaints, and automating inventory management.

These are just a sample of some of the more popular SLM applications we are currently seeing.

Conclusion

SLMs have the potential to transform the AI landscape by making advanced NLP capabilities more accessible and sustainable. Their efficiency and ability to operate with fewer computational resources make them ideal for resource-constrained applications, such as mobile devices and edge computing. Additionally, their usefulness for domain-specific tasks positions them as valuable tools for sectors like customer service, education, finance, healthcare, and retail.

Despite certain limitations, SLMs are an important technology for organizations seeking resource-efficient NLP solutions. Key players like OpenAI, Anthropic, Hugging Face, Google/DeepMind, Alibaba, Microsoft, Meta, and others are driving advancements in this space, highlighting the growing significance of SLMs for modern AI application development.

Part II of this Advisor series will examine SLM offerings and take a closer look at some of the applications organizations are building. In the meantime, I’d like to get your opinion about SLMs in general and, in particular, which applications you see as particularly important or useful. As always, your comments will be held in strict confidence. You can email me at experts@cutter.com or call +1 510 356 7299 with your comments.