Blog

Whether it’s the mesmerising artistry of Midjourney or the uncanny conversational abilities of ChatGPT, you’ve likely experienced the power of large language models (LLMs) by now. These AI models have taken the world by storm. 

While it’s clear that businesses will gain a competitive advantage by using these models, the use cases, strategies, and key considerations when doing so are unclear at best and dangerously deficient at worst. That’s because LLMs promise to enhance productivity, but they require strategic and thoughtful implementation in order to be truly effective. 

Before an LLM can be a workplace superhero, it needs an enterprise-strength strategy.

Deciding your use cases and implementation strategy

When it comes to determining the processes and workflows you want to transform with LLMS, I like to think of it in four tiers: 

• Tier One: This is a basic integration with a simple API call to an LLM. It is suitable for general information tasks like content generation and summarisation, auto-completing email responses, or sentiment analysis. With quick implementations that involve minimal developer resources, tier one use cases are an ideal starting point for organisations venturing into AI-driven assistance.
• Tier Two: These require customised LLMs, fine-tuned on specific organisational data, which allows the LLM to perform domain-specific tasks like translating IT support tickets or drafting an FAQ for the finance department. It requires more resources and advanced techniques like fine-tuning and retrieval augmentation. 
• Tier Three: These involve chaining multiple LLMs for complex, multi-step tasks, such as providing multilingual IT and HR support, moderating content, or optimising supply chain operations for businesses. These use cases tend to have a very high impact if done correctly. 
• Tier Four: The final and most complex tier is designed for enterprise-wide deployment—supporting a wide range of functions with advanced reasoning and deep integration techniques. This requires pairing multiple LLMs with in-house models and hundreds of enterprise systems. These AI “copilots” can provide enterprise-wide employee support, assist with decision-making by providing insights, monitor compliance and security, and much more. These use cases offer strong control and compliance but are highly complex to manage on your own. 

Each tier provides different levels of integration and customisation, suitable for varying needs from basic task automation to complex, organisation-wide solutions.

The higher the tier, the tighter the security

There are several key considerations to think about when it comes to LLM security: 

Understanding and protecting LLMs

• Training: Ensure the integrity and diversity of the training dataset to prevent biases and poisoning. Employ human and machine-assisted reviews to maintain dataset quality.
• Data fine-tuning: Safeguard sensitive data by using adapters to customise LLMs for your enterprise without exposing your data to global models.
• End-user controls: Implement robust access controls and parameter validation to prevent prompt injection attacks and data leakage. Especially in generative models, grounding the responses in task-specific content is crucial for accuracy.

Mitigating threats

• For discriminative models: Address biased classification and overfitting by using balanced and diverse training datasets. Protect against parsing attacks by validating extracted information before its use.
• For generative models: Counteract risks like hallucination and data leakage by ensuring the model’s responses are grounded in accurate and relevant information. Implement strict controls on the model’s output to prevent the generation of sensitive data.

Adapting to the implementation tier:

• Lower tiers (one and two): Focus on basic security measures like data validation and user access controls. In tier two, where customisation begins, enhance privacy protection through data masking and careful dataset management. 
• Higher tiers (three and four): Implement advanced threat detection systems, sophisticated privacy controls, and comprehensive access management strategies suitable for enterprise-level deployment. Regular compliance checks and continuous monitoring become essential to address the increased complexity and risks.

As businesses advance through the tiers of LLM implementation, their security strategies must become increasingly sophisticated. However, I understand that not every business has the engineering resources and know-how to take this on. 

Know when to build versus buy

The build versus buy decision hinges on several critical factors, key among these are the engineering team’s size and expertise with LLMs. Smaller teams or those with limited LLM experience may find the complexity and resource demands of higher-tier implementations daunting, making an out-of-the-box solution more viable. 

Cost and time-to-market are also essential considerations. Building a custom solution might offer long-term value and customisation but often comes with higher initial costs and longer development times.

For businesses facing intense competition or operating in fast-evolving markets, the delay in market entry inherent in building a solution might outweigh these benefits. Buying a ready-made solution, despite possible higher upfront costs, can expedite market entry, a critical advantage in leveraging LLMs for competitive gain. 

Ultimately, the build versus buy decision should reflect a balance between your team’s capabilities, the urgency of deployment, and the specific needs of your business at different LLM implementation tiers. 

Conclusion

Navigating the integration of LLMs requires a blend of strategic planning, security awareness, and discerning the optimal path between building in-house and opting for external solutions. If applied successfully, LLMs can transform from just advanced technological tools into fundamental elements that drive business growth and efficiency.

Vaibhav Nivargi is the Co-founder and CTO of Moveworks.