Maturity models for NLP and LLM usage: How to go from unsafe, unfocused results to focused, business-safe results.

Thu, 07 Sep 2023 00:00:00 +0000

Gaurdrails Link to heading

What is the concept? A block of code, framework, or library which wraps around your request/response cycle with an LLM (or traditional NLP) to cross-check, verify, validate, or otherwise improve the accuracy and safety of the results.

Gaurdrails is a way to make your AI a bit more humble.

Examples of types of gaurdrails (blatantly stolen from NVIDIA’s framework’s docs)

Topical - keeping the chat or response on topic.
Fact checking - make sure the LLM response is true, according to some external data or service.
Moderation - profanity filtering, ethics adjustments, or other techniques to shut down non-useful interactions (ie: to save you cost!)
Jailbreaking attempts - protect the chat from adversarial interactions (ie: trying to get your bot to lie, or do bad or expensive activities)

Maturity model using LLM as the foundation: Link to heading

LLMs are powerful - but, can easily go “off the rails” and in a customer service scenario, that’s not good for business. Could be as benign as a bad restaurant recommendation, or as bad as giving improper advice on a medical issue.

Maturity Model on jason grey

Maturity models for NLP and LLM usage: How to go from unsafe, unfocused results to focused, business-safe results.

Gaurdrails Link to heading

Maturity model using LLM as the foundation: Link to heading