About LangChain

LangChain is a popular and rapidly evolving framework to automate most of the management of, and interaction with, large language models: among its features are support for memory, vector-based similarity search, an advanced prompt templating abstraction and much more.

LangChain comes with a Python and a Javascript implementation. This section targets the Python version.

Info

To be able to run the examples below, first go through the LangChain-specific setup instructions.

Available components

CassIO seamlessly integrates with LangChain, offering Cassandra-specific tools for the following tasks:

Automatic injection of data from Cassandra into a prompt;
... the same, as part of a longer LLM conversation.
Support for "partialing" of prompts (i.e. leaving some input unspecified, to supply later).
Automatic injection of data from a Feast feature store (e.g. backed by Cassandra) into a prompt.
A memory module for LLMs that uses Cassandra for storage;
... that can be used to "remember" the recent exchanges in a chat interaction;
... including keeping a summary of the whole past conversation.
A facility for caching LLM responses on Cassandra, thereby saving on latency and tokens where possible.

Additionally, the "Vector Search" capabilities that are being added to Cassandra / Astra DB enables another set of "semantically aware" tools:

A cache of LLM responses that is oblivious to the exact form a test is phrased.
A "semantic index" that can store a knowledge base and retrieve its relevant parts to buil the best answer to a given question ("Q&A use case");
... whose usage can be adapted to suit many specific needs.
... and that can be configured to retrieve pieces of information as diverse as possible to maximize the actual information flowing to the answer.
A "semantic memory" element for inclusion in LLM chat interactions, that can retrieve relevant past exchanges even if occurred in the far past.

This list will grow over time as new needs are addressed and the current extensions are refined.