Getting Started with LangChain: Essential Patterns and Tips
Written on
Chapter 1: Introduction to LangChain
In my quest to comprehend the full potential of LangChain, I delved deeply into its documentation. Despite having used it for several months, I discovered numerous concepts that were previously unfamiliar to me. Initially, I adhered strictly to the quickstart guide, but I soon recognized that this approach was more convoluted than necessary.
I believe the documentation's first page should feature straightforward examples that address the majority of common use-cases—approximately 80%. This guide aims to serve as that essential onboarding resource, along with some additional insights, essentially acting as a refined quickstart guide—version 2.
LangChain's Objective
LangChain aims to provide a unified abstraction layer for all tools or libraries you may integrate while working with Large Language Models (LLMs). For instance, if your goal is to implement a chatbot on your site capable of answering queries based on your documentation, you will require a vector database. The steps involved include:
- Retrieving the content of each page
- Embedding the content
- Saving the vector store
- Loading the vector store during runtime
- Embedding the user's question
- Searching the vector store for an answer
- Sending the top N documents to an LLM for response generation
With LangChain, this process can be simplified to just a few lines of code—essentially matching the number of steps involved. The documentation includes helpful examples to guide you through these tasks.
Getting Started with LangChain and Llama 2 in 15 Minutes
This video offers a quick overview of the LangChain framework and Llama 2, providing insights into how to start integrating these technologies effectively.
Define Chains Simply
A common pattern in my LangChain code can be summarized with the following lines:
from langchain.chains import LLMChain, SequentialChain
from langchain.chat_models import ChatOpenAI
# Commonly reused LLM instance
LLM = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0.1, request_timeout=60)
# Example of a chain to answer questions
answer_question_chain = LLMChain.from_string(
LLM,
"""
You are a truthful and expert customer service agent.
Answer the user question directly based on the relevant documents.
If you cannot find an answer, respond 'I do not know.'
* Relevant Documents *
{relevant_documents}
* User Question *
{user_question}
""",
)
# Sample execution
print(
answer_question_chain({
"relevant_documents": "* France is in Europe", # It's helpful to prepend bullet points to documents
"user_question": "Where is France located?",
})
)
As you can see, this approach is much more straightforward compared to the examples currently found in LangChain's documentation. I suspect they aim to facilitate more advanced use-cases while preventing users from having to relearn new APIs. This approach has its merits, but there are trade-offs involved.
LangChain is a robust and adaptable library. Once you become accustomed to this foundational example, you may find yourself exploring more complex patterns naturally.
Key Takeaways for Using LangChain
Preference for GPT-3.5 Over GPT-3
I estimate that around 95% of LangChain users incorporate OpenAI into their workflows. If you're testing and iterating with LangChain while calling OpenAI in the background, be cautious; your expenses can escalate quickly. I recommend defaulting to GPT-3.5, as it is significantly cheaper—ten times less than GPT-3 and fifteen times less than GPT-4.
However, GPT-3.5 does have its limitations:
- Slightly less powerful than its predecessors
- Smaller context window compared to GPT-4
- Slower response times (OpenAI is addressing scaling issues)
I've encountered instances where requests to GPT-3.5 took longer than 30 seconds, which is why I increased the request_timeout in the provided code example. If you require greater power, a larger context window, or quicker responses, consider switching models.
Multiple Methods to Instantiate a Chain
There are various ways to set up a chain, which can confuse users. I frequently utilize the method described above. Two options for LLMChain include:
- LLMChain.from_string: Bypass creating a PromptTemplate and directly use .from_string with two arguments: (1) the LLM instance and (2) a string prompt.
- LLMChain.__init__: First, create a prompt and LLM instance, then invoke .__init__.
Diverse Approaches to Call a Chain
LangChain generally prefers that you call and pass variables using dictionaries or keyword arguments. If keyword arguments aren't required, strings are passed and returned.
Here are six methods just for LLMChain:
- LLMChain.predict: Accepts only keyword arguments that correspond to prompt inputs and returns LLM string responses.
- LLMChain.__call__: Accepts a single positional argument (string or dictionary) and returns either a string or a dictionary response based on the output of your chain.
- LLMChain.run: Similar to LLMChain.__call__, but does not allow the keyword argument return_only_outputs.
- LLMChain.generate: Calls the LLM for each string prompt in the input list.
- LLMChain.apply: Calls LLMChain.generate and then refines the responses.
- LLMChain.predict_and_parse: Used when an output_parser is set; first calls predict, then parses the result.
I haven't even touched on the asynchronous methods for calling LangChain! I usually prefer LLMChain.__call__ because it meets all my needs while remaining user-friendly.
Bonus Example
The previously mentioned examples should cover most scenarios. However, those venturing into more advanced LangChain applications may find it challenging to track all prompt inputs and outputs. Here's an example that demonstrates how LangChain can effectively manage inputs and outputs internally:
from langchain import PromptTemplate
from langchain.chains import LLMChain, SequentialChain
from langchain.chat_models import ChatOpenAI
# Commonly reused LLM instance
LLM = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0.1, request_timeout=60)
# Chain to obtain an answer to the user's question
answer_question_chain = LLMChain(
llm=LLM,
prompt=PromptTemplate.from_template("""
You are a truthful and expert customer service agent.
Answer the user question directly based on the relevant documents.
If you could not find an answer, respond 'I do not know.'
* Relevant Documents *
{relevant_documents}
* User Question *
{user_question}
"""),
output_key="response_to_check",
)
# Verify if the answer is accurate based on the documents
double_check_chain = LLMChain(
llm=LLM,
prompt=PromptTemplate.from_template("""
You are a customer service agent reviewing another agent's work.
If the previous agent's response contradicts the relevant documents,
reply 'yes'.
If the previous agent's response aligns with the relevant documents,
reply 'no'.
* Relevant Documents *
{relevant_documents}
* User Question *
{user_question}
* Previous Agent's Response *
{response_to_check}
"""),
output_key="is_truthful_response",
)
# Combine both chains
answer_and_check_chain = SequentialChain(
chains=[answer_question_chain, double_check_chain],
input_variables=answer_question_chain.input_keys,
output_variables=[answer_question_chain.output_key, double_check_chain.output_key],
verbose=True,
)
# Sample execution
print(
answer_and_check_chain({
"relevant_documents": "* France is in my heart",
"user_question": "Where is France located?",
})
)
This more sophisticated application of LangChain is made possible by specifying additional keyword arguments, creating a PromptTemplate for each chain, and carefully tracking the meaning of each key and the chain that generated it.
Thank You!
I encountered some challenges in grasping all aspects of LangChain, but you don't have to face the same hurdles. This overview and the example patterns should set you on the right path. I hope this inspires you to create incredible projects using LangChain and LLMs!
Getting Started With LangChain In 20 Minutes - Build Celebrity Search Application
This video presents a step-by-step guide on how to utilize LangChain to build a celebrity search application, showcasing practical implementation strategies.