7️⃣LangChain vs. LlamaIndex

LLM 개발 프레임워크 중 가장 많이 사용하는 LangChain과 LlamaIndex를 간단하게 비교해 보겠습니다.

Item	LangChain	LlamaIndex
Management	LangChain	MetaAI
Date Founded	2023년 1월	2023년 4월
주요 서비스	LangChain, LangSmith, LangServe	LlmaIndex, LlamaCloud, LlamaParse
GithStars (2024.4 기준)	82K	30.6K

Item

LangChain

LlamaIndex

Management

LangChain

MetaAI

Date Founded

2023년 1월

2023년 4월

주요 서비스

LangChain, LangSmith, LangServe

LlmaIndex, LlamaCloud, LlamaParse

GithStars (2024.4 기준)

82K

30.6K

프레임워크 소개

LangChain, LLM으로 콘텐츠를 개발하기 위한 일반적인 프레임워크
LlamaIndex, RAG 시스템 구축 전용 프레임워크

코드 비교 항목

로컬 LLM 인스턴스에 연결하여 챗봇 구축하기.
로컬 파일 인덱싱 및 RAG 시스템 구축하기.
위의 두 가지를 결합하여 RAG 기능을 갖춘 챗봇 만들기.
챗봇을 에이전트로 전환하여 더 많은 도구를 사용하고 간단한 추론하

비교1. Chatbot with local LLM

LlamaIndex

from llama_index.llms import ChatMessage, OpenAILike  
  
llm = OpenAILike(  
	api_base="http://localhost:1234/v1",  
	timeout=600, # secs  
	api_key="loremIpsum",  
	is_chat_model=True,  
	context_window=32768,  
)  
chat_history = [  
	ChatMessage(role="system", content="You are a bartender."),  
	ChatMessage(role="user", content="What do I enjoy drinking?"),  
]  
output = llm.chat(chat_history)  
print(output)

LangChain

from langchain.schema import HumanMessage, SystemMessage  
from langchain_openai import ChatOpenAI  
  
llm = ChatOpenAI(  
	openai_api_base="http://localhost:1234/v1",  
	request_timeout=600, # secs, I guess.  
	openai_api_key="loremIpsum",  
	max_tokens=32768,  
)  
chat_history = [  
	SystemMessage(content="You are a bartender."),  
	HumanMessage(content="What do I enjoy drinking?"),  
]  
print(llm(chat_history))

비교2. RAG for local files

LlamaIndex

from llama_index import ServiceContext, SimpleDirectoryReader, VectorStoreIndex  
  
service_context = ServiceContext.from_defaults(  
	embed_model="local",  
	llm=llm, # This should be the LLM initialized in the task above.  
)  
documents = SimpleDirectoryReader(  
	input_dir="mock_notebook/",  
).load_data()  
index = VectorStoreIndex.from_documents(  
	documents=documents,  
	service_context=service_context,  
)  
engine = index.as_query_engine(  
	service_context=service_context,  
)  
output = engine.query("What do I like to drink?")  
print(output)

LangChain

from langchain_community.document_loaders import DirectoryLoader  
  
# pip install "unstructured[md]"  
loader = DirectoryLoader("mock_notebook/", glob="*.md")  
docs = loader.load()  
  
from langchain.text_splitter import RecursiveCharacterTextSplitter  
  
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)  
splits = text_splitter.split_documents(docs)  
  
from langchain_community.embeddings.fastembed import FastEmbedEmbeddings  
from langchain_community.vectorstores import Chroma  
  
vectorstore = Chroma.from_documents(documents=splits, embedding=FastEmbedEmbeddings())  
retriever = vectorstore.as_retriever()  
  
from langchain import hub  
  
# pip install langchainhub  
prompt = hub.pull("rlm/rag-prompt")  
  
  
def format_docs(docs):  
	return "\n\n".join(doc.page_content for doc in docs)  
  
  
from langchain_core.runnables import RunnablePassthrough  
  
rag_chain = (  
	{"context": retriever | format_docs, "question": RunnablePassthrough()}  
	| prompt  
	| llm # This should be the LLM initialized in the task above.  
)  
print(rag_chain.invoke("What do I like to drink?"))

비교 3. Combining two RAG chatbot

LlamaIndex: as_query_engine을 as_chat_engine으로 바꾸는 것만큼이나 간단

# Everything from above, till and including the creation of the index.  
engine = index.as_chat_engine()  
output = engine.chat("What do I like to drink?")  
print(output) # "You enjoy drinking coffee."  
output = engine.chat("How do I brew it?")  
print(output) # "You brew coffee with a Aeropress."

LangChain: 5단계의 절차가 필요

# Everything above this line is the same as that of the last task.  
from langchain_core.runnables import RunnablePassthrough, RunnableLambda  
from langchain_core.messages import get_buffer_string  
from langchain_core.output_parsers import StrOutputParser  
from operator import itemgetter  
from langchain.memory import ConversationBufferMemory  
from langchain.prompts.prompt import PromptTemplate  
from langchain.schema import format_document  
from langchain_core.prompts import ChatPromptTemplate  
  
memory = ConversationBufferMemory(  
return_messages=True, output_key="answer", input_key="question"  
)

LLM의 차례가 시작되면 메모리에서 채팅 기록을 로드합니다.

load_history_from_memory = RunnableLambda(memory.load_memory_variables) | itemgetter(  
	"history"  
)  
load_history_from_memory_and_carry_along = RunnablePassthrough.assign(  
	chat_history=load_history_from_memory  
)

LLM에게 Conetext을 더해 질문을 보강해 달라고 요청합니다:

rephrase_the_question = (  
	{  
		"question": itemgetter("question"),  
		"chat_history": lambda x: get_buffer_string(x["chat_history"]),  
	}  
	| PromptTemplate.from_template(  
	"""You're a personal assistant to the user.  
Here's your conversation with the user so far:  
{chat_history}  
Now the user asked: {question}  
To answer this question, you need to look up from their notes about """  
	)  
	| llm  
	| StrOutputParser()  
)

저희는 RAG 파이프라인을 실행합니다. "사용자가 직접 노트를 조회할 것"이라고 암시함으로써 LLM을 속이고 있지만, 사실 우리는 지금 무거운 작업을 LLM에 요청

retrieve_documents = {  
	"docs": itemgetter("standalone_question") | retriever,  
	"question": itemgetter("standalone_question"),  
}

"검색된 문서를 참고(선택 사항으로 지금까지의 대화)하여 사용자의 최근 질문에 대해 어떻게 답변하시겠습니까?"라고 LLM에게 질

def _combine_documents(docs):  
	prompt = PromptTemplate.from_template(template="{page_content}")  
	doc_strings = [format_document(doc, prompt) for doc in docs]  
	return "\n\n".join(doc_strings)  
compose_the_final_answer = (  
	{  
		"context": lambda x: _combine_documents(x["docs"]),  
		"question": itemgetter("question"),  
	}  
	| ChatPromptTemplate.from_template(  
	"""You're a personal assistant.  
	With the context below:  
	{context}  
	To the question "{question}", you answer:"""  
	)  
	| llm  
)

최종 응답을 채팅 기록에 추가

# Putting all 4 stages together...  
final_chain = (  
	load_history_from_memory_and_carry_along  
	| {"standalone_question": rephrase_the_question}  
	| retrieve_documents  
	| compose_the_final_answer  
)  
# Demo.  
inputs = {"question": "What do I like to drink?"}  
output = final_chain.invoke(inputs)  
memory.save_context(inputs, {"answer": output.content})  
print(output) # "You enjoy drinking coffee."  
inputs = {"question": "How do I brew it?"}  
output = final_chain.invoke(inputs)  
memory.save_context(inputs, {"answer": output.content})  
print(output) # "You brew coffee with a Aeropress."

비교 3. Upgrading to agents

LlamaIndex

# Everything above this line is the same as in the above two tasks,  
# till and including where `notes_query_engine` is defined.  
# Let's convert the query engine into a tool.  
from llama_index.tools import ToolMetadata  
from llama_index.tools.query_engine import QueryEngineTool  
  
notes_query_engine_tool = QueryEngineTool(  
	query_engine=notes_query_engine,  
	metadata=ToolMetadata(  
		name="look_up_notes",  
		description="Gives information about the user.",  
	),  
)  
from llama_index.agent import ReActAgent  
  
agent = ReActAgent.from_tools(  
	tools=[notes_query_engine_tool],  
	llm=llm,  
	service_context=service_context,  
)  
output = agent.chat("What do I like to drink?")  
print(output) # "You enjoy drinking coffee."  
output = agent.chat("How do I brew it?")  
print(output) # "You can use a drip coffee maker, French press, pour-over, or espresso machine."

LangChain

# Everything above is the same as in the 2nd task, till and including where we defined `rag_chain`.  
# Let's convert the chain into a tool.  
from langchain.agents import AgentExecutor, Tool, create_react_agent  
  
tools = [  
	Tool(  
		name="look_up_notes",  
		func=rag_chain.invoke,  
		description="Gives information about the user.",  
	),  
]  
react_prompt = hub.pull("hwchase17/react-chat")  
agent = create_react_agent(llm, tools, react_prompt)  
agent_executor = AgentExecutor.from_agent_and_tools(agent=agent, tools=tools)  
  
result = agent_executor.invoke(  
	{"input": "What do I like to drink?", "chat_history": ""}  
)  
print(result) # "You enjoy drinking coffee."  
result = agent_executor.invoke(  
	{  
		"input": "How do I brew it?",  
		"chat_history": "Human: What do I like to drink?\nAI: You enjoy drinking coffee.",  
	}  
)  
print(result) # "You can use a drip coffee maker, French press, pour-over, or espresso machine."

Conclusion

LangChain은 LLM 개발에 있어 다양한 모듈을 제공하여 종합적인 개발에 최적
LlamaIndex는 특히 RAG 구현 절차가 간단하여 RAG 서비스 개발에 있어 최적

PreviousLangChain with NeMo-Guardrails NextLlamaIndex Basic

Last updated 15 days ago