Trying to RAG with standard prompts and not getting very far..

[tomwilde]

Hi,

so I have the following chain already defined an working - it simply pulls the index from a document so a prompt and data but nothing else.

idx_chain = idx_prompt | llm | idx_parser

run using:

combined = RunnableParallel(
    index=index_chain
    ...
    )

and:

result = combined.invoke({"document_content": "some lengthy data here" })

The above works OK but I need this to work for large files so I RAG...

Having chunked my inbound file and built a VectorStoreRetriever I'm now attempting the following:

idx_chain = {"document_content": retriever | format_docs } | idx_prompt | llm | idx_parser

where:

def format_docs(docs:List[Document]):
    return "\n\n".join(doc.page_content for doc in docs)

ad its invoked with:

result = combined.invoke({})

This doesn't work... I get a long traceback that's not very useful.. But what I really need is a little guidance.. is this a valid approach? Where am I going awry?

thx

The exception detail if it helps...

  File "/app/.venv/lib/python3.12/site-packages/langchain_core/runnables/base.py", line 4000, in invoke
    output = {key: future.result() for key, future in zip(steps, futures)}
                   ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 59, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/langchain_core/runnables/base.py", line 3984, in _invoke_step
    return context.run(
           ^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/langchain_core/runnables/base.py", line 3243, in invoke
    input_ = context.run(step.invoke, input_, config, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/langchain_core/runnables/base.py", line 4000, in invoke
    output = {key: future.result() for key, future in zip(steps, futures)}
                   ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 59, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/langchain_core/runnables/base.py", line 3984, in _invoke_step
    return context.run(
           ^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/langchain_core/runnables/base.py", line 3243, in invoke
    input_ = context.run(step.invoke, input_, config, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/langchain_core/retrievers.py", line 263, in invoke
    result = self._get_relevant_documents(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/langchain_core/vectorstores/base.py", line 1067, in _get_relevant_documents
    docs = self.vectorstore.similarity_search(query, **kwargs_)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/langchain_core/vectorstores/in_memory.py", line 482, in similarity_search
    return [doc for doc, _ in self.similarity_search_with_score(query, k, **kwargs)]
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/langchain_core/vectorstores/in_memory.py", line 440, in similarity_search_with_score
    embedding = self.embedding.embed_query(query)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/langchain_together/embeddings.py", line 278, in embed_query
    response = self.client.create(input=text, **params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/openai/resources/embeddings.py", line 132, in create
    return self._post(
           ^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1259, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/openai/_base_client.py", line 1047, in request
    raise self._make_status_error_from_response(err.response) from None
openai.UnprocessableEntityError: Error code: 422 - {'id': 'oCRFWuY-4pPsy7-983c4d678cece57b', 'error': {'message': 'Unprocessable Entity', 'type': 'invalid_request_error', 'param': None, 'code': None}}

[tomwilde]

Apparently I just needed to sleep on this.. >.< I'll detail this for others below:

There's a couple of things going on here but mainly I just needed to reframe things

This was the underlying problem - no question supplied. Even though the prompt has all the info (including the question!) a question is still required to pass through the chain. Personally I think this is.. odd.

result = combined.invoke({})

Therefore the solution here was to supply a question and rework the prompt templates to accommodate that change. After that it worked just fine.