Yes, you can craft your prompt in such a way that if the llm doesn’t know about a referenced legal document it will ask for it, so you can then paste the relevant section of that document into the prompt to provide it with that information.
I’d encourage you to look up some info on prompting LLMs and LLM context.
They’re powerful tools, so it’s good to really learn how to use them, especially for important applications like legalese translators and rent negotiators.
Indexing and tools like llamaindex use LLM generated embeddings to “intelligently” search for similar documents to a search query.
Those documents are usually fed into an LLM as part of the prompt (eg. context)