SIGN IN SIGN UP

community[feat]: Adds LLMLingua as a document compressor (#17711)

**Description**: This PR adds support for using the [LLMLingua project
](https://github.com/microsoft/LLMLingua) especially the LongLLMLingua
(Enhancing Large Language Model Inference via Prompt Compression) as a
document compressor / transformer.

The LLMLingua project is an interesting project that can greatly improve
RAG system by compressing prompts and contexts while keeping their
semantic relevance.

**Issue**: https://github.com/microsoft/LLMLingua/issues/31
**Dependencies**: [llmlingua](https://pypi.org/project/llmlingua/)

@baskaryan

---------

Co-authored-by: Ayodeji Ayibiowu <[email protected]>
Co-authored-by: Harrison Chase <[email protected]>
A
Ayo Ayibiowu committed
ac1d7d9de81997e31eb6982e303ef5294641f686
Parent: a99eb3a
Committed by GitHub <[email protected]> on 2/28/2024, 3:23:56 AM