File: //usr/local/lib/python3.10/dist-packages/langchain/retrievers/__pycache__/ensemble.cpython-310.pyc
o
���g~) � @ s d Z ddlZddlmZ ddlmZ ddlmZ ddlm Z m
Z
m Z m Z m
Z
mZmZmZmZ ddlmZmZ ddlmZ dd lmZmZ dd
lmZ dd l m Z m Z dd l m Z m!Z! dd
l"m#Z# ed�Z$eded�Z%de e$ de
e$ge%f de
e$ fdd�Z&G dd� de�Z'dS )zo
Ensemble retriever that ensemble the results of
multiple retrievers by using weighted Reciprocal Rank Fusion
� N)� defaultdict)�Hashable)�chain) �Any�Callable�Dict�Iterable�Iterator�List�Optional�TypeVar�cast)�#AsyncCallbackManagerForRetrieverRun� CallbackManagerForRetrieverRun)�Document)�
BaseRetriever�
RetrieverLike)�RunnableConfig)�
ensure_config� patch_config)�ConfigurableFieldSpec�get_unique_config_specs)�model_validator�T�H)�bound�iterable�key�returnc c s6 � t � }| D ]}||� }|vr|�|� |V qdS )a
Yield unique elements of an iterable based on a key function.
Args:
iterable: The iterable to filter.
key: A function that returns a hashable key for each element.
Yields:
Unique elements of the iterable based on the key function.
N)�set�add)r r �seen�e�k� r$ �H/usr/local/lib/python3.10/dist-packages/langchain/retrievers/ensemble.py�
unique_by_key( s �
��r& c
@ sr e Zd ZU dZee ed< ee ed< dZe ed< dZ
e e ed< e
dee fd d
��Zed d �ed
ee ef defdd���Z d%de de e dedee fdd�Z d%de de e dedee fdd�Zde dedee fdd�Zde dedee fdd �Z dd �de dede e dee fd d �Z dd �de dede e dee fd d!�Z d"eee dee fd#d$�Z dS )&�EnsembleRetrieverae Retriever that ensembles the multiple retrievers.
It uses a rank fusion.
Args:
retrievers: A list of retrievers to ensemble.
weights: A list of weights corresponding to the retrievers. Defaults to equal
weighting for all retrievers.
c: A constant added to the rank, controlling the balance between the importance
of high-ranked items and the consideration given to lower-ranked items.
Default is 60.
id_key: The key in the document's metadata used to determine unique documents.
If not specified, page_content is used.
�
retrievers�weights�< �cN�id_keyr c C s t dd� | jD ��S )z+List configurable fields for this runnable.c s s � | ] }|j D ]}|V qqd S �N)� config_specs)�.0� retriever�specr$ r$ r% � <genexpr>Q s � � �z1EnsembleRetriever.config_specs.<locals>.<genexpr>)r r( ��selfr$ r$ r% r. N s �z EnsembleRetriever.config_specs�before)�mode�valuesc C s, |� d�st|d �}d| g| |d< |S )Nr) r( � )�get�len)�clsr7 � n_retrieversr$ r$ r% � set_weightsU s
z EnsembleRetriever.set_weights�input�config�kwargsc
K s� ddl m} t|�}|j|�d�d |�dd�|�dg �| j|�di �| jd�}|jd |fd |�d
�p5| �� i|��}z
| j |||d �}W n t
yW } z|� |� |�d }~ww |j |fi |�� |S ) Nr )�CallbackManager� callbacks�verboseF�tags�metadata�rC �inheritable_tags�
local_tags�inheritable_metadata�local_metadata�name�run_name�� run_managerr? )
�langchain_core.callbacksrA r � configurer9 rD rE �on_retriever_start�get_name� rank_fusion� Exception�on_retriever_error�on_retriever_end) r4 r> r? r@ rA �callback_managerrN �resultr" r$ r$ r% �invoke] s@
� ���
����zEnsembleRetriever.invokec
� s� �ddl m} t|�}|j|�d�d |�dd�|�dg �| j|�di �| jd�}|jd |fd |�d
�p6| �� i|��I d H }z
| j |||d �I d H }W n t
ya } z
|� |�I d H |�d }~ww |j |fi |��I d H |S ) Nr )�AsyncCallbackManagerrB rC FrD rE rF rK rL rM )
rO rZ r rP r9 rD rE rQ rR � arank_fusionrT rU rV ) r4 r> r? r@ rZ rW rN rX r" r$ r$ r% �ainvoke~ sF �
� �� ����� �zEnsembleRetriever.ainvoke�queryrN c C s | � ||�}|S )z�
Get the relevant documents for a given query.
Args:
query: The query to search for.
Returns:
A list of reranked documents.
)rS �r4 r] rN �fused_documentsr$ r$ r% �_get_relevant_documents� s z)EnsembleRetriever._get_relevant_documentsc � s �| � ||�I dH }|S )z�
Asynchronously get the relevant documents for a given query.
Args:
query: The query to search for.
Returns:
A list of reranked documents.
N)r[ r^ r$ r$ r% �_aget_relevant_documents� s �z*EnsembleRetriever._aget_relevant_documents)r? c sR � ��fdd�t | j�D �}tt|��D ]
}dd� || D �||< q| �|�}|S )z�
Retrieve the results of the retrievers and use rank_fusion_func to get
the final result.
Args:
query: The query to search for.
Returns:
A list of reranked documents.
c �6 g | ]\}}|� �t� �jd |d � �d�d���qS ��
retriever_r8 )�tag)rB )rY r � get_child�r/ �ir0 �r? r] rN r$ r% �
<listcomp>� s ����z1EnsembleRetriever.rank_fusion.<locals>.<listcomp>c S s* g | ]}t |t�rttt|�d �n|�qS �)� page_content)�
isinstance�strr r
�r/ �docr$ r$ r% rj � s ��)� enumerater( �ranger: �weighted_reciprocal_rank�r4 r] rN r? �retriever_docsrh r_ r$ ri r% rS � s � �
z EnsembleRetriever.rank_fusionc � s` �t j� ��fdd�t| j�D �� I dH }tt|��D ]
}dd� || D �||< q| �|�}|S )z�
Asynchronously retrieve the results of the retrievers
and use rank_fusion_func to get the final result.
Args:
query: The query to search for.
Returns:
A list of reranked documents.
c rb rc )r\ r rf rg ri r$ r% rj s ����z2EnsembleRetriever.arank_fusion.<locals>.<listcomp>Nc S s$ g | ]}t |t�st|d �n|�qS rk )rm r ro r$ r$ r% rj s ��)�asyncio�gatherrq r( rr r: rs rt r$ ri r% r[ � s ��
� �
z EnsembleRetriever.arank_fusion� doc_listsc s� t |�t �j�kr
td��tt�� t|�j�D ](\}}t|dd�D ] \}}� �jdu r.|jn|j �j ||�j
7 < q!qt � |�}t
t|�fdd��d� �fdd�d �}|S )
a�
Perform weighted Reciprocal Rank Fusion on multiple rank lists.
You can find more details about RRF here:
https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf
Args:
doc_lists: A list of rank lists, where each rank list contains unique items.
Returns:
list: The final aggregated list of items sorted by their weighted RRF
scores in descending order.
z<Number of rank lists must be equal to the number of weights.r8 )�startNc s � j d u r| jS | j� j S r- �r, rl rE �rp r3 r$ r% �<lambda>F s
� z<EnsembleRetriever.weighted_reciprocal_rank.<locals>.<lambda>Tc s" � �j d u r
| j S | j�j S r- rz r{ �� rrf_scorer4 r$ r% r| M s
�
�)�reverser )r: r) �
ValueErrorr �float�ziprq r, rl rE r+ r �
from_iterable�sortedr& )r4 rx �doc_list�weight�rankrp �all_docs� sorted_docsr$ r} r% rs s2 �
�
� ��
� �z*EnsembleRetriever.weighted_reciprocal_rankr- ) �__name__�
__module__� __qualname__�__doc__r
r �__annotations__r� r+ �intr, r rn �propertyr r. r � classmethodr r r= r r rY r\ r r` r ra rS r[ rs r$ r$ r$ r% r' 9 s�
����
�"����
�#��
���
�����
�.����
�,
��r' )(r� rv � collectionsr �collections.abcr � itertoolsr �typingr r r r r r
r r r
rO r r �langchain_core.documentsr �langchain_core.retrieversr r �langchain_core.runnablesr � langchain_core.runnables.configr r � langchain_core.runnables.utilsr r �pydanticr r r r& r' r$ r$ r$ r% �<module> s" , (