utils.func_retrieval

module documentation

(source)

Undocumented

Function	`search_documents`	Search documents based on the given processes.
Function	`search_engine`	Search engine function that takes a collection of documents and a list of keywords, and returns a sorted list of documents ids based on their similarity to the keywords and a score this is a custom IT-IDF.

def search_documents(processes: list[dict]) -> Document: (source) ¶

Search documents based on the given processes. Args: processes (list): A list of dictionaries containing the process details. Each dictionary should have the following keys: - type (str): The type of process to be performed. - keyword (str): The keyword to search for. - quantity (int, optional): The maximum number of results to retrieve. Defaults to 10. Example: processes = [ {'type':'reddit', 'keyword':'MachineLearning'}, {'type':'arxiv', 'keyword':'machine learning'} ] Returns: api_results (Document): The retrieved documents. Raises: ValueError: If no data is provided or if the credentials are incorrect. TypeError: If there is a type error.

def search_engine(collection: list, keywords: list): (source) ¶

Search engine function that takes a collection of documents and a list of keywords, and returns a sorted list of documents ids based on their similarity to the keywords and a score this is a custom IT-IDF. Args:: collection (list): A list of documents. keywords (list): A list of keywords. Returns: list: A sorted list of dictionaries containing the document URL, similarity score, and document text.