Move LLM-related components into LLM module (#74)

* Move splitter into indexing module
* Rename post_processing module to parsers
* Migrate LLM-specific composite pipelines into llms module

This change moves the `splitters` module into `indexing` module. The `indexing` module will be created soon, to house `indexing`-related components.

This change renames `post_processing` module into `parsers` module. Post-processing is a generic term which provides very little information. In the future, we will add other extractors into the `parser` module, like Metadata extractor...

This change migrates the composite elements into `llms` module. These elements heavily assume that the internal nodes are llm-specific. As a result, migrating these elements into `llms` module will make them more discoverable, and simplify code base structure.
This commit is contained in:
Nguyen Trung Duc (john)
2023-11-15 16:26:53 +07:00
committed by GitHub
parent 9945afdf6f
commit f8b8d86d4e
13 changed files with 41 additions and 35 deletions

View File

@@ -1,8 +1,9 @@
from langchain.schema.messages import AIMessage, SystemMessage
from .chats import AzureChatOpenAI, ChatLLM
from .chats.base import BaseMessage, HumanMessage
from .branching import GatedBranchingPipeline, SimpleBranchingPipeline
from .chats import AzureChatOpenAI, BaseMessage, ChatLLM, HumanMessage
from .completions import LLM, AzureOpenAI, OpenAI
from .linear import GatedLinearPipeline, SimpleLinearPipeline
from .prompts import BasePromptComponent, PromptTemplate
__all__ = [
@@ -20,4 +21,9 @@ __all__ = [
# prompt-specific components
"BasePromptComponent",
"PromptTemplate",
# strategies
"SimpleLinearPipeline",
"GatedLinearPipeline",
"SimpleBranchingPipeline",
"GatedBranchingPipeline",
]