The "Knowledge" feature in the SUPERWISE® platform enables SUPERWISE® to ingest and index external information (e.g., websites, documents) for contextual access within AI-driven applications. This allows you to seamlessly integrate additional information, such as website content, into your application. SUPERWISE® processes and embeds this content, enabling your application to access it directly and enhancing its contextual capabilities.

How to Create Knowledge via the UI:

Step 1: Add Knowledge

• Navigate to the Knowledge section in the sidebar.
• Choose the type of Knowledge you want to create from the available options: Website URL or File.

Step 2: Configure the Knowledge

Website URL

• Name: Assign a meaningful name to the Knowledge that will help you and your teammates easily identify it. Best practices include:
  • Avoid using periods (.)
  • Keep names short and descriptive
  • Ensure the name is unique across your environment
• URL: Enter the URL of the website you want to index. The information from this site will then be accessible within your application.
• Max Depth: Define the maximum number of links or levels, ranging from 1 to 5, that the crawler should follow from the initial page, limiting how deeply it explores the website’s structure
• Embedding Model: Select an embedding model to assist in indexing your data within a vector database. This model converts the data into a searchable format, enabling efficient retrieval and utilization.

👍

Good to know

A vector database is a system that stores data in a way that enables efficient searching and retrieval by converting information into 'vectors' or numerical representations. This allows the application to quickly and accurately access relevant content from the indexed data.

File

• Name: Assign a meaningful name to the Knowledge that will help you and your teammates easily identify it. This name will be visible when connecting it with your application. Avoid using periods (.) in the Knowledge name to prevent any technical issues or misunderstandings.
• Files: Select one or more PDF or TXT files you wish to index as a Knowledge object.
• Embedding Model: Select an embedding model to assist in indexing your data within a vector database. This model converts the data into a searchable format, enabling efficient retrieval and utilization.

❗️

PDF file limitations

• Each file should not exceed 10MB
• Currently, images within PDF files are not supported and will not be indexed.

Create Knowledge via the SDK:

To create a Knowledge the SDK, use the code snippet below as a guide and include the following details:

Website (URL):

• Name: Assign a meaningful name to the Knowledge that will help you and your teammates easily identify it. This name will be visible when connecting it with your application. Avoid using periods (.) in the Knowledge name to prevent any technical issues or misunderstandings.
• URL: Enter the URL of the website you want to index. The information from this site will then be accessible within your application.
• Max Depth: Define the maximum number of links or levels, ranging from 1 to 5, that the crawler should follow from the initial page, limiting how deeply it explores the website’s structure
• Embedding Model: Select an embedding model to assist in indexing your data within a vector database. This model converts the data into a searchable format, enabling efficient retrieval and utilization.

from superwise_api.models.tool.tool import OpenAIEmbeddingModel
from superwise_api.models.tool.tool import UrlKnowledgeMetadata

knowledge = sw.knowledge.create(
        name="Knowledge Name",
        knowledge_metadata=UrlKnowledgeMetadata(url="Insert URL here", max_depth='Number between 1-5'),
        embedding_model=OpenAIEmbeddingModel(provider="OpenAI", version="text-embedding-3-small", api_key="OPEN_AI_KEY"),
    )

In order to get all available embedding model providers, you can use this code:

from superwise_api.models.tool.tool import EmbeddingModelProvider

[provider.value for provider in EmbeddingModelProvider]

In order to get all available embedding model by the provider, you can use this code:

from superwise_api.models.tool.tool import OpenAIEmbeddingModelVersion, GoogleAIEmbeddingModelVersion

display([model_version.value for model_version in OpenAIEmbeddingModelVersion])
display([model_version.value for model_version in GoogleAIEmbeddingModelVersion])

File:

• Name: Assign a meaningful name to the Knowledge that will help you and your teammates easily identify it. This name will be visible when connecting it with your application. Avoid using periods (.) in the Knowledge name to prevent any technical issues or misunderstandings.
• Files: Select one or more PDF or TXT files you wish to index as a Knowledge object.
• Embedding Model: Select an embedding model to assist in indexing your data within a vector database. This model converts the data into a searchable format, enabling efficient retrieval and utilization.

❗️

PDF file limitations

• Each file should not exceed 10MB
• Currently, images within PDF files are not supported and will not be indexed.

from superwise_api.models.tool.tool import OpenAIEmbeddingModel
from superwise_api.models.tool.tool import FileKnowledgeMetadata


files = {"my_file": "path/to/my/file.txt"}
knowledge = sw.knowledge.create(
        name="Knowledge Name",
        knowledge_metadata=FileKnowledgeMetadata(),
        embedding_model=OpenAIEmbeddingModel(provider="OpenAI", version="text-embedding-3-small", api_key="OPEN_AI_KEY"),
        files=files,
    )

In order to get all available embedding model providers, you can use this code:

from superwise_api.models.tool.tool import EmbeddingModelProvider

[provider.value for provider in EmbeddingModelProvider]

In order to get all available embedding model by the provider, you can use this code:

from superwise_api.models.tool.tool import OpenAIEmbeddingModelVersion, GoogleAIEmbeddingModelVersion

display([model_version.value for model_version in OpenAIEmbeddingModelVersion])
display([model_version.value for model_version in GoogleAIEmbeddingModelVersion])

Re index knowledge:

When working with website knowledge, SUPERWISE® provides the option to re-index your content, ensuring your website information is current and accurate.

🚧

Pay attention!

Re-indexing website knowledge will erase the existing data.

Using the UI

Using the SDK

knowledge =  sw.knowledge.reindex_url_knowledge(Knowledge_ID)