Configuring AI settings

The settings allow you to control which AI model server components are enabled, where the models will be downloaded, which models are preloaded, and various RAG and model fine-tuning settings that enhance AI model communication.

To modify settings:

  1. Click the AI Services > Settings tab.

  2. Make changes to any of the following options:

    • Services: control which services are provided by the model server.

      • Embedding

      • Generation - for text generation

      • RAG

      • Fine-Tuning

        Note  This option is only availabe for systems with hardware that support fine-tuning models.

      • Image Caption Generation

    • Server Configuration: configure how the model server operates.

      • Require API Key for Access

      • Preload Models - load the models specified in the Preloaded Models section

      • Load Previous Models - load the models previously loaded before the AI model server shutdown

      • Log Token Usage

      • Log Model Prompts and Responses

    • Preloaded Models: control which models to load when the model server starts.

      • Preloaded Embedding Model

      • Preloaded Text Generation Model

    • RAG Settings: control options for retrieval-augmented generation (RAG) on the model server.

      • Use Cosine Similarity Threshold for RAG Results

      • Cosine Similarity Threshold for RAG Results

        Note  This option is not displayed until you enable Use Cosine Similarity Threshold for RAG Results.

      • Number of Top-Ranked Results for RAG Summarization

      • RAG Cache Path

      • Use OpenAI Embeddings

      • OpenAI API Key for RAG - enter an OpenAI API key to access an OpenAI RAG resource

        Note  This option is not displayed until you enable OpenAI Embedding Model for RAG.

      • OpenAI Embedding Model for RAG

      • Batch Size for Creating RAG Embeddings

      • Number of Tokens per Text Chunk - default size of new text chunks when adding text or files to a RAG space; to resize existing chunks, you must delete and recreate the RAG space, then add data again

      • Do RAG Sync - the button gives you access to RAG sync settings; see Perform RAG Sync

      Note  You must click Show Advanced Settings to see this section.

    • Fine-Tuning Settings: control options for fine-tuning models.

      • System Prompt for Fine-Tuning

      Note  This section is only available for systems with hardware that support fine-tuning models. You must enable the Fine-Tuning option and click Show Advanced Settings to see this section.