Chatting with AI models

The Chat tab allows you to test chatting with a downloaded text generation model while in Admin Console.

Important  You must first enable the model server before the Chat tab will be available. You must download a text generation model before the chat feature will be usable. You must enable and configure RAG to use the advanced RAG feature in a chat session. Some models require you to include a Hugging Face token in the Model Server tab to download the model. See Starting, stopping, and updating the AI model server.

To chat:

  1. Click the AI Services > Chat tab.

  2. Enter text into the message box.

    Note  You can attach or drop images into chat messages for supported multimodal models.

  3. Press Enter or Return, or click ➤.

To use RAG for a chat session:

  1. Click the AI Services > Chat tab.

  2. Click Advanced Settings.

  3. Select Enable RAG to enable the setting.

  4. Enter the RAG API Key.

  5. Enter the RAG Space ID.

  6. Enter a descriptive RAG Prompt Template.

    Note  The RAG Prompt Template is the placeholder text in the textbox. If the "{question}" and "{context}" macros are not specified in the template, the default template is used.

  7. (Optional) Click Advanced Settings to hide the settings.

To manage the chat history:

  1. Click the AI Services > Chat tab.

  2. Click chat history icon.

  3. To start a new chat, click + New Chat.

  4. To return to a previous chat session click the chat in the Chat History.