Insert Image Captions in Found Set

For every record in the found set, sends an image from a source field to an image captioning model and inserts the returned caption into a target field.

Options 

• Account Name is a text expression for the name of the AI account to use. In the current file, use the Configure AI Account script step to set up the account and assign it this name any time before this script step runs.

• Model is the name of the image captioning model. Specify the model name as a text expression. For supported models, see FileMaker technical specifications.

• Source Field is the container field whose images this script step sends to the model for caption generation. If the field is empty, nothing is sent to the model and the target field is not changed.

• Target Field specifies the text field to insert the returned captions into.

• Replace target contents replaces the target field's contents, if there is any. When this option is off, results are added to the target field only if the field is empty; otherwise, the field is left unchanged. Turning on this option is useful if the contents of Source Field changes frequently.

• Continue on error continues processing the found set even after errors are encountered in some records. With this option selected, records for which an error occurred remain empty, making them easier to find and address. If not selected, the script step stops when an error occurs.

• Parameters is a text expression for a JSON object that can include FileMaker-specific parameters that specify limits on the size, number, and frequency of requests sent to the model. Use these to optimize your script depending on the amount of data, the model being used, the model provider's token limits, and the technical specifications (cores, RAM, and so on) of the machine the model is running on.

  See Description for details.

Originated in version 

26.0

Description 

Image captioning models generate text descriptions of images. The caption returned depends on the model used and the content of the image.

This script step is supported only for Claris AI Model Server as the model provider. (See Configuring AI services in FileMaker Server Help.) Set the Account Name option to an account configured by the Configure AI Account script step for a Custom model provider whose Endpoint is your AI model server. For supported image captioning models, see FileMaker technical specifications.

For every record in the current found set, this script step sends the image in Source Field to the specified model for caption generation and saves the returned caption in Target Field.

For the Parameters option, you can use the following FileMaker-specific keys and values in a JSON object to adjust this script step's behavior, if needed. If a key isn't specified or the Parameters option isn't used, the default values are used.

Note  If using the JSONSetElement function to create the JSON object, use the JSONNumber constant for the type parameter.

For example, this JSON object sets these keys in the Parameters option:

{
    "MaxRecPerCall" : 40,
    "MaxRetryPerWorker" : 10
}

Notes 

• To automate updating the Target Field in a hosted FileMaker Pro file on a regular basis, you can set up a schedule on the host to run your FileMaker script without user interaction. See FileMaker script schedules in FileMaker Server Help or FileMaker Cloud Help.

Example 1 - Generate captions for all photos

Configures an AI account for Claris AI Model Server, goes to the Products layout, shows all records, then generates captions for the images in the Products::Image field and stores them in the Products::Caption text field. Because the Replace target contents option is on, Products::Caption is updated even if it already contains data.

Configure AI Account [ Account Name: "my-account" ; Model Provider: Custom ; Endpoint: "https://my-server.example.com/llm/v1/" ; Verify SSL Certificates ; API key: Global::API_Key ]

Go to Layout [ "Products" (Products) ; Animation: None ]
Show All Records

Insert Image Captions in Found Set [ Account Name: "my-account" ; Model: "Salesforce/blip2-opt-2.7b" ; Source Field: Products::Photo ; Target Field: Products::Caption ; Replace target contents ]

Example 2 - Generate missing captions for photos in found set

Configures an AI account for Claris AI Model Server, goes to the Product layout, then finds the records in which the Products::Category field contains "Hand Tools." For this found set, generates captions for the images in the Products::Photo field and stores them in the Products::Caption text field, if there's no data already present (Replace target contents is off). If an error is encountered when generating a caption for a record, the script step continues to the next record.

Configure AI Account [ Account Name: "my-account" ; Model Provider: Custom ; Endpoint: "https://my-server.example.com/llm/v1/" ; Verify SSL Certificates ; API key: Global::API_Key ]

Go to Layout [ "Products" (Products) ; Animation: None ]
Enter Find Mode [ Pause: Off ]
Set Field [ Products::Category ; "Hand Tools" ]
Perform Find [ ]

Insert Image Captions in Found Set [ Account Name: "my-account" ; Model: "Salesforce/blip2-opt-2.7b" ; Source Field: Products::Photo ; Target Field: Products::Caption ; Continue on error ; Parameters: JSONSetElement ( "" ;
    ["MaxRecPerCall" ; 40 ; JSONNumber] ; 
    ["MaxRetryPerWorker" ; 10 ; JSONNumber]
) ]

Related topics 

• Insert Image Caption script step

• Script steps reference