GetTokenCount

Returns the token count for the specified text. Use for guidance only; actual counts used by models may vary.

Format 

GetTokenCount ( text )

Parameters 

text - any text expression.

Data type returned 

number

Originated in version 

21.0

Description 

AI models work with text as tokens, which may represent letters, words, or groups of words. Knowing how many tokens are in a text string can indicate whether the text may be too long for a specific model to process, or indicate the relative cost for the model to process the text.

Example 1 

GetTokenCount ( "Claris FileMaker" ) returns 4.

Example 2 

Gets embedding vectors for the Meeting::Note field only if the token count is less than or equal to 1024 and saves the data in the Meetings::Note_Embedding field for all records.

Copy 
Configure AI Account [ Account Name: "my-account" ; Model Provider: OpenAI ; API key: "sk-RZCtpWT..." ]

Go to Layout [ "Meeting Details" (Meetings) ; Animation: None ]
Show All Records

If [ GetTokenCount ( Meetings::Note ) > 1024 ]
    Show Custom Dialog [ "The token count is greater than 1024. Reduce the text in the Note field and try again." ]
Else
    Insert Embedding in Found Set [ Account Name: "my-account" ; 
    Embedding Model: "text-embedding-3-small" ; Source Field: Meetings::Note ; 
    Target Field: Meetings::Note_Embedding ; Replace target contents ]
End If

Related topics