> For the complete documentation index, see [llms.txt](https://docs.imerit.net/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.imerit.net/plugins/first-party-ango-plugins/chatgpt.md).
# ChatGPT
## Overview
The ChatGPT Plugin enables seamless integration of OpenAI’s language and multimodal models into your annotation workflows on Ango Hub. This plugin is designed to support a wide range of use cases, including text processing, image understanding, named entity recognition (NER), and audio processing.
## Plugin Functionality
The ChatGPT Plugin processes assets and generates outputs based on the selected mode, model, and prompt configuration.
* **Text Processing:** Generate summaries, classifications, structured outputs, or rewritten content from text inputs.
* **Image Understanding:** Analyze visual content and generate descriptions, extract text (OCR-style), or answer questions about images.
* **Named Entity Recognition (NER):** Identify and extract structured entities such as names, organizations, locations, and dates from text.
* **Audio Processing:** Transcribe, summarize, or analyze audio content using prompt-based instructions.
#### Prompt Construction
The plugin dynamically constructs the final prompt using:
```
[prompt_prefix] + [input content] + [prompt_suffix]
```
* **prompt\_prefix** defines the task or instruction
* **Input content** comes from the asset or annotation
* **prompt\_suffix** refines output format or constraints
This flexible structure allows you to standardize instructions across all assets.
### Supported Modes
#### Workflow-1: Text Processing (Asset)
{
"mode": "text-prompt",
"model_name": "gpt-4.1-nano",
"openai_api_key": "<YOUR_OPENAI_API_KEY>",
"prompt_prefix": "What is the capital of ",
"prompt_suffix": "? Provide only the city name."
}
```json
{
"mode": "ner",
"model_name": "gpt-4.1-nano",
"openai_api_key": "",
"prompt_prefix": "",
"prompt_suffix": ""
}
```
{% hint style="info" %}
In the "ner" mode, class mapping is not required. The plugin automatically assigns entity labels based on the project's category schema, so no manual mapping configuration is needed.
{% endhint %}
Category Schema
```json
{
"tools": [
{
"schemaId": "796d1292cfbf6229e793571",
"tool": "ner",
"title": "Adjective",
"required": false,
"classifications": [],
"multiple": false,
"color": "#f44336",
"shortcutKey": "1"
},
{
"schemaId": "fd74cf3157e7805f080f405",
"tool": "ner",
"title": "Verb",
"required": false,
"classifications": [],
"multiple": false,
"color": "#03a9f4",
"shortcutKey": "2"
},
{
"schemaId": "c956228f2b8e2effebac917",
"tool": "ner",
"title": "Noun",
"required": false,
"classifications": [],
"multiple": false,
"color": "#9c27b0",
"shortcutKey": "3"
}
],
"classifications": [],
"relations": []
}
```
#### Workflow-5: Speech-to-Text
```json
{
"mode": "audio-prompt",
"model_name": "whisper-1",
"openai_api_key": "",
"prompt_prefix": "",
"prompt_suffix": ""
}
```
Category Schema
```json
{
"tools": [
{
"schemaId": "8280042f0f98b800b6cc860",
"tool": "ner",
"title": "Transcription Entity",
"required": false,
"classifications": [
{
"schemaId": "f9a4e85dc1d215022e31013",
"tool": "text",
"title": "Transcription Text",
"required": false,
"classifications": [],
"multiple": false,
"options": [],
"parentToolType": "ner",
"parentToolId": "8280042f0f98b800b6cc860",
"shortcutKey": "3",
"frameSpecific": false,
"richText": false,
"katex": false,
"katexBottom": true,
"regex": ""
}
],
"multiple": false,
"color": "#f44336",
"shortcutKey": "1"
}
],
"classifications": [
{
"schemaId": "0912500394b493f68cdd406",
"tool": "text",
"title": "Full Transcription Text",
"required": false,
"classifications": [],
"multiple": false,
"options": [],
"shortcutKey": "2",
"frameSpecific": false,
"richText": false,
"katex": false,
"katexBottom": true,
"regex": ""
}
],
"relations": []
}
```
### Supported Data Types
* Audio (*audio-prompt*)
* Image (*image-prompt*)
* Text (*NER, text-prompt+asset source*)
* Any Data Type (*text-prompt+text annotation source*)
### Supported Annotation Tools
* Text
* Entity
* Radio
* Checkbox
* Single-Select Dropdown
* Multi-Select Dropdown
## Plugin Configuration
The **Overwrite** setting in model plugins controls whether existing annotations are replaced or kept. When enabled, the plugin replaces all existing annotations with new model predictions; when disabled, it simply adds the new results without deleting what’s already there.
In the **class mapping**, the ChatGPT plugin uses two fields: “Prompt Body” and “Reply Text”.
* **Prompt Body** refers to the text tool from which the main content of the prompt is retrieved.
* **Reply Text** refers to the text tool where the ChatGPT response will be written.
ChatGPT plugin integration to the workflow
You may vary a number of settings related to the model plugin from the **Config JSON** field. Each option is detailed below:
```json
{
"mode": "text-prompt",
"model_name": "gpt-4.1-nano",
"openai_api_key": "",
"prompt_prefix": "",
"prompt_suffix": "",
"model_parameters": {
"reasoning_effort": "none",
"temperature": 0.7,
"max_output_tokens": 1000
}
}
```
* **"mode":** Defines the operating mode of the plugin. The selected mode determines the type of input the plugin processes and the kind of task it performs.
* Options:
* "text-prompt"
* "image-prompt"
* "ner"
* "audio-prompt"
* Example:
* "mode": "text-prompt"
* **"model\_name":** Specifies the OpenAI model that the plugin will use for inference. See the OpenAI [models](https://developers.openai.com/api/docs/models) and [pricing](https://developers.openai.com/api/docs/pricing) pages for available options and details.
* Options:
* "gpt-5.4"
* "gpt-5.4-mini"
* "gpt-5.4-nano"
* "gpt-5.4-pro"
* ...
* Example:
* "model\_name": "gpt-4.1-nano"
* **"openai\_api\_key":** Specifies the OpenAI API key used to authenticate requests sent by the plugin. You can generate and manage your API keys from the [OpenAI dashboard](https://platform.openai.com/settings/organization/api-keys).
* Example:
* "openai\_api\_key": "\"
* **"prompt\_prefix":** A text string added before the main input content when constructing the prompt sent to the model.
* Example:
* "prompt\_prefix": "Extract all company names from the following text:"
* **"prompt\_suffix":** A text string added after the main input content when constructing the prompt sent to the model.
* Example:
* "prompt\_suffix": "Return the result as a JSON array."
* **"reasoning\_effort":** Controls how much internal reasoning the model performs before generating a response. Higher reasoning effort can improve performance on complex tasks but may increase latency and token usage.
* Options:
* "none"
* "minimal"
* "low"
* "medium"
* "high"
* "xhigh"
{% hint style="danger" %}
Not all OpenAI models support reasoning.
{% endhint %}
{% hint style="warning" %}
Enabling `reasoning_effort` disables temperature control. OpenAI reasoning models determine their own reasoning strategy and do not allow customization of the `temperature` parameter. Any provided temperature value will be ignored.
{% endhint %}
* **"temperature":** Controls the randomness and creativity of the model’s responses.
* Example:
* "temperature": 0.7
* **"max\_output\_tokens":** Defines the maximum number of tokens that the model can generate in its response.
* Example:
* "max\_output\_tokens": 1000
