The Fireworks AI component is an AI component that allows users to connect the AI models served on the Fireworks AI Platform. It can carry out the following tasks:
#Release Stage
Alpha
#Configuration
The component definition and tasks are defined in the definition.yaml and tasks.yaml files respectively.
#Setup
In order to communicate with Fireworks AI, the following connection details need to be
provided. You may specify them directly in a pipeline recipe as key-value pairs
within the component's setup
block, or you can create a Connection from
the Integration Settings
page and reference the whole setup
as setup: ${connection.<my-connection-id>}
.
Field | Field ID | Type | Note |
---|---|---|---|
API Key | api-key | string | Fill in your Fireworks AI API key. To find your keys, visit the Fireworks AI API Keys page. |
#Supported Tasks
#Text Generation Chat
Fireworks AI's text generation models (often called generative pre-trained transformers or large language models) have been trained to understand natural language, code, and images. The models provide text outputs in response to their inputs. The inputs to these models are also referred to as "prompts". Designing a prompt is essentially how you “program” a large language model model, usually by providing instructions or some examples of how to successfully complete a task.
Input | Field ID | Type | Description |
---|---|---|---|
Task ID (required) | task | string | TASK_TEXT_GENERATION_CHAT |
Model Name (required) | model | string | The OSS model to be used. Enum values
|
Prompt (required) | prompt | string | The prompt text. |
System Message | system-message | string | The system message helps set the behavior of the assistant. For example, you can modify the personality of the assistant or provide specific instructions about how it should behave throughout the conversation. By default, the model’s behavior is set using a generic message as "You are a helpful assistant.". |
Prompt Images | prompt-images | array[string] | The prompt images (Note: According to Fireworks AI documentation on 2024-07-24, the total number of images included in a single API request should not exceed 30, and all the images should be smaller than 5MB in size). |
Chat History | chat-history | array[object] | Incorporate external chat history, specifically previous messages within the conversation. Please note that System Message will be ignored and will not have any effect when this field is populated. Each message should adhere to the format: : {"role": "The message role, i.e. 'system', 'user' or 'assistant'", "content": "message content"}. |
Seed | seed | integer | The seed. |
Temperature | temperature | number | The temperature for sampling. |
Top K | top-k | integer | Integer to define the top tokens considered within the sample operation to create new text. |
Max New Tokens | max-new-tokens | integer | The maximum number of tokens for model to generate. |
Top P | top-p | number | Float to define the tokens that are within the sample operation of text generation. Add tokens in the sample for more probable to least probable until the sum of the probabilities is greater than top-p (default=0.5). |
Input Objects in Text Generation Chat
Chat History
Incorporate external chat history, specifically previous messages within the conversation. Please note that System Message will be ignored and will not have any effect when this field is populated. Each message should adhere to the format: : {"role": "The message role, i.e. 'system', 'user' or 'assistant'", "content": "message content"}.
Field | Field ID | Type | Note |
---|---|---|---|
Content | content | array | The message content. |
Role | role | string | The message role, i.e. 'system', 'user' or 'assistant'. |
Content
The message content.
Field | Field ID | Type | Note |
---|---|---|---|
Image URL | image-url | object | The image URL. |
Text | text | string | The text content. |
Type | type | string | The type of the content part. Enum values
|
Image URL
The image URL.
Field | Field ID | Type | Note |
---|---|---|---|
URL | url | string | Either a URL of the image or the base64 encoded image data. |
Output | Field ID | Type | Description |
---|---|---|---|
Text | text | string | Model Output. |
Usage (optional) | usage | object | Token usage on the Fireworks AI platform text generation models. |
Output Objects in Text Generation Chat
Usage
Field | Field ID | Type | Note |
---|---|---|---|
Input Tokens | input-tokens | number | The input tokens used by Fireworks AI models. |
Output Tokens | output-tokens | number | The output tokens generated by Fireworks AI models. |
#Text Embeddings
An embedding is a list of floating point numbers that captures semantic information about the text that it represents.
Input | Field ID | Type | Description |
---|---|---|---|
Task ID (required) | task | string | TASK_TEXT_EMBEDDINGS |
Model Name (required) | model | string | The OSS embedding model to be used. Enum values
|
Text (required) | text | string | The text. |
Output | Field ID | Type | Description |
---|---|---|---|
Embedding | embedding | array[number] | Embedding of the input text. |
Usage (optional) | usage | object | Token usage on the Fireworks AI platform embedding models. |
Output Objects in Text Embeddings
Usage
Field | Field ID | Type | Note |
---|---|---|---|
Token Count | tokens | number | The token count used by Fireworks AI models. |
#Example Recipes
Recipe for the Fireworks Chinese Content Writer pipeline.
version: v1betacomponent: fireworks-0: type: fireworks-ai task: TASK_TEXT_GENERATION_CHAT input: max-new-tokens: 200 model: qwen2-72b-instruct prompt: ${variable.prompt} system-message: You are a expert social media content writing assistant. Output the result in chinese for 小紅書. temperature: 0.05 top-k: 10 top-p: 0.5 setup: api-key: ${secret.INSTILL_SECRET}variable: prompt: title: prompt description: input prompt i.e. "寫一份生椰拿鐵文案", "write about horses" type: stringoutput: output: title: output value: ${fireworks-0.output.text}