Skip to content
SparquetSparquet

AI assistant

The assistant writes Jobs in the language Studio already understands. Its system prompt is generated from the same catalog that drives the forms and the linter, so it cannot suggest a transformation the framework does not have.

  1. Open Settings → AI assistant.

  2. Pick a provider and paste a key:

    Provider Get a key Default model
    Anthropic console.anthropic.com claude-opus-5
    OpenAI platform.openai.com gpt-5.6-sol
    Google aistudio.google.com gemini-3.7-flash
    OpenAI-compatible your own gateway, Ollama, vLLM free text
  3. Optionally press Test connection — it sends a one-token ping and reports the result inline.

Open the panel with Ctrl/⌘+/. Pick an intent, describe what you want, send.

Intent What it does
Generate A complete Job from a description
Modify Changes the Job currently on the canvas
Explain Plain-English walkthrough of what the Job does
Fix issues Takes the linter’s findings and proposes corrections
Optimize Suggests checkpoints, pushdown and cheaper reads
Document Writes the description fields

Good prompts are specific about sources, keys and destinations:

Read orders from the Delta table sales.orders, keep the confirmed ones, aggregate revenue per customer, and upsert the result into analytics.customer_revenue on customer_id.

Add a data quality block that fails the run when id has nulls or duplicates, and write the report to quality.report as Delta.

The join is slow. Push the customer id list into the right side instead of reading the whole events table.

When a reply contains a pipeline JSON, it arrives as a card, not as an applied change:

  • Apply to canvas replaces the graph — and is a single Ctrl/⌘+Z away from being undone.
  • View JSON shows the raw document first.

An explanation that merely quotes a fragment does not produce a proposal — only a complete pipeline JSON does, which keeps a “what does this do?” answer from wiping your work.

The system prompt is generated from the catalog at build time and carries:

  • every transformation type with its required and optional keys
  • readable and writable formats, with their valid write modes
  • validators and the on_failure policies
  • the hard rules: agg is a list of SQL strings, select entries are expressions, binary is read-only, merge needs merge_keys, skip_if_false is a top-level key, {param} and {{variable}} are different mechanisms

Context is added per request: the compiled JSON, the current lint issues and the selected node — only when Share job context is on. Turn it off and nothing about your Job leaves the browser.

You pay your provider directly, at their rates. A generation is typically a few thousand tokens in and under a thousand out. Long conversations trim older turns rather than growing without bound.

The linter is the backstop: apply the proposal, read the Issues panel, and either fix it by hand or send the issues back with Fix issues. The assistant is a drafting tool — the file it produces is checked by the same rules as anything you write yourself.