Your first job in Studio

Studio is an editor for the file you wrote in the quickstart. Same document, different input device.

1. Open Studio

   cd sparquet-studio
   npm install
   npm run dev

   Go to http://localhost:5273. The first launch creates a Getting Started Workflow with working Jobs — open one to look around, or follow along and build your own.

2. Create a Workflow and a Job

   A Workflow groups related Jobs; one per domain keeps names short. A Job is one pipeline JSON — the file the framework runs. Click New workflow, name it Sales, then New job inside it and call it Orders curated.

   You land on an empty canvas with the node palette on the left and the inspector on the right.

3. Add a source

   Drag CSV from Sources onto the canvas. The inspector opens on the right with the fields that format actually reads.

   Set Path to data/orders.csv. Leave the options alone — header and inferSchema are already on, matching the framework defaults.

4. Add transformations

   Drag Filter onto the canvas and connect the CSV node’s right handle to its left handle. In the inspector, set Condition to:

   status = 'CONFIRMED'

   Add Cast, With column and Drop duplicates the same way, chained left to right. The order on the canvas is the order in the transformations array.

   Tip

   Double-click empty canvas for a quick-add search box, or press / to focus the palette search. Every field carries an info button with the same notes you find in this reference.

5. Add quality rules

   Drag Validations onto the canvas and connect it after the last transformation. Set On failure to warn, then add two rules: not_null on id, and unique on id.

   Validations always run after every main transformation and before any write — the canvas position reflects that.

6. Add a destination

   Drag Parquet from Destinations, connect the validations node into it, and set:

   • Path: out/orders
   • Write mode: overwrite
   • Partition by: country

7. Read the file you just drew

   Press Ctrl/⌘+J.

   The JSON panel shows the compiled pipeline — exactly the document the framework runs. Copy it, download it, or commit it next to your job. Nothing is generated behind your back; what you see is what executes.

8. Check the linter

   Press Ctrl/⌘+E for the Issues panel. It runs while you type and catches the mistakes Spark only reports after minutes of compute: a merge without keys, a runtime variable nothing publishes, a parameter you never declared, an output writing to a path another output already claimed.

   Click an issue to jump to the node and field that caused it.

9. Run it

   Press Ctrl/⌘+Enter for the Run panel. If the local runner is not started yet, the panel shows the two commands you need:

   cd sparquet-studio
   pip install -r server/requirements.txt
   uvicorn server.main:app --port 8787

   The runner prints a token — paste it into the panel. Then Run executes the compiled JSON with the real framework and brings back row counts, per-rule validation results, a 50-row preview and the framework’s own logs.

The mapping, in one table

On the canvas	In the file
Source node	input
Transformation nodes on the shared chain	transformations
Validation rule nodes	validations.rules (one node per rule)
Destination node	output / outputs
Nodes after the graph forks	that destination’s own transformations
Second input of a join / union	with plus with_transformations
Sticky note	nothing — annotations never compile

Import what you already have

Have a pipeline JSON already? Open the JSON panel, switch to Edit, paste the file and click Apply to canvas. Studio lays it out and every node becomes editable. Unknown transformation types — custom ones you registered yourself — are preserved untouched through the round trip.

Next

• The canvas — connections, branching, keyboard control.
• AI assistant — describe a Job instead of drawing it.
• Pipelines — chain several Jobs into one sequential run.
• Core concepts — what runs when, and why the graph maps the way it does.