How to Import CSV Into Databricks

Databricks documents several ways to move a CSV into a table. A workspace page takes up to ten files at a time, under two gigabytes in total, and turns them into a managed Delta table. A notebook reads one out of a Unity Catalog volume with spark.read. COPY INTO and Auto Loader pull files that already sit in cloud object storage. Every one of those starts from a seat that already holds the workspace. Your customer sits outside that seat, holding a claims file their service desk exported this morning.

The door a web backend has

Rows reach a Databricks table through SQL. The Statement Execution API takes a statement over HTTPS, runs it on a SQL warehouse, and answers with a statement id and a state. That is the whole surface a web backend gets, and everything below is built on it.

Two things are checked before the statement touches anything. Unity Catalog wants MODIFY on the table, and the documentation adds that the caller "must also have SELECT on the table, USE SCHEMA on the parent schema, and USE CATALOG on the parent catalog". The warehouse wants CAN USE. So the grants your data team already manages decide what your import route can write.

One thing is not checked. Databricks states that primary key, foreign key and unique constraints "are informational only and aren't enforced". A second row carrying a key the table already holds lands beside the first and nothing complains. NOT NULL and CHECK do hold, and a violated one fails the transaction. In how to import CSV into Snowflake the same job runs through a staging table. Here the identity work moves earlier, into the browser.

The files that arrive

Four repair partners send their April claims in the same week. Two of them look like this.

1Claim Ref,Dealer ID,Serial,Model,Fault,Repair Date,Labour Hrs,Parts,Currency,Status2WC-00811,NG-14,SN0041872,X200,No power,12/04/2026,1.5,48.20,EUR,approved3WC-00812,NG-14,SN0041905,X200 Pro,flickering screen,3/4/2026,0.75,0.00,EUR,pending4WC-00813,NG-14,SN0042110,X200,NP,22/04/2026,2,115.00,EUR,APPROVED⋮138 rows not shown143WC-00952,NG-14,SN0043918,X100,Battery fault,29/04/2026,1.25,62.40,EUR,approved

1Claim Ref,Servicer,Serial No.,Model,Fault Code,Date of repair,Labour hours,Parts cost,Curr,Status2WC-00811,VG-07,SN0067341,X200 Pro,H2O damage,2026-04-08,3.25,240.00,EUR,Rejected - no fault found3WC-00812,VG-07,SN0067402,X100,Battery fault,2026-04-19,0.5,31.90,EUR,approved⋮85 rows not shown89WC-00898,VG-07,SN0068890,X200,No power,2026-04-30,2,88.50,EUR,approved

Every partner names the same things differently. One says Dealer ID and the other says Servicer. One writes dates day first and the other writes them ISO. Curr, Labour Hrs and NP are somebody's local shorthand. H2O damage and flickering screen mean two of the four faults your catalogue holds. Both files carry a claim WC-00811, because each partner runs its own numbering.

The person drags all four files into the importer inside your app. Updog Importer reads them in the browser, matches each file's headers to your schema, holds the fault codes to your list, and puts every row in front of them. Your onComplete handler receives the rows grouped by file. The handler posts them to a route you own, and that route runs one MERGE.

No Updog server stands between the browser and Databricks.

The table the rows land in

One Delta table holds the claims, and the key is two columns wide.

create table main.service.warranty_claims
(
  partner_code  string not null,
  claim_ref     string not null,
  serial_number string,
  model         string,
  fault_code    string,
  repair_date   date,
  labour_hours  decimal(5, 2),
  parts_cost    decimal(10, 2),
  status        string,
  source_file   string,
  updated_at    timestamp,
  constraint warranty_claims_pk primary key (partner_code, claim_ref) rely
)
cluster by (partner_code, repair_date);

alter table main.service.warranty_claims
  add constraint labour_hours_sane check (labour_hours >= 0 and labour_hours <= 24);

The key constraint is declared for the optimizer and for the next person reading the schema. Databricks will not enforce it. The check constraint arrives in a second statement, because the CREATE TABLE grammar carries key constraints alone and points at ALTER TABLE for a check.

CLUSTER BY is liquid clustering, generally available for Delta tables from Databricks Runtime 15.4 LTS, and it refuses to combine with PARTITIONED BY. Clustering on the partner and the repair date matches how the table gets read, one partner at a time over a month.

The schema in Updog Importer

The columns array is the same table written for the person looking at the file.

import type { DataEditorColumn } from "@updog/data-editor";

const FAULTS = ["No power", "Screen flicker", "Battery fault", "Water damage"];
const STATUSES = ["Approved", "Rejected", "Pending"];
const CURRENCIES = ["EUR", "GBP", "USD"];

export const columns: DataEditorColumn[] = [
  {
    id: "partnerCode",
    title: "Partner code",
    size: 140,
    validators: [
      { type: "required" },
      { type: "regex", pattern: "^[A-Z]{2}-\\d{2}$" },
    ],
  },
  {
    id: "claimRef",
    title: "Claim ref",
    size: 140,
    validators: [
      { type: "required" },
      { type: "regex", pattern: "^WC-\\d{5}$" },
    ],
  },
  {
    id: "serialNumber",
    title: "Serial number",
    size: 150,
    transformer: (value) => String(value).trim().toUpperCase(),
  },
  { id: "model", title: "Model", size: 120 },
  {
    id: "faultCode",
    title: "Fault code",
    size: 170,
    editor: { type: "select", options: FAULTS, enableCustomValue: false },
    validators: [{ type: "oneOf", values: FAULTS }],
  },
  {
    id: "repairDate",
    title: "Repair date",
    size: 140,
    editor: { type: "date" },
    validators: [
      { type: "required" },
      { type: "date" },
    ],
  },
  {
    id: "labourHours",
    title: "Labour hours",
    size: 140,
    editor: { type: "number" },
    validators: [
      { type: "number", min: 0, max: 24, decimalPlaces: 2 },
    ],
  },
  {
    id: "partsCost",
    title: "Parts cost",
    size: 130,
    editor: { type: "number" },
    validators: [
      { type: "number", min: 0, decimalPlaces: 2 },
    ],
  },
  {
    id: "currency",
    title: "Currency",
    size: 120,
    editor: { type: "select", options: CURRENCIES, enableCustomValue: false },
  },
  {
    id: "status",
    title: "Status",
    size: 150,
    editor: { type: "select", options: STATUSES, enableCustomValue: false },
    validators: [{ type: "oneOf", values: STATUSES }],
  },
];

Each editor earns its place against what arrives. The date editor turns 22/04/2026 into 2026-04-22, and since 22 is above 12 that file settles day first, so 12/04/2026 and 3/4/2026 land as 2026-04-12 and 2026-04-03. The select editors hold the fault code and the status to a fixed list, and a value nobody maps is dropped from the row. Uniqueness stays off claimRef, because { type: "unique" } scores one column against the whole grid and both partners number their claims from the same series. Identity here is two columns wide, and primaryKey is where that lives.

The same column under a different header

Column matching repeats per file. Each file gets its own screen and its own mapping, so the same schema column is fed by a different header in each one.

Serial reaches serialNumber because the contains tier fires once the shorter string runs to four characters or more. Curr sits exactly on that floor and gets in. Dealer ID and Servicer get nothing from fuzzy matching at all, since neither shares a whole word with partnerCode and both sit too far away in edits, so one synonyms entry carries every partner's word for the same thing.

Value matching runs once for the whole import and collects values per schema column across all four files. flickering screen reaches Screen flicker on the word they share. H2O damage reaches Water damage the same way. NP is two characters, so it reaches nothing on its own and needs the synonym table. Rejected - no fault found contains rejected and lands at eighty.

Each file also settles its own dates. The verdict comes from a sample of that file alone, so a day-first export and an ISO export in the same import never borrow each other's reading.

The mount

The props tie the four files, the schema and the warehouse together.

<DataEditor<Claim>
  apiKey="your-license-key"
  open={open}
  onClose={closeEditor}
  columns={columns}
  primaryKey={["partnerCode", "claimRef"]}
  enableDeleteRow="all"
  blockSubmitOnError
  synonyms={{
    columns: {
      partnerCode: ["dealer id", "servicer", "asc code", "workshop"],
      claimRef: ["claim no", "job number"],
    },
    values: {
      "No power": ["np", "dead unit"],
      "Water damage": ["liquid ingress"],
    },
  }}
  onComplete={onComplete}
/>

primaryKey takes two columns, because a claim reference identifies a claim only inside one partner. Both files carry WC-00811, and with the partner beside it those stay two claims. Values are compared after trimming, and a row missing either part merges with nothing and arrives as new.

enableDeleteRow="all" lets the person drop a claim a partner sent twice by hand. blockSubmitOnError keeps submit disabled while any row carries an error. Whatever the person fixes by hand comes back on the result as learnedSynonyms, source-and-target pairs split into columns and values, which you store and fold into the synonyms tables next month.

Every snippet here is React. Those props reach Vue, Angular and Svelte through the web component build. The install and the modal wiring beneath this mount live in how to import a CSV file into a React app.

The result on submit

On submit, Updog Importer hands your handler every row grouped by source. Each file lands as its own entry, carrying the file name, so the row that came from Vantage still says so on the way out.

import type { DataEditorResult, ResultRow } from "@updog/data-editor";

const CHUNK_SIZE = 5_000;

const toRow = (entry: ResultRow<Claim>, sourceFile: string) => {
  if (entry.isDeleted && entry.isNew) return [];
  return [{
    partner_code: entry.row.partnerCode,
    claim_ref: entry.row.claimRef,
    serial_number: entry.row.serialNumber,
    model: entry.row.model,
    fault_code: entry.row.faultCode,
    repair_date: entry.row.repairDate,
    labour_hours: entry.row.labourHours,
    parts_cost: entry.row.partsCost,
    status: entry.row.status,
    source_file: sourceFile,
    op: entry.isDeleted ? "delete" : "upsert",
  }];
};

const onComplete = useCallback(async (result: DataEditorResult<Claim>) => {
  for (const source of result.sources) {
    const rows = [
      ...new Map(
        source.rows
          .flatMap((entry) => toRow(entry, source.sourceName))
          .map((row) => [row.partner_code + "\u0000" + row.claim_ref, row] as const),
      ).values(),
    ];

    for (let start = 0; start < rows.length; start += CHUNK_SIZE) {
      const written = await fetch("/api/claims/merge", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ rows: rows.slice(start, start + CHUNK_SIZE) }),
      });
      if (!written.ok) throw new Error((await written.json()).message);
    }
  }
}, []);

Three flags become one column. An insert and an update are the same payload, since the merge settles which one it is, and a delete is the same payload with op set, which happens for rows the editor loaded from your own table. A row the person added and then deleted goes nowhere. The Map keeps one row per partner-and-claim pair, so a chunk never carries the same key twice.

The chunk size belongs to you. Databricks publishes two ceilings here, sixteen mebibytes of query text and twenty-five mebibytes of inline result. Neither one binds this route, because the statement is a constant and the rows travel beside it as a parameter value, which carries no published ceiling. So five thousand rows is a number chosen against what your own endpoint accepts.

Throw when a route answers with a failure. Updog holds submit until your promise resolves, then empties the editor. A handler that traps the error and returns counts as a finished import, and four partners' claims clear the grid unwritten. A thrown error keeps the grid as it stands, with every mapping and hand correction on it. The person submits again on rows that never left the screen.

The route that merges

The route holds the credentials, and the browser stops there.

const CLAIM_SCHEMA =
  "array<struct<partner_code: string, claim_ref: string, " +
  "serial_number: string, model: string, fault_code: string, " +
  "repair_date: date, labour_hours: decimal(5,2), " +
  "parts_cost: decimal(10,2), status: string, " +
  "source_file: string, op: string>>";

const MERGE_CLAIMS =
  "merge into main.service.warranty_claims as t " +
  "using (select r.* from (select explode(from_json(:payload, '" +
  CLAIM_SCHEMA + "', map('mode', 'FAILFAST'))) as r)) as s " +
  "on t.partner_code = s.partner_code and t.claim_ref = s.claim_ref " +
  "when matched and s.op = 'delete' then delete " +
  "when matched then update set " +
  "  serial_number = s.serial_number, model = s.model, " +
  "  fault_code = s.fault_code, repair_date = s.repair_date, " +
  "  labour_hours = s.labour_hours, parts_cost = s.parts_cost, " +
  "  status = s.status, source_file = s.source_file, " +
  "  updated_at = current_timestamp() " +
  "when not matched and s.op = 'upsert' then insert " +
  "  (partner_code, claim_ref, serial_number, model, fault_code, " +
  "   repair_date, labour_hours, parts_cost, status, source_file, updated_at) " +
  "  values (s.partner_code, s.claim_ref, s.serial_number, s.model, " +
  "   s.fault_code, s.repair_date, s.labour_hours, s.parts_cost, " +
  "   s.status, s.source_file, current_timestamp())";

app.post("/api/claims/merge", async (request, response) => {
  const session = await getVerifiedSession(request);
  if (!session) return response.status(401).json({ message: "Not signed in" });

  const rows = request.body.rows;
  const allowed = await partnersFor(session.accountId);
  if (rows.some((row) => !allowed.has(row.partner_code))) {
    return response.status(403).json({ message: "A partner code is not yours" });
  }

  try {
    await runStatement(MERGE_CLAIMS, [
      { name: "payload", value: JSON.stringify(rows), type: "STRING" },
    ]);
  } catch (error) {
    request.log.error({ err: error });
    return response.status(502).json({ message: "The merge did not run" });
  }

  response.json({ merged: rows.length });
});

The rows never become SQL text. Databricks warns that generating SQL dynamically "can result in SQL injection attacks", and answers it with parameter markers, which handle input arguments "separately from the rest of your SQL code". This API takes named markers alone, each carrying one value cast to the type you name, so the whole chunk rides as one STRING and from_json rebuilds it inside the warehouse. The schema beside it is a constant your code owns, written the way CREATE TABLE writes types. FAILFAST makes a malformed payload raise MALFORMED_RECORD_IN_PARSING rather than quietly nulling the fields it could not read.

getVerifiedSession() stands in for your own server-side authentication check. The partner codes arrive in the body, so the route checks each one against the partners that session is allowed to load. A guessed code reaches no other manufacturer's rows.

Waiting for the statement

wait_timeout accepts 0s or five to fifty seconds, and defaults to ten. The route asks for fifty and takes what comes back. A merge that finishes inside the window arrives SUCCEEDED in the first response, and everything slower comes back PENDING with a statement id to poll. A serverless warehouse that auto-stopped after its default ten idle minutes is starting up inside that same wait.

The poll is where the trap sits. A warehouse that fails to run your statement answers HTTP 200 with status.state set to FAILED, and the error waits at status.error. A rejected request is a non-200, and a failed merge is not. So a route that reads response.ok alone reports a clean import over a table nothing was written to, and the grid clears.

All of this happens with the confirm dialog open and a spinner on the button. Anything on the result worth keeping gets copied inside the handler, since the editor drops its rows, its sources, its history and its learned synonyms once the promise resolves.

When the merge refuses

A merge fails as a statement. DELTA_MULTIPLE_SOURCE_ROW_MATCHING_TARGET_ROW_IN_MERGE is the one this import can produce, and Databricks raises it when more than one source row matches the same target row under the ON and WHEN MATCHED conditions. Two lines carrying the same partner and claim reference are exactly that input. The documentation asks the caller to preprocess the source, and the handler does it by keeping one row per partner-and-claim pair before the chunk leaves the browser.

There is no per-row failure list. The statement carries one error code and one message at status.error, populated only when the state is FAILED. The bad row is named by the message rather than by an index, which is why the checking sits in the browser where a row has a screen position and somebody to fix it.

The routes nobody ships for you

Updog Importer integrates with nobody. There is no Databricks connector, no destination list, no webhook and no server of ours. onComplete hands your code an object, and the routes in the middle are work you do.

Databricks already ships its own ways in for the other case. The upload page turns up to ten files into a table and states plainly that "Joining or merging records during file upload is not supported", which is the line between the two jobs. A volume takes a file up to five gigabytes through the UI, and larger ones through the Databricks SDK, up to what the underlying cloud storage holds. COPY INTO and Auto Loader keep object storage flowing in, and a notebook reads a CSV where it sits. For an export your own team downloaded, those tools are the shorter way in. Everything above exists for the four exports your partners sent, opened in a browser inside a session your app issued. Client-side and server-side CSV import names the jobs each of the two models fits.

The pieces you wrote and next month

You wrote one Delta table, a schema with ten columns, one synonym entry covering four partners' vocabularies, a token cache, a statement runner that polls, and one merge route. The files stay on the machine that opened them. The rows travel from your own front end to your own route and into Unity Catalog, and the only party you added to the chain is yourself. Point the same setup at a React CSV importer modal or at the web component and the middle stays the same.

May will bring the same four partners and a fifth one with a fifth spelling of Partner code. The mappings from April are already stored, the two-column key keeps the numbering apart, and the new partner costs one more line in synonyms.