Every input and report SHA-256 hashed DON'T TRUST, VERIFY see the Evidence

Quick Start — PreFlight v1.01

PreFlight — Deterministic Upstream Data Ingestion Gate

pf_v1.01 delivers a first verdict in minutes, in the web app or through the API.

Purpose

Execute deterministic validation of dataset structure prior to ingestion.

When To Run

Run PreFlight immediately after dataset acquisition and before any ingestion step: database import, ETL load, warehouse ingestion, analytics processing, model training input.

Pipeline position: Download → PreFlight → Ingest

Data Handling

Datasets are deleted after evaluation. Data Handling.

Web Application

PreFlight runs from the web dashboard — this is the primary way to run a validation.

Run a validation:

  1. Go to Run Validation
  2. Upload your dataset (CSV)
  3. Submit the validation job

The report appears in History and Report View.

Verdict — every run returns one deterministic verdict:

  • PASS — no structural ingestion risk detected
  • WARN — structural exposure detected; review before ingestion
  • FAIL — deterministic ingestion failure detected
  • ANALYSIS_INCOMPLETE — insufficient structural evidence (dataset too small or narrow to assess: needs >= 30 logical data rows after the header, >= 2 columns, >= 20 non-empty logical rows).

The report includes full test results, triggering conditions (if any), and the final verdict.

Completion — PreFlight has evaluated the dataset and declared ingestion state. Proceed with your ingestion decision based on the verdict.

API / Integration

PreFlight's validation runs over a REST API — same engine and verdicts as the web app.

Authentication: send your key (Settings → API Access → Reveal) (requires an active subscription) on every request:

X-API-Key: <your-key>

Base URL: https://api.eolasdata.com

Prerequisites — bash example needs curl + jq; Windows needs curl (built into Windows 10+) and PowerShell 7+.

The flow — 6 steps. Each step's common errors and fixes are noted inline.

  1. Set your key, base URL, and file.
    • Use the key for the correct account and environment (a local/test key will be rejected by the live API).
    • File path errors — use the full, correct path to your CSV; quote paths that contain spaces. A wrong path fails the upload with a "file not found" error.
  2. Presign — POST /v1/uploads/presign → { url, fields, object_key, max_file_mb }
    • 401 "Missing API key or token" — set the X-API-Key header.
    • 403 "Invalid API key" — wrong key, or a different environment's key. Reveal the correct account's key in Settings.
  3. Upload — multipart POST to url, file LAST → 204
    • 400 "Key not specified" — file field must be last; include every fields entry.
    • 400 "EntityTooLarge" — exceeds your plan's max_file_mb; smaller file or upgrade.
    • Fails after a delay — presigned URLs expire in a few minutes; re-run Step 2.
  4. Create job — POST /v1/jobs/from-object?object_key=<object_key> → { job_id, status }
    • 404 "Dataset upload not found" — (a) object_key was URL-encoded; send literal slashes, don't pre-encode; or (b) too long between upload and create-job; run them promptly.
    • 403 "Dataset exceeds your plan size limit (N MB)" — smaller file or upgrade.
    • 403 "Run limit reached" — monthly cap; wait for reset (see Usage) or upgrade.
  5. Poll — GET /v1/jobs/id/<job_id> → { status } (poll until "complete")
    • status "failed" — re-submit. Always poll until "complete" before the result.
  6. Result — GET /v1/jobs/id/<job_id>/result → { job_id, result }
    • 400 "Job not complete" — poll Step 5 until "complete" first.
    • final_verdict "ANALYSIS_INCOMPLETE" — not an error; dataset too small/narrow (needs >= 30 logical data rows after the header, >= 2 columns, >= 20 non-empty logical rows). Run a fuller dataset.
    • result is a JSON object (already parsed) — read it directly; do not parse it a second time.

Complete example — bash / curl (needs jq)

# 1. set values
API_KEY="your-key"; BASE="https://api.eolasdata.com"; FILE="your_dataset.csv"

# 2. presign
PRESIGN=$(curl -s -X POST "$BASE/v1/uploads/presign" -H "X-API-Key: $API_KEY" \
  -H "Content-Type: application/json" -d "{\"filename\":\"$(basename "$FILE")\"}")
URL=$(echo "$PRESIGN" | jq -r .url); OBJECT_KEY=$(echo "$PRESIGN" | jq -r .object_key)

# 3. upload (fields first, file LAST)
ARGS=(); while IFS= read -r kv; do ARGS+=(-F "$kv"); done \
  < <(echo "$PRESIGN" | jq -r '.fields | to_entries[] | "\(.key)=\(.value)"')
curl -s -o /dev/null -w "upload: %{http_code}\n" -X POST "$URL" "${ARGS[@]}" -F "file=@$FILE"

# 4. create job
JOB_ID=$(curl -s -X POST "$BASE/v1/jobs/from-object?object_key=$OBJECT_KEY" -H "X-API-Key: $API_KEY" | jq -r .job_id)

# 5. poll until complete
while [ "$(curl -s "$BASE/v1/jobs/id/$JOB_ID" -H "X-API-Key: $API_KEY" | jq -r .status)" != "complete" ]; do sleep 3; done

# 6. result
curl -s "$BASE/v1/jobs/id/$JOB_ID/result" -H "X-API-Key: $API_KEY" | jq .result

Complete example — PowerShell (Windows, PowerShell 7+)

# 1. set values
$ApiKey = "your-key"; $Base = "https://api.eolasdata.com"; $File = "C:\path\to\your_dataset.csv"

# 2. presign
$presign = Invoke-RestMethod -Uri "$Base/v1/uploads/presign" -Method Post `
  -Headers @{ "X-API-Key"=$ApiKey; "Content-Type"="application/json" } `
  -Body (@{ filename = [IO.Path]::GetFileName($File) } | ConvertTo-Json)

# 3. upload (fields first, file LAST)
$c = @("-s","-o","NUL","-w","upload: %{http_code}","-X","POST",$presign.url)
$presign.fields.PSObject.Properties | ForEach-Object { $c += "-F"; $c += "$($_.Name)=$($_.Value)" }
$c += "-F"; $c += "file=@$File"
curl.exe @c

# 4. create job (curl keeps the object_key slashes literal)
$job = (curl.exe -s -X POST "$Base/v1/jobs/from-object?object_key=$($presign.object_key)" -H "X-API-Key: $ApiKey") | ConvertFrom-Json

# 5. poll until complete
do { Start-Sleep 3; $s = Invoke-RestMethod -Uri "$Base/v1/jobs/id/$($job.job_id)" -Headers @{ "X-API-Key"=$ApiKey } } while ($s.status -in @("pending","running"))

# 6. result
(Invoke-RestMethod -Uri "$Base/v1/jobs/id/$($job.job_id)/result" -Headers @{ "X-API-Key"=$ApiKey }).result | ConvertTo-Json -Depth 10

The result

result is the full report: final_verdict (PASS / WARN / FAIL / ANALYSIS_INCOMPLETE), triggering_tests, dataset_metadata (name, size, columns, delimiter, encoding, header), test_results[] (7 tests, each with verdict, evidence, metrics).

Usage

GET /v1/jobs/usage -> { credits_remaining, monthly_runs_used, monthly_runs_limit, reset }
See the evidence →
Access PreFlight Tool →