Start Batch Run

Start a batch evaluation run on all files in the collection.

POST /api/v1/eval_batch_runs

Start Batch Run

Python

import requests

url = "https://www.datalab.to/api/v1/eval_batch_runs"

payload = {
    "collection_id": 123,
    "eval_rubric_id": 123,
    "pipeline_type": "convert",
    "pipeline_id": "<string>",
    "page_schema": "<string>",
    "segmentation_schema": "<string>",
    "output_format": "<string>"
}
headers = {
    "X-API-Key": "<api-key>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

200

{
  "id": 123,
  "collection_id": 123,
  "eval_rubric_id": 123,
  "pipeline_id": "<string>",
  "pipeline_type": "<string>",
  "status": "<string>",
  "started_at": "2023-11-07T05:31:56Z",
  "completed_at": "2023-11-07T05:31:56Z",
  "total_files": 123,
  "completed_files": 123,
  "avg_score": 123,
  "created": "2023-11-07T05:31:56Z"
}

Authorizations

X-API-Key · string · header · required

Cookies

wos-session · string

datalab_active_team · string

Body

application/json

collection_id · integer · required

ID of the collection to run against

eval_rubric_id · integer | null

ID of the eval rubric to use. Required for standard pipelines, optional for custom pipelines (falls back to pipeline's eval_definition).

pipeline_type · enum<string> · default: convert

One of: convert, extract, segment

Available options: convert, extract, segment

pipeline_id · string | null

Optional custom pipeline ID

page_schema · string | null

Required when pipeline_type=extract

segmentation_schema · string | null

Required when pipeline_type=segment

output_format · string | null

Comma-separated output formats (e.g. 'markdown,json,html')

Response

200

application/json

Successful Response

id · integer · required

collection_id · integer · required

eval_rubric_id · integer | null · required

pipeline_id · string | null · required

pipeline_type · string · required

status · string · required

started_at · string<date-time> | null · required

completed_at · string<date-time> | null · required

total_files · integer · required

completed_files · integer · required

avg_score · number | null · required

created · string<date-time> | null · required