Skip to main content

Dataset API Overview

Looking for the product view?
The Help Center explains this surface as users see it: Working with Datasets.

The Informer Dataset API provides comprehensive endpoints for creating, managing, and querying Datasets. Datasets are the core data structures in Informer: records loaded from a Datasource query, an uploaded file, or pushed in externally, indexed in Elasticsearch for search, aggregation, and export. All routes are prefixed with /api.

Features​

  • CRUD Operations - Create, read, update, and delete Datasets
  • Data Management - Import, append, replace, and clear Dataset records
  • Field Operations - Add, update, and remove fields with type mapping
  • Drafts & Versioning - Create draft copies for safe editing without affecting production
  • Filters - Create saved filters and apply complex query logic
  • Aggregations - Group and aggregate data with hierarchical structures
  • Visuals - Manage charts and visual representations
  • Copy & Snapshots - Create point-in-time copies and promote snapshots
  • Import & Export - Support for multiple formats (CSV, Excel, PDF, etc.)
  • Sharing & Permissions - Team-based access control with field-level restrictions
  • Flow Builder - Multi-step data transformation pipelines
  • Elasticsearch Integration - Full-text search, mapping management, index operations

Dataset Types​

The type filter on GET /api/datasets groups Datasets by where their data comes from:

TypeDescription
datasourceLoaded by running a query against a connected Datasource
uploadCreated from an uploaded file (CSV, Excel, …); backed by an internal embedded datasource
externalNo Datasource; records are pushed in via the data API

Authentication & Permissions​

All Dataset API endpoints require authentication via session cookies or API tokens.

  • Read access is not a permission. Every route resolves the Dataset through a read-access scope: a Dataset you cannot read answers 404, never 403, so its existence stays private. Read access comes from ownership, team membership, or sharing.
  • dataset:write gates updates, deletes, field changes, params, and data modifications. It requires the Data Wizard role or above on the owning team; most other write-style permissions (edit, delete, rename) delegate to it.
  • datasets:create gates POST /api/datasets. Without a query.language it requires the Designer role or above; with one, the query-language driver authorizes the call (Data Wizard or above unless the driver overrides).
  • Per-action permissions (share, copy, refresh, export, …) are documented on their endpoint pages, and every Dataset response carries a permissions object with your resolved flags.

Elasticsearch Integration​

Datasets store their data in Elasticsearch indices. Each Dataset has:

  • esIndex - Unique Elasticsearch index name (rebuilt on refresh)
  • Mapping - Field type definitions and analyzer settings
  • Search API - Full-text search with aggregations and filters

Drafts & Versioning​

Datasets support draft mode for safe definition editing:

  1. Create draft - POST /api/datasets/{id}/_edit
  2. Modify draft - Make changes without affecting production
  3. Commit draft - POST /api/datasets/{id}/draft/_commit to publish changes

Common Query Parameters​

List endpoints built on the standard collection handler support:

ParameterTypeDefaultDescription
qstring—Case-insensitive prefix search; embed * wildcards for contains
sortstring—Attribute to sort by (prefix with - for descending); no default sort
startinteger0Pagination offset
limitinteger20Number of results per page (0 disables)

Error Responses​

Standard HTTP status codes:

  • 200 - Success
  • 201 - Created (POST /api/datasets)
  • 400 - Bad request (validation error)
  • 403 - Forbidden (you can read the Dataset but lack the required permission)
  • 404 - Not found, or you lack read access to the Dataset
  • 409 - Conflict (duplicate, constraint violation)
  • 500 - Internal server error

Error responses include:

{
  "statusCode": 403,
  "error": "Forbidden",
  "message": "Insufficient permissions to modify dataset"
}

Long-Running Operations​

Several endpoints support long-running operations with progress tracking via a progress parameter in the payload:

  • POST /api/datasets - Create with file upload import
  • POST /api/datasets/{id}/_refresh - Refresh from datasource
  • POST /api/datasets/{id}/_import - Import data

POST /api/datasets allows up to 15 minutes per request.

Next Steps​

Explore the specific endpoint categories: