Validate Command

Validate QPX datasets and individual structures against their canonical schemas.

Overview

The validate command checks QPX datasets and individual Parquet files against their canonical schemas. It verifies column presence, type matching, null values in required columns, and primary key uniqueness.

Parameters

Parameter	Type	Required	Default	Description
--dataset-path	DIRECTORY	No	-	Path to a QPX dataset directory
--file	FILE	No	-	Path to a single QPX Parquet file to validate
--structure	CHOICE	No	-	Structure(s) to validate (repeatable). Default: all.
--verbose	FLAG	No	-	Enable verbose logging

Description

Checks column presence, type matching, null values in required columns, and primary key uniqueness. 

Usage Examples

Validate QPX datasets and files:

# Validate an entire dataset
qpxc validate --dataset-path ./PXD014414

# Validate specific structures
qpxc validate --dataset-path ./PXD014414 --structure feature
qpxc validate --dataset-path ./PXD014414 --structure feature --structure pg

# Validate a single Parquet file
qpxc validate --file ./data.feature.parquet

Validation Checks

The validator performs the following checks on each structure:

Check	Description
Column presence	All required columns defined in the canonical schema must exist
Type matching	Column Arrow types must match the schema definition
Null values	Non-nullable columns must not contain null values
Primary key uniqueness	Primary key columns must have unique values

Cross-view checks

When a dataset has a run view, rows are also resolved through it. A row that names no run, or a label that names no single sample, drops out of every per-sample view or is counted twice, so these findings are errors with or without strict validation:

Check	View	Description
dangling_grouped_run	pg	Every grouped_runs element must be a run.run_file_name
ambiguous_grouped_run_sample	pg	Each label must match the channel of exactly one sample across the row's grouped runs; reported once per distinct (grouped runs, label) unit
dangling_run_file_name	feature, psm, mz	run_file_name must be a run.run_file_name (skipped for an mz view without the optional column)
ambiguous_label_sample	feature	Each intensities[].label must match the channel of exactly one sample of the row's run, also when the run has a single sample; reported once for all (run, label) pairs that match none, once for those that match several and once for those whose run lists the sample's channel more than once
duplicate_intensity_label	feature	A row must not list a label twice in intensities

A cross-view check that cannot run is reported as *_check_skipped with the reason (DuckDB out of memory or temp space, an I/O error, a missing or retyped column, or another query error). It is an error under strict validation, which qpxc validate always uses, and a warning otherwise.

Programmatic Validation

You can also validate from Python:

import qpx

with qpx.open_dataset("./PXD014414") as ds:
    results = ds.validate()
    for name, result in results.items():
        print(result.summary)
        for issue in result.issues:
            print(f"  [{issue.severity}] {issue.message}")

Exit Codes

Code	Meaning
0	All validated structures are valid
1	One or more structures have validation errors