Troubleshooting¶
Common issues and solutions when using qpx.
Installation Issues¶
ModuleNotFoundError: No module named 'qpx'¶
Problem: Python cannot find the qpx package after installation.
Solutions:
- Ensure you're using the correct Python environment:
- Reinstall in the active environment:
- If using conda, ensure the environment is activated:
Older releases fail with DuckDB 2.0¶
QPX 1.0.0 to 1.1.6 set no upper bound on DuckDB, so a fresh install takes DuckDB 2.0, which they do not support: the DIA-NN, MaxQuant and Spectronaut converters, validate and to_arrow() fail with BinderException or a missing fetch_record_batch. Upgrade to 1.2.0 or later, or pin the older DuckDB:
A conversion records the DuckDB and PyArrow versions it ran with as dependency_duckdb and dependency_pyarrow in its provenance step.
Python version incompatibility¶
Problem: Installation fails with Python version errors.
Solution: qpx requires Python 3.10 or higher. Check your version:
If needed, install a compatible Python version:
Missing dependencies¶
Problem: Import errors for packages like venn or pyopenms.
Solution: Install the missing package (anndata / mudata / scipy ship with core qpx):
Conversion Issues¶
File not found errors¶
Problem: FileNotFoundError when running convert commands.
Solutions:
- Use absolute paths:
- Verify the file exists:
Memory errors with large files¶
Problem: MemoryError or system becomes unresponsive with large datasets.
Each converter uses memory differently, and for most of them a larger --max-memory makes the peak higher, not lower. See Memory for what to do with each one.
Invalid file format errors¶
Problem: ValueError or parsing errors when reading input files.
Solutions:
-
Verify the file format matches the expected format for the converter
-
Check for file corruption:
- Ensure the file encoding is UTF-8:
Output Issues¶
Empty output files¶
Problem: Parquet files are created but contain no data.
Solutions:
-
Check if input data passes quality filters (q-value, PEP thresholds)
-
Verify column names match expected format for the software
-
Use
--verboseflag to see processing details:
Missing columns in output¶
Problem: Expected columns are not present in the output Parquet file.
Solutions:
-
Check if the input file contains the required source columns
-
For spectral data, ensure
--spectral-dataflag is used:
- Review the Format Specification for required vs optional fields
Validation Issues¶
Validating a dataset¶
Use the validate command to check a dataset against the canonical schemas:
# Validate all structures in a dataset
qpxc validate --dataset-path ./PXD014414
# Validate a specific structure
qpxc validate --dataset-path ./PXD014414 --structure feature
# Validate a single Parquet file
qpxc validate --file ./data.feature.parquet
Common validation errors¶
Missing required column: A required column is absent from the Parquet file. Check the schema reference for required fields.
Missing column ... (its values may be null, but the column must be present): Required: no in the view tables means the values may be null, not that the column may be left out. Only fields marked optional in the schema may be absent. The QPX writers add these columns for you; a producer writing Parquet itself writes them as all-null columns of the schema's type.
Type mismatch: A column has a different Arrow type than the schema expects. This usually means the data was written with an older version of qpx or a different tool.
Null values in non-nullable columns: Required columns should not contain null values. Check your input data and conversion pipeline.
Duplicate primary key: Rows with identical primary key values exist. This may indicate duplicate entries in the source data.
Programmatic validation¶
You can also validate from Python:
import qpx
with qpx.open_dataset("./PXD014414") as ds:
results = ds.validate()
for name, result in results.items():
print(result.summary)
for issue in result.issues:
print(f" [{issue.severity}] {issue.message}")
SDRF Issues¶
Sample name mismatches¶
Problem: Samples in data files don't match SDRF sample names.
Solutions:
-
Ensure
source namecolumn in SDRF matches file names (without extension) -
Check for whitespace or case sensitivity issues:
import pandas as pd
sdrf = pd.read_csv("experiment.sdrf.tsv", sep="\t")
print(sdrf["source name"].unique())
Missing factor values¶
Problem: Factor values are not extracted from SDRF.
Solution: Ensure factor columns follow the format factor value[factor_name]:
source name factor value[disease] factor value[organism part]
sample1 healthy liver
sample2 cancer liver
Converting SDRF to QPX metadata¶
Use the dedicated SDRF converter to produce sample.parquet and run.parquet:
Query Issues¶
SQL query errors¶
Problem: SQL queries fail with column or table not found errors.
Solutions:
- Check available structures in the dataset:
- Check the schema for a specific structure:
- Table names in SQL match the QPX structure names:
psm,feature,pg,mz,sample,run,dataset,ontology,provenance.
Memory issues with large queries¶
Solutions:
- Use
--duckdb-memoryto increase DuckDB memory (validate,query sqland thetransform gene-map/protein-propertiescommands):
For convert commands, use --max-memory instead:
qpxc convert diann \
--report-path report.tsv \
--sdrf-file data.sdrf.tsv \
--output-folder ./output \
--max-memory 32GB
- DuckDB writes what does not fit in memory to disk. Convert commands, and transform commands given
--output-folder, put these spill files in.tmpin the output folder; the other commands use.tmpin the working directory. When that disk is small or read-only, point--temp-dirat one with free space:
- Use SQL
LIMITand select only the columns you need. Parquet reads only those columns.
High memory usage¶
Close other applications, and for a conversion see Memory.
Getting More Help¶
If your issue isn't listed here:
-
Search existing issues: GitHub Issues
-
Enable verbose logging: Add
--verboseto any command for detailed output -
Create a new issue: Include:
- qpx version (
qpxc --version) - Python version (
python --version) - Operating system
- Complete error message
- Minimal reproducible example