Skip to content

Statistics and Quality Control

Compute statistics and quality metrics of a QPX dataset with the Python API. Every example here runs on a converted dataset in ./output/.

Getting Started

import qpx

ds = qpx.Dataset("./output/")

print(ds.available_structures)
print(f"PSMs: {ds.psm.count()}")
print(f"Features: {ds.feature.count()}")
print(f"Protein group rows: {ds.pg.count()}")

A view opens lazily: count(), filter() and select() run in DuckDB without loading the file, and to_df() loads only what the query asks for.

Built-in Summaries

import qpx

ds = qpx.Dataset("./output/")

# One row: unique peptides and proteins, runs, features (targets only)
print(ds.qc_view.metrics().to_df())

# One row per run: n_proteins, n_peptides, n_features
print(ds.identification_summary.summary().to_df())

# One row per modification: n_psms, n_peptides
print(ds.modification_view.frequency().to_df())

# Through the sample table
print(ds.sample_summary.peptides_per_sample().to_df())
print(ds.sample_summary.proteins_per_sample().to_df())

Your Own Queries

ds.sql() runs DuckDB SQL over the views, which are registered by name:

import qpx

ds = qpx.Dataset("./output/")

# Charge states of target features
charges = ds.sql("""
    SELECT charge, count(*) AS n
    FROM feature
    WHERE NOT coalesce(is_decoy, false)
    GROUP BY charge
    ORDER BY charge
""").to_df()
print(charges)

# Peptides and peptidoforms
counts = ds.sql("""
    SELECT count(DISTINCT sequence) AS peptides, count(DISTINCT peptidoform) AS peptidoforms
    FROM feature
""").fetchone()
print(counts)

unique is a reserved word

Write "unique" in SQL when you select that feature column.

The same results are available through the query builder:

import qpx

ds = qpx.Dataset("./output/")

print(ds.feature.filter("charge = 2").count())
print(ds.feature.filter("charge = 2").select("sequence", "charge").limit(5).to_df())
print(ds.psm.targets_only().select("peptidoform").to_df()["peptidoform"].nunique())

Missing Values

design_matrix() pivots protein quantities to one row per sample and one column per protein. A missing quantity is NaN.

import qpx

ds = qpx.Dataset("./output/")

matrix = ds.design_matrix()
print(f"{matrix.shape[0]} samples, {matrix.shape[1]} proteins")
print(f"Missing overall: {matrix.isna().to_numpy().mean():.1%}")
print(matrix.isna().mean(axis=1).sort_values(ascending=False).head())

A Quality Report

import qpx


def quality_report(path, min_peptides_per_run=1000):
    """Flag runs with few peptides, against the dataset's own median."""
    ds = qpx.Dataset(path)
    runs = ds.identification_summary.summary().to_df()
    median = runs["n_peptides"].median()
    low = runs[(runs["n_peptides"] < min_peptides_per_run) | (runs["n_peptides"] < median / 2)]
    return {
        "metrics": ds.qc_view.metrics().to_df().iloc[0].to_dict(),
        "median_peptides_per_run": float(median),
        "low_runs": low["run_file_name"].tolist(),
    }


print(quality_report("./output/"))

The thresholds are examples: choose them for your instrument and gradient. Do not take a fixed PSM count as a sign of quality across experiments.

Validating Alongside Statistics

validate() returns one result per view. Use it before reporting numbers:

import qpx

ds = qpx.Dataset("./output/")

for structure, result in ds.validate().items():
    errors = [issue for issue in result.issues if issue.severity == "error"]
    print(structure, "valid" if not errors else f"{len(errors)} error(s)")

Comparing Datasets

Ids and identities are per analysis (Row Identity), so compare natural measures. To compare several conversions, loop over their folders:

import pandas as pd
import qpx

rows = []
for path in ["./output/"]:
    ds = qpx.Dataset(path)
    rows.append({"dataset": path, **ds.qc_view.metrics().to_df().iloc[0].to_dict()})
print(pd.DataFrame(rows))