Skip to content

Protein Group View

The protein group (PG) view is a tabular Parquet file that contains the details of protein groups identified and quantified per quantification unit. A quantification unit is the group of raw files aggregated together (grouped_runs) — a protein quantity only exists after aggregating peptides across a sample's fractions, so the view keys on this group of raw files rather than a single raw file (for unfractionated or DIA data the group is a single file). It captures the relationship between protein groups and the grouped runs in which they were detected, including peptide counts, feature counts, quality metrics, and intensity-based quantification. The sample is resolved downstream via (any file in grouped_runs, label) -> run.samples[].sample_accession.

This view is analogous to outputs from tools such as MaxQuant (proteinGroups.txt), DIA-NN (pg_matrix), and FragPipe protein group reports. Each row has a mandatory opaque pg_id, which is the primary key. A producer-supplied ID is preserved by default; otherwise QPX derives the ID from the footer-declared identity_composite.

Use cases

  • Retrieve all protein groups identified or quantified in a given raw file.
  • Retrieve protein group abundance by file and condition.
  • Store and query FDR q-values for protein groups at both the run and experiment level.
  • Support downstream statistical analysis by providing per-file protein-level quantification.

Schema

pg_id is the non-null primary key, meaningful within one file (Row Identity). The schema-default identity_composite is [pg_accessions, grouped_runs, label]; label is null only for identification-only protein groups with no label context. A known label is retained even when its intensity is null. Fields marked with (nullable) may have null values. See the full YAML schema in pg.yaml.

Row identity

Field Description Type Required
pg_id Opaque primary key, supplied by the producer or derived by QPX from the footer-declared identity_composite int64 Yes

Identity

Field Description Type Required
pg_accessions Protein accessions of all proteins within this group, as bare accessions (P04217, isoform P04217-2, contaminant CONTAM_P05787), never the UniProt FASTA header form sp\|P04217\|A1BG_HUMAN; identifiers from other databases are kept as written array[string] Yes
pg_group_id Hash of the sorted pg_accessions; the same for all rows of a group and on feature (specification 1.3) int64, null No
pg_names UniProt entry names of the proteins in the group, aligned with pg_accessions (A1BG_HUMAN) array[string] No
gg_accessions Gene group accessions as a string array array[string] No
gg_names Gene names corresponding to the proteins in the group array[string] No
gg_qvalue Gene group q-value (e.g., DIA-NN GG.Q.Value) float64, null No
anchor_protein Representative protein of the group (leading protein), as a bare accession; descriptive and not part of the default identity composite string Yes
grouped_runs The group of raw files aggregated into one quantification unit (fractions aggregated together; single-element for unfractionated/DIA). The sample is resolved downstream via (any file in grouped_runs, label) -> run.samples[].sample_accession; part of the default identity composite array[string] Yes

Accession convention

Every converter writes a UniProt id as its bare accession: P04114, not sp|P04114|APOB_HUMAN. The entry name goes to pg_names when the source has one. A decoy or contaminant marker from the search database stays on the accession (DECOY_P04114, REV__P04114, CONTAM_P02768), so that a decoy never shares an accession with its target, but is_decoy and contaminant state the status; do not parse it from the prefix. Non-UniProt accessions, such as RefSeq, are written as the source reports them.

What intensity holds

intensity is the primary quantity of the producer, and differs between tools and tool versions: DIA-NN 1.8 reports PG.Quantity and DIA-NN 2.x PG.MaxLFQ, which differ by an order of magnitude on the same samples, and QPX versions before 1.1 wrote MaxLFQ for most DIA-NN rows without marking it. Where a converter knows the method, the row's cv_params has a quantification_method entry. Compare intensity across datasets only when that entry agrees; merging datasets warns when it does not.

Protein group semantics

A pg row is an analyte: the quantified unit is the protein group as a whole, identified by pg_id (a function of the full pg_accessions membership together with grouped_runs and label). The group — not any single protein — is the object of downstream analysis. This follows the guidance of DIA-NN's author (vdemichev/DiaNN#1149) and the QPX design discussion (bigbio/qpx#266).

  • The group is the unit of analysis, not any single protein. anchor_protein is a descriptive representative only (e.g. for display). Do not run per-protein statistics on anchor_protein as if it were the analyte — for groups that share protein IDs this misattributes and can double-count shared signal.
  • Groups are NOT guaranteed disjoint, and leaders are NOT guaranteed unique. Depending on the producing tool's inference settings, two distinct groups may share protein accessions — including the leading one. For example, DIA-NN with heuristic protein inference disabled (--no-prot-inf), or its pre‑1.8.1 grouping, reports a group as the estimated joint contribution of the listed proteins, so the same protein can lead more than one group (e.g. P0AC33 fumA and P0AC33;P14407 fumA;fumB, carrying distinct quantities). QPX therefore keys pg_id on the full pg_accessions membership, so every group is a distinct, joinable analyte regardless of the producer or its inference mode. Consumers MUST NOT assume a protein appears in at most one group.
  • Be fail-safe when parsing group members. In the most general case a producer's group label may be an opaque string; QPX parses it into pg_accessions where possible, but a consumer that cannot resolve protein IDs from a group (e.g. for biological annotation) should continue without error rather than fail.
  • For gene- or pathway-level analysis, aggregate on gg_accessions (gene groups) / unique gene entries — do not pick a single protein out of a group.
  • feature → pg is a computed softlink, not a persisted column. By default the association is derived on read (Dataset.link_feature_pg(), bigbio/qpx#269) by a label-aware join of the feature and pg views:
  • canonical(feature.pg_accessions) = canonical(pg.pg_accessions) — the same order-independent membership set that keys pg_id; AND
  • feature.run_file_name ∈ pg.grouped_runs; AND
  • a label in feature.intensities IS NOT DISTINCT FROM pg.label, so a feature links only to pg rows for the channels/labels it actually carries (LFQ: one LFQ row; TMT/plexDIA: one row per channel the feature has, never the missing ones). pg_id is read straight from the matched pg row (never re-derived); never join on anchor_protein alone. No match means no matching PG link in that channel/fraction — the feature simply produces no link (not an error). Link presence does not establish whether a quantity exists; inspect the matched PG row's intensity. feature.pg_ids remains an optional producer hardlink (a slot a producer MAY populate); qpx's own converters do not materialize it, and when it is absent consumers use the computed softlink above.
  • Integrity check: a row with a non-null anchor_protein MUST also carry pg_accessions (its full group membership). Since the group is the analyte and the join key, a protein-mapped feature that lacked membership would be an orphan. QPX validates this (a warning on the write/convert path and from Dataset.validate(), an error from qpxc validate, which always validates strictly, and from Dataset.validate(strict=True)). Blank/whitespace anchor_protein is treated as unset (not a membership violation).

Counts

Field Description Type Required
peptide_counts Peptide sequence counts for this protein group in this file struct No
peptide_counts.unique_sequences Number of peptide sequences unique to this protein group within this file int --
peptide_counts.total_sequences Total number of peptide sequences identified for this protein group in this file int --
feature_counts Peptide feature counts (peptide-charge combinations) for this protein group struct No
feature_counts.unique_features Number of unique peptide features specific to this protein group within this file int --
feature_counts.total_features Total number of peptide features identified for this protein group in this file int --

Quality

Field Description Type Required
global_qvalue Global q-value of the protein group at the experiment level float64 No
pg_qvalue Protein group q-value at the run level float64 No
is_decoy Whether the protein group is a decoy (true) or a target (false) bool Yes
contaminant Contaminant flag bool, null No
sequence_coverage Percentage of the protein sequence covered by identified peptides float32 No
molecular_weight Molecular weight of the protein in kDa float32 No

A PG view may contain decoy protein groups (is_decoy = true) when the input reports them, for example to support protein-level FDR estimation across datasets. QPX does not remove decoy rows when it writes or reads the view, so consumers MUST filter on is_decoy themselves (for example with PG.targets_only()) when they need target protein groups only.

Quantification

Field Description Type Required
label Channel/label (e.g., TMT126, LFQ); one row per label in each unit where the group has identification evidence, including labels without a quantity (see Missing quantities and absent groups). Null only when identification-only input provides no label context. Part of the default identity composite. string No
intensity Primary raw intensity value for this label. Null when the identified group has no reported quantity. float32 No
additional_intensities Pre-computed intensity values from the upstream tool (normalized, LFQ, iBAQ, etc.) for this row's label. See Intensities array[struct] No
additional_scores Additional scores and metrics (posterior error probability, confidence, etc.). See Scores array[struct] No

Since QPX 1.1 the protein-group quantification is flattened: instead of an intensities list, each row carries a scalar label + intensity, so there is at most one row per (pg_accessions, grouped_runs, label). The next section defines which of these combinations have a row.

Missing quantities and absent groups

For a known quantification unit and label, distinguish identification from quantification:

Evidence in the producing workflow PG representation
Identified and quantified A row with the known label and reported intensity
Passed identification/FDR filtering, but no quantity is available A row with the known label and intensity = null
No identification or quantification evidence for this unit No row for that group/unit/label combination

A group has identification evidence in a unit when at least one PSM that passed the FDR threshold comes from a raw file of the unit and matches a protein of the group. The PSM can be assigned to a feature or unassigned, and its peptide can be unique to the group or shared. In a unit with evidence, write one row for each applicable label (each isobaric channel, or LFQ); a label without a measured quantity keeps its label and has intensity = null. In a unit without identification evidence, write a row only for each label with a measured quantity (for example from match-between-runs), never a row with intensity = null; with no quantity either, the group has no row in that unit (bigbio/qpx#280).

Do not drop an identified group merely because its quantity is missing, and do not fill unobserved combinations with zero or synthetic null rows. Global peptide or spectral counts alone do not establish identification in a particular unit; the input must support that attribution. Missing evidence fields must not be treated as proof that a group was identified in every run.

A tool's zero sentinel for an unavailable quantity is represented as null when the input supplies identification evidence for that unit. A genuine measured zero is a numeric quantity, not a missing value. Identification-only inputs with no label context (for example, mzIdentML) retain null label and intensity.

Each entry in additional_intensities contains:

Sub-field Description Type
label Label identifier (e.g., TMT126, LFQ) string
intensities Array of name-value pairs for derived intensities array[struct{intensity_name, intensity_value}]

Peptide detail

Field Description Type Required
peptides Number of peptides per individual protein in the protein group array[struct] Yes

Each entry in peptides contains:

Sub-field Description Type
protein_name Protein accession string
peptide_count Number of peptides for this protein int

CV params

Field Description Type Required
is_imputed Whether the tool imputed this quantity instead of measuring it; null when the tool does not say bool, null No
cv_params Optional list of controlled vocabulary parameters for additional metadata array[struct{cv_name, cv_value}] No

qpxc transform relayout can store additional_intensities, additional_scores and cv_params as a struct with one field per name, per label and name for additional_intensities (Struct lists).

Example

{
  "pg_id": 2937018471546916399,
  "pg_accessions": ["P04217", "A0A024R4E5"],
  "pg_names": ["Alpha-1B-glycoprotein", "Alpha-1B-glycoprotein variant"],
  "gg_accessions": ["A1BG"],
  "gg_names": ["A1BG"],
  "anchor_protein": "P04217",
  "grouped_runs": ["20230101_sample_01"],
  "peptide_counts": {
    "unique_sequences": 12,
    "total_sequences": 18
  },
  "feature_counts": {
    "unique_features": 24,
    "total_features": 36
  },
  "global_qvalue": 0.001,
  "pg_qvalue": 0.005,
  "is_decoy": false,
  "contaminant": false,
  "sequence_coverage": 45.2,
  "molecular_weight": 54.3,
  "label": "TMT126",
  "intensity": 1.5e8,
  "additional_intensities": [
    {
      "label": "TMT126",
      "intensities": [
        {"intensity_name": "LFQ", "intensity_value": 1.2e8},
        {"intensity_name": "iBAQ", "intensity_value": 3.4e7}
      ]
    }
  ],
  "peptides": [
    {"protein_name": "P04217", "peptide_count": 15},
    {"protein_name": "A0A024R4E5", "peptide_count": 3}
  ],
  "additional_scores": [
    {"score_name": "posterior_error_probability", "score_value": 0.0001, "higher_better": false}
  ],
  "cv_params": null
}

The following names are commonly used in additional_intensities for the PG view. Use consistent naming so downstream tools can recognise them across datasets.

intensity_name Source tool(s) Description
maxlfq DIA-NN, MaxQuant, FragPipe MaxLFQ normalised protein group quantity
lfq MaxQuant Label-free quantification intensity
ibaq MaxQuant Intensity-based absolute quantification
topn DIA-NN Top-N normalised protein group quantity
normalize_intensity quantms Median-normalised intensity
spectral_count FragPipe, MaxQuant Number of PSMs for razor peptides
unique_spectral_count FragPipe Number of PSMs for unique peptides only
total_spectral_count FragPipe Number of PSMs for all peptides (razor + shared)
genes_maxlfq DIA-NN Gene-group level MaxLFQ quantity
genes_maxlfq_unique DIA-NN Gene-group MaxLFQ using only proteotypic peptides
ratio_h_l MaxQuant, OpenMS (SILAC) Heavy-to-light protein ratio, on the SILAC heavy row
ratio_h_l_normalized MaxQuant, OpenMS (SILAC) Normalised heavy-to-light ratio, on the SILAC heavy row
unnormalized_unique_peptide_sum OpenMS consensusXML Sum of the group's unique peptides, beside OpenMS' own protein abundance
reporter_intensity MaxQuant (TMT/iTRAQ) Raw reporter-ion intensity
reporter_intensity_corrected MaxQuant (TMT/iTRAQ) Isotope-purity-corrected reporter intensity
normalized_abundance, scaled_abundance Proteome Discoverer Abundances (Normalized) and Abundances (Scaled) of the file group and channel
tmt_integrator_log2_abundance FragPipe --tmt-report TMT-Integrator's normalized log2 abundance of the protein in the channel

Spectral counts as intensities

Spectral counts are quantitative measures that vary per run and per label, so they fit naturally in additional_intensities. Store them as float values (e.g., 3.0 instead of 3) since the struct uses float32.

Gene-group quantities

DIA-NN reports protein-group level (PG.MaxLFQ) and gene-group level (Genes.MaxLFQ) quantities separately. Store gene-group quantities in additional_intensities with names prefixed genes_ (e.g., genes_maxlfq, genes_maxlfq_unique). The gg_accessions field identifies the gene group.

The following score names are commonly used in additional_scores for the PG view. For the full list of recommended score names and naming conventions, see Scores.

score_name Source tool(s) Direction Description
posterior_error_probability MaxQuant lower is better Protein-group PEP
andromeda_score MaxQuant higher is better Andromeda protein-level score
protein_probability FragPipe higher is better ProteinProphet probability
top_peptide_probability FragPipe higher is better Highest PeptideProphet probability among peptides
pg_maxlfq_quality DIA-NN higher is better QuantUMS quality for PG.MaxLFQ estimate
genes_maxlfq_quality DIA-NN higher is better QuantUMS quality for gene-level MaxLFQ
pg_pep DIA-NN lower is better Posterior error probability at the protein group level
gg_qvalue DIA-NN lower is better Gene group q-value
lib_pg_qvalue DIA-NN lower is better Library protein group q-value
protein_qvalue DIA-NN lower is better Unique protein q-value (proteotypic evidence)

Tool Mappings

This section shows how output columns from common search engines and pipelines map to pg.parquet fields.

Wide-to-long conversion

MaxQuant, DIA-NN, and FragPipe output protein groups in wide format (one row per protein group, one column per experiment/run). QPX uses long format (one row per protein group per quantification unit). Converters must melt wide-format columns into separate rows keyed by grouped_runs (the group of raw files aggregated into one quantification unit; single-element for unfractionated/DIA).

MaxQuant (proteinGroups.txt)

MaxQuant column QPX field Notes
Protein IDs pg_accessions Semicolon-separated → array
Majority protein IDs anchor_protein First accession of majority IDs
Protein names pg_names Semicolon-separated → array
Gene names gg_names Semicolon-separated → array
Sequence coverage [exp] [%] sequence_coverage Coverage in the experiment
Mol. weight [kDa] molecular_weight
Reverse is_decoy "+" → true, else false
Potential contaminant contaminant "+" → true, else false
Only identified by site cv_params {cv_name: "only_identified_by_site", cv_value: "true"}
Identification type [exp] cv_params {cv_name: "identification_type", cv_value: "By MS/MS"}
MaxQuant column QPX field Notes
Unique peptides [exp] peptide_counts.unique_sequences Peptides exclusive to this PG in the experiment
Peptides [exp] peptide_counts.total_sequences Peptides of this PG identified in the experiment
Peptide counts (all) peptides Per-protein peptide counts, aligned with Protein IDs
--- feature_counts Not reported by MaxQuant; null
MaxQuant column QPX field Notes
Intensity [exp] intensity Primary raw intensity; scalar per row (one row per label)
LFQ intensity [exp] additional_intensities → lfq MaxLFQ normalised
iBAQ [exp] additional_intensities → ibaq Intensity-based absolute quantification
MS/MS count [exp] additional_intensities → spectral_count PSM count as float; the plex's count on every channel row
Reporter intensity [channel] intensity For TMT/iTRAQ, one row per channel (label = channel)
Reporter intensity corrected [channel] additional_intensities → reporter_intensity_corrected
Intensity L [exp], Intensity M [exp], Intensity H [exp] intensity For SILAC, one row per label (label = SILAC light/medium/heavy)
Ratio H/L [exp] additional_intensities → ratio_h_l SILAC ratio, on the numerator (SILAC heavy) row
Ratio H/L normalized [exp] additional_intensities → ratio_h_l_normalized Normalised SILAC ratio, on the numerator row
MaxQuant column QPX field Notes
Q-value global_qvalue Protein group FDR
Score additional_scores → andromeda_score
PEP additional_scores → posterior_error_probability

DIA-NN (main report)

DIA-NN's main report is precursor-level. PG-level columns (PG.*, Genes.*) are repeated for every precursor in the group within a run. Deduplicate by Protein.Group + Run, and Channel in a plexDIA report, to produce one pg.parquet row per label; each channel keeps its own quantities and scores. The pg_matrix is optional and only fills a label-free MaxLFQ the report lacks; DIA-NN writes none for a plexDIA report. A quantity of 0 is DIA-NN's mark for a group it identified but did not quantify, written as a null intensity (or no additional_intensities entry). Global.PG.Q.Value, PG.Q.Value and PG.PEP are carried for every protein group and never used as a cutoff; --qvalue-threshold filters precursors on Q.Value.

DIA-NN column QPX field Notes
Protein.Group pg_accessions Semicolon-separated → array; anchor_protein = first accession. Blank/whitespace groups (e.g. DIA-NN unmapped rows) → anchor_protein NULL, not "".
Protein.Names pg_names
Genes gg_names / gg_accessions
Run grouped_runs Raw file name without path, wrapped as a single-element list
Decoy is_decoy Present with --report-decoys. DIA-NN reports a decoy precursor (Decoy = 1) under its target's Protein.Group; QPX writes those rows as a separate group with DECOY_ before each accession, so is_decoy = true. Without the column, is_decoy is true only when every accession carries a decoy prefix (DECOY_, decoy_, rev_, REV_).
DIA-NN column QPX field Notes
PG.Quantity intensity Non-normalised protein group quantity (scalar; label = LFQ, or the plexDIA channel's label). Without a positive value, PG.MaxLFQ (then the PG matrix value) is the intensity. A 0 is DIA-NN's value for a quantity it did not compute: an identified group without a positive quantity keeps its label with intensity = null
PG.MaxLFQ additional_intensities → maxlfq QuantUMS/MaxLFQ normalised; the intensity itself when the report has no PG.Quantity (DIA-NN 2.x). QuantUMS writes 0 for a group it did not quantify (and leaves its pg_matrix cell empty); a 0 is left out
PG.Normalised additional_intensities → normalize_intensity Normalised PG quantity
PG.TopN additional_intensities → topn Top-N normalised quantity
Genes.MaxLFQ additional_intensities → genes_maxlfq Gene-group MaxLFQ
Genes.MaxLFQ.Unique additional_intensities → genes_maxlfq_unique Gene-group MaxLFQ (proteotypic only)
Genes.TopN additional_intensities → genes_topn Gene-group Top-N
DIA-NN column QPX field Notes
Global.PG.Q.Value global_qvalue Experiment-level PG FDR
PG.Q.Value pg_qvalue Run-level PG FDR
PG.PEP additional_scores → pg_pep
PG.MaxLFQ.Quality additional_scores → pg_maxlfq_quality QuantUMS quality metric
GG.Q.Value gg_qvalue Gene group q-value
Lib.PG.Q.Value additional_scores → lib_pg_qvalue Library PG q-value
Protein.Q.Value additional_scores → protein_qvalue Unique protein q-value
Genes.MaxLFQ.Quality additional_scores → genes_maxlfq_quality

FragPipe (combined_protein.tsv)

Decoys are filtered out before FragPipe writes the report; contaminants (contam_ accessions) are kept. Per-experiment columns are prefixed with the experiment name, and any of them names an experiment, so a spectral-count-only report still gives one row per experiment.

FragPipe column QPX field Notes
Protein ID anchor_protein Leading protein accession
Protein ID + Indistinguishable Proteins pg_accessions Combine into array
Description pg_names Protein description
Gene gg_names / gg_accessions
Coverage sequence_coverage Percentage (0–100)
Protein Length cv_params {cv_name: "sequence_length", cv_value: "495"}
Protein Existence cv_params {cv_name: "protein_existence", cv_value: "1"}
FragPipe column QPX field Notes
[exp] Total Peptides peptide_counts.total_sequences Per-experiment peptide count, with [exp] Unique Peptides
[exp] Unique Peptides peptide_counts.unique_sequences peptide_counts is null unless both are reported
FragPipe column QPX field Notes
[exp] Intensity intensity Primary razor peptide intensity (scalar per row); [exp] Total Intensity in older reports
[exp] MaxLFQ Intensity additional_intensities → maxlfq MaxLFQ normalised
[exp] MaxLFQ Unique Intensity additional_intensities → maxlfq_unique MaxLFQ using only unique peptides
[exp] Unique Intensity additional_intensities → unique_intensity Intensity from unique peptides only
[exp] Spectral Count additional_intensities → spectral_count Razor peptide PSMs
[exp] Unique Spectral Count additional_intensities → unique_spectral_count Unique peptide PSMs
[exp] Total Spectral Count additional_intensities → total_spectral_count All peptide PSMs
FragPipe column QPX field Notes
Protein Probability additional_scores → protein_probability ProteinProphet score
Top Peptide Probability additional_scores → top_peptide_probability Best PeptideProphet score

FragPipe q-values

FragPipe does not report explicit protein-group q-values. The Protein Probability (from ProteinProphet) is filtered at a given FDR threshold (typically 1%) before writing combined_protein.tsv. Set global_qvalue to null and store the probability in additional_scores.

Notes

Relationship to other views

The PG view provides per-file protein group quantification. For derived per-sample summaries (protein counts, abundances, etc.), see API Views. For downstream absolute or differential expression results, see the Absolute Expression and Differential Expression views.

Identity constraints

pg_id is the primary key. When QPX derives it, the default identity composite is pg_accessions, grouped_runs, and label. pg_accessions and grouped_runs MUST NOT be null. label is null only for identification-only protein groups with no label context (e.g. mzIdentML); a known label is retained when its intensity is null, and there is one row per applicable label. Both pg_accessions and grouped_runs are compared as sets for identity: duplicate members are ignored and order does not affect the derived ID. grouped_runs itself MUST contain distinct raw files and is stored in fraction order — sorting is not applied, because it would destroy fraction ordering (and lexicographically misorder names like F1, F10, F2). Each record represents a single protein group identified or quantified in one quantification unit (the group of raw files/fractions aggregated together) for one label. A protein quantity only exists after aggregating peptides across a sample's fractions, so the PG view identifies this group of raw files rather than a single raw file; for unfractionated or DIA data the list has a single element.

Within one (pg_accessions, label), the grouped_runs sets across rows MUST be disjoint — every raw file contributes to at most one row, so no measurement is counted twice (the run-disjointness invariant enforced by validation).