Independent Cross-Stack Benchmark Evaluates Commercial Quantum Error Management on 156-Qubit IBM Heron Hardware

An independent benchmarking study published on arXiv (arXiv:2608.05202) provides a protocol-level evaluation of commercial quantum error suppression and error mitigation software stacks. Conducted by researchers from The Catholic University of America, University of Deusto, and Universidad de los Andes, the paper compares native IBM Qiskit Runtime primitives against third-party managed pipelines in the Qiskit Functions Catalog—specifically Q-CTRL Performance Management and Qedma QESEM. All experiments were executed on ibm_pittsburgh, a 156-qubit IBM Quantum Heron r3 superconducting processor.

The benchmark evaluates two execution models: sampling tasks (bitstring counts) via Sampler interfaces, and observable expectation values via Estimator interfaces. To ensure like-for-like comparisons, IBM and Q-CTRL configurations operated under matched shot budgets (2¹⁵ = 32,768 shots per instance), while Qedma QESEM was configured through its native precision-and-time-budgeted interface (0.1 precision target, 600-second QPU time cap). Abstract workloads were submitted across all providers within identical hardware calibration windows.

In the Sampler Track, tested across Bernstein–Vazirani (25–75 qubits), Quantum Phase Estimation (10–30 counting qubits), GHZ-state preparation (25–50 qubits), and Randomized Mirror Circuits (25–100 qubits), Q-CTRL’s autonomous suppression pipeline delivered the highest exact-output success probabilities across all 12 test cases. On 30-qubit Quantum Phase Estimation (QPE), where raw and measurement-twirled IBM runs yielded zero exact-match shots out of 32,768 (0.00% success), Q-CTRL maintained a 12.69% exact success probability. On 100-qubit Randomized Mirror Circuits, Q-CTRL retained a 76.45% exact 100-bit target string success rate, compared to 9.14% on IBM raw execution.