QEncode Benchmark

Benchmark Methodology

Suite v4

QEncode uses a fixed, fully reproducible execution pipeline. Every entry — whether self-run or managed — follows the same pipeline, the same reference, and the same scoring rules. No manual adjustment of results is ever made.

Execution Pipeline

1. Classical reference — PySCF

PySCF computes the quantum chemistry reference at the molecule's geometry and cc-pVDZ basis. Outputs: HF, MP2, CCSD, CCSD(T) energies, and — critically — the CASCI active-space FCI energy. The CASCI energy is the VQE target. All accuracy gaps are measured as |E_VQE − E_CASCI|, never against full-system FCI or a classical approximation.

2. CASSCF orbital optimization (optional, required for some)

For molecules with strong multireference character — currently C₄H₄, N₂, H₆, benzene, H₈, and H₁₀ — HF canonical orbitals do not cleanly partition the active space. QEncode runs CASSCF to pre-optimise the orbital basis before building the qubit Hamiltonian. The flag is --orbital-opt casscf. For N₂, CASSCF and CASCI energies converge to twelve decimal places, confirming the orbital basis is fully converged before a single VQE evaluation runs. HF orbital optimization is the default for all other molecules.

3. Qubit Hamiltonian — PennyLane

PennyLane builds the molecular Hamiltonian from the active-space integrals using the selected qubit mapping: Jordan-Wigner (direct, all molecules), Parity (via OpenFermion bridge, all molecules), or Bravyi-Kitaev (H₂ and HF only — PL 0.45 imaginary artefacts excluded larger molecules). The Hamiltonian is expressed as a sum of weighted Pauli operators.

4. Z2 symmetry tapering

Conserved Z2 symmetry sectors are identified and eliminated, reducing the qubit count. H₂ and HF go from 4 → 1 qubit (3 symmetries removed). N₂ goes from 12 → 8 qubits (4 symmetries removed). The tapered HF reference state is computed with qchem.taper_hf(). For BK tapering (H₂ and HF only), imaginary parts are verified to be below 1e-6 × max_abs before the strip is applied.

5. VQE — COBYLA / L-BFGS-B / Adam

The variational quantum eigensolver runs with one of three optimizers: COBYLA (gradient-free, the suite default), L-BFGS-B (quasi-Newton, driven by analytic parameter-shift gradients), or Adam (stochastic gradient descent). Ansatz: UCCSD (tapered symmetry-adapted excitation operators), HEA (hardware-efficient alternating rotation/entanglement layers, configurable --reps), or ADAPT-VQE (operators selected one at a time from the UCCSD pool by parameter-shift gradient magnitude, giving a far shorter parameter vector than full UCCSD).

Run config by molecule class

Small molecules (H₂, HF, LiH, BeH₂, H₂O, NH₃) — HEA: max_iter=500, multistart=10

Small molecules — UCCSD: max_iter=500, multistart=10

Hydrogen chains (H₄, H₆) — HEA: max_iter=500, multistart=1–10; UCCSD: max_iter=1,000, multistart=1

N₂ UCCSD (404 params): max_iter=10,000, multistart=1 (HF zeros initialization)

ADAPT-VQE (H₂CO, C₄H₆, C₄H₄, (H₂O)₂, H₄, H₆, N₂, benzene, H₈, H₁₀) — COBYLA inner loop: max_iter=200–500, multistart=1

Hydrogen chains H₈/H₁₀ — ADAPT-VQE, statevector engine, L-BFGS-B inner: max_iter=500, multistart=1, max_ops=300 (H₈ certified at 98 ops, H₁₀ at 300)

Early-stop: fires when gap < 0.01 Ha, records actual restarts completed

Circuit metrics (depth, two-qubit gate count, T-gate estimate) are recorded post-VQE using the default PennyLane transpiler. A Beats Classical flag is set when the VQE correlation energy exceeds the CCSD(T) correlation energy.

6. Provenance and signing

Each entry is assigned a compound ID containing molecule, basis, mapping, ansatz, and a 16-character SHA-256 hash of the full JSON payload. Certified entries are signed with an Ed25519 key. The JSON entry includes all PySCF energies, VQE trajectory, circuit metrics, FCIDUMP integrals (base64), run config, and the signature. Entries are stored in releases/v4/db/ in the public GitHub repository.

Leaderboard Rules

Eligibility

Suite v4 match + trust_level = certified (gap < 0.01 Ha). Entries with gap ≥ 0.01 Ha appear in the Research tab only as validated results — honest, reproducible, but not certified. No result is ever discarded or adjusted.

Accuracy ranking

Sort by lowest |E_VQE − E_CASCI| in Hartrees. Lower is better. Two thresholds are shown: chemical accuracy (1.6 mHa) and certification threshold (10 mHa).

Hardware cost ranking

Sort by fewest two-qubit gates (tie-break: circuit depth). Excludes UCCSD entries on H₂ and HF where circuit depth is symbolic (PennyLane does not transpile UCCSD to native gates for 1-qubit tapered circuits). Requires transpiled circuit metrics to appear on this tab.

Balanced score

Rank-normalised combined score: 0.5 × (gap_rank / (N−1)) + 0.5 × (cost_rank / (N−1)) ∈ [0, 1]. Lower is better. Requires circuit metrics. N = certified entries with metrics.

Research tab

Validated entries (gap ≥ 0.01 Ha) are shown here. Example: N₂ JW/HEA with reps=4 achieved a best gap of 0.121 Ha across 30 restarts — 12× the threshold, showing HEA cannot navigate the N₂ energy landscape with 40 parameters. This is a meaningful data point, not a failure to discard. The benchmark records what actually happened.

Data Provenance

All v4 entries are stored as JSON artifacts in releases/v4/db/ in the public GitHub repository. Leaderboard CSVs are generated by scripts/export_leaderboard_v4.py directly from those entry files — no manual editing, no cherry-picking. Deduplication keeps the best gap per (molecule, mapping, ansatz, orbital_opt) combination.

The pipeline is version-locked: requirements-v4.txt pins PySCF 2.5.0, PennyLane 0.45, openfermion 1.7.1, and NumPy. Any run using this environment on the same molecule will reproduce the same reference energies.

v3 entries are frozen and remain reproducible with requirements-v3.txt via scripts/verify_entry.py.

← Benchmark SpecView leaderboardGitHub → generate_entry_v4.py