SubQ: excerpt versus full manuscript

We tested whether SubQ Snap can classify the role of a prerequisite using either a local manuscript passage or the complete native manuscript. The available roles are proof input, background, explicitly excluded and insufficient context. No mathematical validity verdict is requested, and no graph edges or panel conclusions are changed.

Primary paired test

There are 24 source-role cases: eight each from the six-page noncoordinate-polynomial paper, seven-page Foulkes–Howe paper and 199-page quasi-Riemann manuscript. Each case was requested once in each context condition, in a reproducibly shuffled order with a fixed choice ordering. The same consumer, prerequisite and excerpt are supplied in both conditions; the full condition additionally includes the exact complete source. Frozen source hashes are checked before inference. Researcher labels and panel findings are never sent to the API.

All 48 responses were valid typed decisions. The decisions were identical in every pair. Full-source agreement was 24/24; excerpt agreement was 21/24 against context-specific annotations. The three excerpt discrepancies were conservative insufficient-context labels for which the model selected the eventual full-source role already. Those labels are AI judgments, particularly debatable where wording such as “compare” provides background evidence. This first run does not demonstrate that the added context improved classifications.

The actual long requests used about 269,000 model-reported input tokens. This records that this workload fit and returned, rather than inferring compatibility from a different tokenizer or the vendor's performance claims.

Post-hoc context-omission follow-up

After the primary run, we selected the same eight quasi-Riemann role queries and replaced the informative local passages with the main-claim excerpt at native lines 99–113. This removes their use/exclusion passages. The full condition still includes the complete source. This is a deliberately harder post-hoc follow-up, not a held-out benchmark or eight new mathematical problems.

In the 16 follow-up calls, SubQ returned insufficient context on 7/8 excerpts. Full-source choices matched 7/8 source annotations, at a median 2.426 seconds, with up to 268,831 input tokens. Seven choices changed; six moved from an insufficient-context answer to the annotated full-source role. No pair moved from a matched full-source label to a mismatched one. These directional counts use the full-source annotation, while context-appropriate agreement separately rewards abstention for an uninformative excerpt.

The exception promoted the Baier–Bansal reference from background to proof input. A source check finds only two literal citations, both comparisons, while the needed Eisenstein-lattice lemma is proved locally. Under our applied-premise definition we retain background. The earlier full-source question, with the informative excerpt included, gave background for this same reference. This is prompt-state sensitivity: the choice-only API does not reveal whether retrieval, context prioritization or taxonomy caused the change. A whole-paper prerequisite and whole-manuscript consumer also permit a broader methodological-antecedent interpretation. Read the bounded source adjudication.

Costs and records

The primary 48 calls cost $0.114570 in recorded-usage estimates; the 16 follow-up calls cost $0.107706. Across all 64 calls, 4,445,506 reported input tokens give $0.222275 at the published $0.05 per million input-token rate. Output tokens are free. This is usage accounting, not a provider invoice. There were no transport/schema errors or automatic retries. The respective runners reserved both conditions at the documented maximum context window before sending requests.

The primary summary, primary answers, primary cases, follow-up summary, follow-up answers and follow-up cases preserve probabilities, labels, hashes, usage and timing. Full request bodies, private reasoning streams, secrets and provider headers are outside these downloads. The original complete manuscripts are available on their separate reading pages.

Interpretation and next evaluation

This is encouraging evidence for an optional long-context dependency classifier on actual mathematical source. It does not establish proof-auditing accuracy or superiority over another model. The labels are researcher annotations with a second AI source check, not independent human gold. There is one request per condition, fixed option order, a small selected sample and no matched competing decision model. The follow-up was designed after seeing the first result and must remain separately reported.

Use result-level prerequisites and local consumers, require source evidence for any proposed graph change, and preserve uncertainty as an explicit outcome. The next evaluation should use independently labeled passages across several held-out papers, with separate metrics for missed proof inputs, unsupported promotion, abstention, cost and latency. Confidence remains a decision-distribution signal rather than a theorem probability.

Sources

• SubQ Decisions API — request schema, typed probabilities and documented context limit.
• SubQ models and pricing — input rate and free output.
• OpenAI mathematics collection at the selected checkout — the frozen new manuscripts; quasi-Riemann retains its separately recorded reviewed source hash and earlier review commit.