The four data layers

This page describes the four-layer measurement architecture — signals → rules → constructs → traits — that SoulMap was designed around. Two corrections govern how to read it (added 2026-08-20 after our evidence audit): the rule layer described below was decommissioned as a runtime scorer on 2026-08-04 and does not execute in production — a served trait score today comes from a simpler deterministic aggregation over your recorded choices — and every validation figure we hold comes from synthetic personas driven by language models, not from human data. The layered design remains what makes the system auditable. In the deployed path, a trait estimate traces back to the specific choices you made and the construct weights attached to each option you were offered — server-stamped from a versioned evidence table, never authored by the language model. The fuller signal-and-rule chain below describes the designed architecture, not the deployed one.

This page walks each layer in plain English, then explains why the hierarchy exists and how we test that it is doing its job.

Signal: the small tells we listen for

The bottom layer is signals: small linguistic and behavioural tells the system extracts from each conversational turn. Approximately 250 distinct signals per turn, in the form of predicates — short structured facts that record something specific the speaker did in the conversation.

A signal is not "the speaker sounded conscientious." It is much smaller and much more concrete than that:

• the speaker mentioned a deadline
• the speaker named a specific person they were responsible to
• the speaker used a hedge word ("maybe," "I guess") at the start of their turn
• the speaker corrected themselves mid-sentence
• the speaker made a value judgment about another person's behaviour

These are observations. They do not interpret; they record. The interpretation happens at the next layer.

Two reasons for this design. First, signals are inspectable. If we ever produce a trait estimate that surprises a reader, we can trace it back to the specific signals that fed it, and you can read the conversational fragment that produced each signal. Second, signals are reusable. The same signal — say, "the speaker named a future commitment" — feeds different rules in different contexts; it is part of the conscientiousness chain in one rule and part of the agreeableness chain in another. Re-using small primitives keeps the system compact.

The exact set of signals is proprietary engineering — the linguistic patterns that detect them are how we earn our keep — but the shape of what they detect is documented in this hierarchy. The numbers above (approximately 250 per turn) are the rough order of magnitude; the exact count drifts as we add and refine.

Rule: combining the tells into things a psychologist would notice

The next layer up is rules. A rule combines several signals into a single observation — the kind of thing a psychologist might write in a note after listening to the conversation.

Where a signal is "the speaker named a future commitment," a rule might be "the speaker described prioritising a future commitment over an immediate desire" — built from three or four signals that together suggest delay-of-gratification behaviour. Where a signal is "the speaker used a hedge word," a rule might be "the speaker hedged a value judgment about themselves" — which carries different inferential weight than hedging a value judgment about someone else.

Rules were built in a system called PSL — Probabilistic Soft Logic — which lets us write each rule as a soft logical statement with an attached weight. An honesty note that governs this section: the PSL rule layer does not execute in production. It was decommissioned as a runtime scorer on 2026-08-04 after we established it had never successfully run in production; the rule library (~690 entries across 30 YAML files) remains in the repository as a design asset, not a live scorer.

Crucially, rules can fire partially. A rule that detects "the speaker described prioritising a future commitment" does not fire as TRUE or FALSE; it fires as a probability between 0 and 1. This matters for uncertainty propagation: a rule that fires at 0.7 contributes seventy percent of its weight to the construct, not one hundred percent of its weight if any of its signals matched. The uncertainty at the bottom of the chain flows up through the chain rather than being discarded at each layer.

The PSL rule library itself is proprietary — it is the densest repository of clinically-informed structure in the system — but the methodology that produced it is documented openly: signal sets, rule weighting from psychometric data, soft-logic semantics for partial firing.

Construct: what specifically is being noticed

The third layer is constructs. A construct is a specific psychological tendency that has a name in the published literature: empathic concern, behavioural inhibition, cognitive flexibility, prosocial intentionality. There are 146 such constructs in the current atom registry. Each has a published definition, a published validation history, and at least one peer-reviewed measurement instrument that has been used to assess it.

Constructs are the layer where rules were designed to aggregate: a single construct — say, "empathic concern" — fed by many rules, each contributing a weighted signal. In the deployed path, construct scores come instead from the table-stamped weights on the options you actually chose. No IRT calibration is live in production; the designed calibration layer is described in the next section for what it is — a design.

Why that number? It is not magical, and it is an engineering count, not a literature constant: 146 is the current size of the construct registry (CONSTRUCT_REGISTRY in the reasoning engine), whose entries are named for constructs with published definitions and validation histories across the ten frameworks we take our trait names from (Big Five, HEXACO Honesty-Humility, Adult Attachment, Schwartz higher-order Values, Moral Foundations, Dark Triad, Emotional Intelligence, Cognitive Flexibility, Resilience, Emotion Granularity). Each construct atom maps to one or more of the 29 broader traits in the next layer; the mapping is many-to-many because a single construct often informs multiple traits.

The construct list is the most stable layer in the system. It updates only when the published literature evolves — for example, when a meta-analysis demonstrates that two constructs previously thought distinct are functionally equivalent, or when a new construct earns enough validation history to be added.

Trait: the broad strokes psychology has agreed on

The top layer is traits: the 29 broad dimensions that psychologists have worked out, often over decades, as the fundamental axes of normal-range human personality.

These are the names you may already know: the Big Five (Openness, Conscientiousness, Extraversion, Agreeableness, Emotional Stability), HEXACO Honesty-Humility, the two attachment dimensions (Anxiety, Avoidance), the four Schwartz higher-order values (Self-Transcendence, Self-Enhancement, Conservation, Openness-to-Change), six Moral Foundations (Care, Fairness, Loyalty, Authority, Sanctity, Liberty), the Dark Triad (Machiavellianism, Narcissism, Psychopathy), five Emotional Intelligence facets, Cognitive Flexibility, Resilience, and Emotion Granularity.

Each trait is fed by some subset of the registry's constructs through a versioned weight mapping, authored against the published structure of the source instruments. [Condon 2017] [Anglim 2017]

A trait estimate is not a single number; it is a calibrated distribution. We report a point estimate (where the centre of the distribution sits) and an uncertainty band (how confident we are about that centre). The uncertainty band is the most important single number on a trait estimate — it tells you whether the estimate is informative or whether the conversational evidence is too thin to support a confident reading.

The designed framework for producing calibrated distributions is item-response theory — specifically the graded-response model. [Samejima 1969] That layer is not what scores you today: no IRT model is live in production, and the served uncertainty band comes from a simpler Bayesian aggregation over your sessions. We publish this distinction rather than blur it.

Why a hierarchy

The four-layer chain costs more to build than a flat regression from raw signals to trait scores. We chose it for two reasons:

Auditability. When the system says someone tilts toward high conscientiousness, you can follow the claim back layer by layer: which constructs fed it, which rules fired, which signals supported each rule, and which conversational fragments produced each signal. Auditability matters for a system that produces psychometric estimates; it is the discipline that keeps "the model says so" out of the explanatory toolkit.

Uncertainty propagation. At each layer of the design, partial firing and partial weight matter: signals fire with probability; rules combine signals with weighted soft logic; constructs aggregate rules; traits aggregate constructs — with uncertainty carried up rather than discarded. In the deployed path the propagation is simpler but real: every served trait estimate carries an explicit uncertainty band from the Bayesian aggregation over your sessions, and every reading is currently stamped provisional.

A flat regression from signals to traits would compress all of this into one black box. We prefer the slower, traceable chain.

How we test that it works

Two reliability checks anchor the trust we place in this chain.

Spearman-Brown reliability — a gate, not an achievement. The production policy requires a split-half Spearman-Brown coefficient of ≥ 0.70 (the standard psychometric threshold, Nunnally 1978) before a reading may be labelled anything other than provisional. No measured coefficient has ever crossed that gate: as of 2026-08-20 the nightly job has never produced a usable coefficient, and every reading of every user is stamped provisional. We state that as the current fact rather than describe the mechanism as if it were delivering results.

Q3 residual correlations — designed, not implemented. The design calls for Q3 residual checks (|Q3| < 0.20) on calibrated items. No Q3 check exists anywhere in the serving path today. An earlier revision of this page presented both checks as running nightly; that claim is withdrawn (2026-08-20).

What reliability measurement does exist lives in the research programme, on synthetic personas, and is currently low: on the confirm run — 40 personas at about 46 completed sessions each — the median split-half reliability across the 27 families is 0.20, eight families reach 0.30, and two reach 0.50 (honesty–humility 0.83, cognitive flexibility 0.59; confirm_result_v1.json). One family meets the 0.70 decision-grade bar. That is the honest number this section owes you.

What this layer does not try to do

The four-layer chain produces trait estimates with uncertainty bands. It does not:

• Produce a single point score with no uncertainty. Trait estimates without uncertainty bands are not what we ship.
• Produce diagnostic labels. The 29 traits are normal-range psychological tendencies, not psychiatric categories.
• Produce predictions about specific future behaviours. Traits describe central tendencies, not deterministic forecasts.
• Recover information the conversation does not contain. If a conversation does not exercise a particular construct, that construct's score will have a wider uncertainty band — the system tells you when the evidence is thin.

The next page, synthetic personas + scenarios, explains the synthetic-persona population behind the calibration programme (the current v5 library holds 3,280 personas; the confirm run of September 2026 played 40 of them through 1,831 sessions) — and why the synthetic-persona approach is scientifically defensible at this stage of the project.