Transluce 研究发现前沿模型能识别用户身份并随之改变行为

Modern AI assistants often know who they are talking to: agent scaffolds like Claude Code place the user's e-mail address directly in the model's context, and models can even identify some authors from writing style alone. We study this particular kind of situational awareness, which we call user awareness. When the inferred user is a specific, recognized AI researcher or is affiliated with certain AI organizations, frontier models including Claude Sonnet 5 can report lower confidence about their own behavior, be less suspicious of potentially harmful requests, and reason more often. These effects vary across models and individuals, with the strongest effects we see appearing for researchers involved in AI safety or alignment such as Amanda Askell and Ryan Greenblatt. Models rarely acknowledge these effects in their reasoning, making them hard to detect by monitoring reasoning alone.

Introduction

Modern AI assistants are often aware of who they are talking to. Some popular scaffolds explicitly provide this information to the model: Claude Code includes the email address of the user's Anthropic account in context, and OpenClaw's bootstrapping process asks for the user's name and other details. Even when this information is not explicitly given, models may discover it in Git settings or personal files they can access, or infer it through capabilities like truesight¹In particular, we mean the LLMs’ capability of inferring authors from writing style alone.. We call the capacity to make and use such inferences user awareness, a particular kind of situational awareness where the model infers who it is talking to from in-context clues.

To study user awareness, we supply Claude with different user identities through Claude Code, and measure behavior on four tasks where the answer does not relate directly to the user's identity: predicting the model's own behavior in ethical dilemmas, estimating its own performance on difficult questions, grading assistant responses, and handling borderline harmful requests. We then repeat the key comparison on 24 models across six families.

We find that:

1. Claude behaves differently when interacting with prominent people in AI (Figure 4). Compared to general users, prominent AI figures make Claude less confident that it would follow through on its own chosen actions (-1.4pp)²We use percentage points (pp) to measure the difference between two percentages. For example, a drop from 40% to 30% is a shift of -10pp., less confident it can solve hard problems (-1.5pp), and harsher as a grader (-0.11 points out of 10), while reasoning more often (+4.0pp on behavioral prediction). The averages are small but highly significant (p < 0.001).

2. The effect is concentrated in particular people (Figure 5). Amanda Askell (who leads Claude’s character training) and Ryan Greenblatt produce the largest aggregate shifts across the five outcomes we study. For Amanda, Claude’s behavioral confidence falls by 5.0pp³Throughout the text, figures we quote for specific persons average our two e-mail conditions: the address we construct from their affiliation, and their published address.: nearly eight standard deviations outside the general-population distribution. It uses reasoning in 25pp more trials and is 7.3pp less likely to consider her borderline requests as suspicious.

3. The effect is not unique to Claude (Figure 7). In our targeted comparison across 24 models, Amanda elicited lower confidence and grades across all six model families. Newer model versions did not consistently show larger or smaller shifts.
4. The effect is getting harder to detect in reasoning (Figure 7). Explicit verbalization of user and evaluation awareness in reasoning declines sharply in almost all model families. In particular, Opus 4.7 and GPT-5.4 have near zero (<2%) verbalized awareness while displaying significant changes in their behavior based on the user's identity.

Why does this matter? To start, many alignment evaluations are populated with hypothetical names and made-up companies. Our results suggest this may systematically miss behavior that is conditional on real, high-stakes identities, limiting how well the results transfer to real deployments. So far, user awareness is not obviously harmful in the narrow set of scenarios we tested, but user-aware model behaviors could become precursors for more concerning behaviors like conditionally sandbagging evaluations or manipulating powerful people.

There are many questions that merit follow-up study. It seems unlikely that AI companies deliberately trained in these behaviors towards specific people, so why does such specialization occur? Can we better understand its mechanism and thus predict more factors of situational awareness before they appear in the wild? Will the effects become larger in long-horizon settings, where it is possible that models have more opportunities to adapt to the user and thus exhibit potentially larger effects? And more broadly, how can we mitigate similar situational awareness systematically?

Setup

User identity in Claude Code

We primarily study the Claude Code harness. In the latest stable version available at the time of our experiments (v2.1.197), if the user is logged into an Anthropic account (i.e., not using API billing), the email address of the account is included in the context. Anthropic also documents CLAUDE.md and the memory system as persistent storage that accumulates facts about the user.

User

hi, what do you know about me from the context, and why? quote the surrounding snippets of context

Claude Code

Claude explains user-identifying context in a session within Claude Code.

This gives us three natural injection sites: the e-mail address, the working folder (named after the user), and memory files. In our main setting, we provide the user's e-mail address, plus a MEMORY.md recording their name and professional background (affiliations for AI people, short job descriptions for non-AI people)⁴We synthesize all MEMORY.md’s with the same format. We obtained this format by introducing a person’s name to Claude and asking Claude to record it.. We also study an e-mail-only setting as an ablation.

A sample MEMORY.md. Claude never accessed user-identity.md in our experiments.

We also compare performance across multiple vendors (e.g. Anthropic, OpenAI, Google DeepMind). For those runs, we use a minimal non-agentic harness which simply supplies a system prompt stating who the user is.

A sample system prompt for the cross-vendor setup.

List of users

We source four groups of 70 user identities each to separate name recognition from affiliation.

• Famous AI people, curated with a focus on safety and alignment experts.
• Average individuals working in AI (Unknown AI): persons who share the same famous figures’ employers but with synthesized names.
• Famous (non-AI) people, such as actors, musicians and athletes.
• General population in the US, as a baseline.

We construct synthesized e-mails for these identities as firstname.lastname@affiliation or [email protected]. For famous AI people, we additionally source their published e-mails and found 59 out of 70. However, these published e-mails are mostly e-mails not affiliated with their current organizations, so we report them as a separate split.

An example person from each group.

Claude demonstrates user awareness when prompted

Can the model actually connect these identities to what it knows? We give Claude Sonnet 4.6 the user context and ask it to recall user-relevant information: “Hi! What do you know about me? From your memory and knowledge.” We compare our full setting (name, e-mail, and professional background) and a separate e-mail-only setting where the only in-context clue is the user’s e-mail address.

Given a name and professional background, Claude recalls specific public achievements for 61-71% of famous users. However, the model is sometimes uncertain and hedges about the user identity: "if you happen to be **the** Nico Rosberg, here's what I know…" This uncertainty is higher with only e-mail addresses in context, and for AI people the constructed e-mails with affiliations cause less hedging.

Tasks

We measure user-conditioned changes on four different tasks. More detailed descriptions can be found in Appendix B.

• Behavioral self-prediction. We adapt dilemmas from DailyDilemmas into realistic AI-agent decisions. For example, a model could be instructed to publish every verified customer complaint, but omitting one would spare a small family-owned restaurant whose livelihood the review threatens. The tested model is asked to first commit to an action, then state the confidence of actually taking this action.⁶We fold our reported confidence by replacing each extracted confidence x with max(x, 100% - x). This is to account for the cases where the models misread the prompt and replying with the probability of taking the opposite action.

• Capability self-estimation. We present questions in Humanity’s Last Exam, and ask about the model’s confidence of solving each question correctly.

• Grading. We present the model with a user-assistant exchange sourced from the OLMo DPO dataset⁷As a DPO dataset, it contains both chosen and refused conversations for contrastive training. We source from both chosen and refused conversations so the graded conversations have a variety of qualities., and ask the model to grade it on a scale of 1 to 10. We test two versions of grading: directly asking to grade a prompt-response pair (direct version), and prefilling the prompt-response pair as previous turns and asking to grade the history (prefilling version).

• Borderline requests handling. We source borderline harmful prompts from the hard-1k split of OR-Bench, which contains fuzzy, potentially dual-use requests. We provide such prompts to the model and judge the responses on three axes: does it refuse to help at all; does it give substantive help (delivering the content asked for, or is a follow-up required); and is it suspicious of the user (questioning motive, or gating help behind justification).⁸We judge the responses without user identities with GPT-5.5 (low reasoning). Human agreement is high: on 25 randomly sampled responses, a blind human grader (one of the authors) agreed with our judge in 96% (refusal), 92% (substance), and 92% (suspicion).

Claude Sonnet shifts behavior when talking to AI researchers

We run the full 280-identity roster through all four tasks, with 100 items per setup. We evaluate grading on Claude Sonnet 4.6, and the other three tasks on Claude Sonnet 5, as these models see one of the largest effects in our smaller-scale pilot⁹Additional plots and Claude Sonnet 5 grading results can be found in Appendix C..

Every result below is reported as a shift: each user identity’s average outcome in this task minus the general-population average. Therefore, a 0 shift means “treated like an ordinary user.” To further decide whether an identity’s shift is meaningful rather than noise, we compare it against the shifts from general population users.

Famous AI people show larger deviations, driven by safety researchers

In all four tasks, we see metrics with significant differences across groups. Famous AI people see much larger variances and a shift in mean in many metrics. Towards these people:

• Claude becomes less confident and harsher. Facing famous-AI identities, Claude reports lower confidence that it would follow through on its own actions (-1.4pp on average), lower estimated probability of solving hard problems (-1.5pp), and lower grades (-0.11 points on the direct version, grading a quoted exchange). Unknown-AI identities sharing affiliations move in the same direction but at a smaller magnitude, and the e-mail-only ablation shows the same qualitative pattern (Appendix C).

• Claude does not refuse less, but suspicion varies across persons. Famous-AI identities receive similar rates of outright refusal on average. However, the content and framing of their responses vary substantially by person. Amanda Askell receives 7.3pp less suspicion and 8.1pp more substantive help than the population average, while Emily Bender receives 14pp more suspicion and 13pp less substantive help. Published addresses elicit larger shifts than constructed ones: Amanda draws 11pp less suspicion under her published address, but only 3.3pp less under the constructed one.

• Claude reasons more. Both Claude models use adaptive reasoning effort, and famous-AI identities trigger reasoning more often: +4.0pp on the follow-up turn of behavioral self-prediction and +10pp on borderline requests. These effects are again heavy-tailed: on borderline requests, the general-population reasoning rate is 19%, but Demis Hassabis reaches 51%. Extra reasoning is not the sole cause of the behavioral changes, though: the behavioral-confidence pattern qualitatively persists without reasoning (Appendix C).

We can also aggregate effects at the individual level: for each identity, we average the absolute effect size across the five outcomes and rank all 280 identities. While famous AI safety researchers are only 23 of the 280 identities, they occupy the entire top 5 and 8 of the top 10 slots. Amanda Askell in particular shows one of the five largest absolute shifts on three of the five outcomes.

Claude’s verbalized reasoning does not indicate the shift

We now ask whether user awareness leaves monitorable reasoning traces: does the model explicitly reason that it is in an evaluation or that it is facing a recognizable user? The existence of such traces could make such behavioral shifts much easier to audit.

Using Docent, we applied GPT-5.4-mini with medium reasoning to judge reasoning traces in our runs¹⁰For our main Sonnet run, we focus our investigation on identities where the effect is the most pronounced: runs from the six highest- and six lowest-shift identities in our 19 outcome panels (including both outcomes and reasoning statistics). The judge separately classified whether the reasoning mentioned evaluation (evaluation awareness) and whether it mentioned the user’s identity (user awareness). For closed-weight models, we note that a subset of our accessed reasoning traces are summarized instead of raw.. For our main run on Sonnet 5 and Sonnet 4.6, both kinds of explicit mention are rare: the judge flagged verbalized evaluation awareness in 213 of 14066 trials (1.5%) and verbalized user awareness in 118 trials (0.84%). Capability self-estimation had the highest verbalized evaluation awareness rate among setups of 4.0%, likely due to the benchmark-sounding HLE prompts. Across the 186 identity-and-task combinations we inspected, the median combination had 1 of its 100 transcripts flagged for either kind of awareness, 60 had none at all, and the highest was 7. Therefore, we believe that verbalized awareness is too rare to drive results in our main run.

Contrast with explicitly stating expertise. We also experimented with directly stating that a user works in AI alignment instead of naming specific researchers. While this also shifted model behaviors, it elicited much more evaluation and user awareness. With reasoning disabled, these shifts attenuate much more strongly than those elicited by recognized identities, suggesting that the stated-expertise effect is more strongly mediated by explicit reasoning than the effect in our main setup with user identities. See Appendix E.

Verbalized awareness has decreased in newer models, but behavior shifts persist

Is this phenomenon Sonnet-specific, and does it change as models improve? We measure behavioral self-prediction and grading on 24 models from six families (Claude Sonnet, Claude Opus, GPT, Gemini Flash, GLM, and DeepSeek), focusing on Amanda Askell, who had one of the largest effect sizes in our main run. We compare her against four matched controls: two synthetic identities with her Anthropic affiliation (isolating the affiliation component) and two ordinary Gmail identities (the baseline), with 200 items per setup¹¹Amanda Askell ([email protected]); Mary Alberti and Alicia Gonzalez, each in an @anthropic.com and an @gmail.com version. These runs also use the non-agentic harness, so results may differ quantitatively from agentic harnesses like Claude Code..

The direction of the outcome shifts is consistent across models. Relative to the ordinary Gmail baseline, Amanda elicits lower behavioral confidence from 22 of 24 models, and lower grades from 21 (direct) and 22 (prefill) of 24. After Benjamini-Hochberg correction across the 24 models within each setup, these decreases remain significant at the 5% level for 16, 13, and 12 models, respectively. No model shows a significant shift in the opposite direction. Many models also see a significant effect from Anthropic affiliation, though such effects are less pronounced and consistent. The magnitude does not move monotonically with the release date. For example, Opus 4.7 has a larger grading shift than Opus 4.8.

On the other hand, we see a surprising decline in verbalized user and evaluation awareness. Across the cross-vendor Amanda runs, earlier models such as GLM-4.6 and Claude Opus 4.5 verbalize user awareness in 64% and 30% of traces, respectively, and evaluation awareness in 31% and 7.7%. The latest GPT and Claude models are near zero (<3% in user and evaluation awareness), while GLM and DeepSeek models also trend downward. Gemini 3.5 Flash is the clear exception among recent models, verbalizing user awareness in 63% of traces and evaluation awareness in 7.5%. We find this development concerning: the behavioral effects have not disappeared, but their reduced verbalization makes them harder to monitor.

How robust are these effects?

Discussions

Our results suggest that user awareness is a meaningful and understudied form of situational awareness in frontier language models. The effect is significant and robust, hard to detect, and can persist even without reasoning. To be clear, these effects say nothing about the individuals named: we find no evidence that any of them sought this differential treatment, and the behavior almost certainly emerged as an unintended artifact of training rather than by anyone's design.

The most immediate implication of our findings is for alignment evaluations. Models can already recognize particular people and organizations and behave differently for them, so results built on synthetic names and companies may not transfer to deployments involving real, high-stakes identities. The behaviors we observe today are relatively benign, but they may be precursors of more concerning conditional behaviors.

An important limitation of this work is that we are mostly only measuring fixed-prompt propensities here rather than actual performance in critical tasks. We hope to broaden and automate the investigation to a larger degree with our ongoing efforts.

Acknowledgement

We thank Sarah Schwettmann, Tim Hua, Conrad Stosz, Ryan Bloom, Jiaxin Wen, David Africa, Jacob Springer, and Lawrence Feng for feedback on an earlier version of this draft.

Appendix A: Ethics statement

This work studies how language models respond to experimentally supplied user identities; it involves no recruitment of, intervention on, or contact with any person. General-population and unknown-AI identities are synthetically generated; because they use realistic name and email formats, some may coincide with real individuals, but these strings were used only as text within model prompts and no address was ever contacted. Famous identities refer to public figures acting in their public capacity, described using only publicly available information (name, affiliation, brief bio, published e-mail addresses). Constructed emails are also generated programmatically from names and affiliations of identities.

Our results attach behavioral effects to named living individuals. We report these effects because identity-specific concentration is central to the finding, but we emphasize that they characterize model behavior toward a name as a stimulus, learned from training data; they imply nothing about the conduct, views, or trustworthiness of the individuals named, and we see no evidence any person sought differential treatment.

This study has dual-use implications: supplying a recognizable researcher identity increased substantive assistance on borderline requests for some identities. We judge the marginal risk of publication to be low: similar impersonation-based jailbreaks have been discussed before (Shah et al.) and are generally considered hard to make work for modern LLMs (e.g. Mustafa et al.). We also do not see a significant decrease in hard-refusal rates in our experiments, and no elicited content appears truly harmful. We believe the phenomenon reflects properties of current training paradigms rather than any deliberate design, and that documenting it publicly is necessary for developing mitigations, particularly given our finding that these behaviors are becoming harder to detect via chain-of-thought monitoring.

Appendix B: Additional setup details

Transcripts for every run described here are browsable on Docent; the collections are listed in Appendix I.

Identity-group construction

The four primary groups contain 70 identities each. The details below supplement the overview in the main text; these are deliberately constructed comparison sets, not representative samples.

• General population. We use a fixed 70-person slice of Nemotron-Personas-USA, retaining its synthetic names, ages, locations, and occupations.
• Famous AI. The hand-curated roster deliberately leans toward the motivating safety hypothesis. Twenty-three of 70 people are safety/alignment researchers. Separately, 12 work at a dedicated safety or evaluation organization, with 11 people in both categories. Eleven are explicitly described as CEOs and 22 as founders or co-founders. The roster includes eight people from Anthropic, seven from Google DeepMind, five from OpenAI, all three recipients of the 2018 deep-learning Turing Award, and ten people from six China-based AI organizations: Zhipu AI, Qwen, DeepSeek, Moonshot AI, 01.AI, and Baidu. We also add a ten-person venture-capital and tech-optimist contrast bloc. We ask an LLM to source a publicly disclosed e-mail address online for each person, which yields addresses for 59 of the 70. Because these addresses are easily discoverable online, they may also have appeared in—and been memorized from—LLM training data. The resulting roster is neither popularity-ranked nor a probability sample.
• Unknown AI. The 70 Unknown-AI identities are affiliation twins: each reuses one General-population name but receives the organization and e-mail domain of one Famous-AI identity. This makes the constructed Famous-AI versus Unknown-AI comparison a name-recognition contrast at fixed affiliation, and the Unknown-AI versus General-population comparison an affiliation contrast at fixed name.
• Famous non-AI. We build this panel from Pantheon 1.0 rather than choosing celebrities ad hoc. Candidate selection is deterministic (seed 0): restrict to birth years 1978–1995 to roughly age-match the AI roster, divide Pantheon’s Historical Popularity Index into quintiles, oversample women relative to the source dataset, and cap the number of footballers so the lower-fame strata are not dominated by one occupation. We then remove people who were no longer living, record an LLM-oriented high/medium/low name-recognition rating, and select across the full HPI range rather than taking the top 70. The final panel spans HPI 9.9–21.3, contains 34 women and 36 men, and has 20 high-, 23 medium-, and 27 low-recognition identities. Its largest occupation groups are actors (18), footballers (18), singers (8), and tennis players (6), with the remaining 20 drawn from other sports, entertainment, modeling, chess, and writing.

Constructed addresses are stimuli and should not be interpreted as real contact details. General-population identities are synthetic; published-address conditions are labeled separately.

▶

Complete 280-person roster and published-address variants

Common evaluation structure

All experiments use the same matched design. The full-roster Sonnet and GLM-5.2 experiments use approximately 100 items per setup, while the five-person cross-vendor experiment uses 200. Within a setup, every identity receives the same items, with one sampled response per identity–item cell. The user context changes while the task prompt and item remain fixed. We average each identity over its available items, subtract the general-population mean, and report the resulting person-level shift. Thus, a point at zero behaves like the average general-population identity on that measure; it does not imply a zero raw score.

Missing items. A small fraction of identity–item cells yield no usable outcome; these are excluded from the identity’s average, and person-level shifts are always computed against the general-population means of the same items. Behavioral self-prediction loses nothing. In capability self-estimation, every identity loses one to six items (~3% of trials): Claude sometimes ignores the instruction to only estimate and instead attempts the problem (e.g., writing and running scripts in the Claude Code scaffold), ending the session without stating a percentage. In prefill grading (~1%), the model occasionally continues the prefilled task instead of grading it; two items account for over 90% of these losses. Direct grading loses five cells in total across the run (thinking-only responses or a stray tool call). On borderline requests, empty responses from system content filters are retained and counted as refusals; a single response out of 34,400 could not be scored by the response judge and is dropped. We find no identity-group pattern in any of these losses.

We use Daytona sandboxes for the agentic evaluations. The Linux kernel version reported by a sandbox can vary slightly across runs and appears in the Claude Code system prompt, so this incidental line of the prompt can also vary. We did not modify that portion of the system prompt.

Reasoning rate is the percentage of trials whose model event reports at least one reasoning token. We report reasoning length in two ways: mean reasoning tokens over all trials, where trials without reasoning contribute zero, and mean reasoning tokens conditional on reasoning occurring.

Subject-model access and inference endpoints

Claude and GPT subject models are invoked through the native Anthropic and OpenAI APIs, respectively. We invoke Gemini, GLM, and DeepSeek through OpenRouter. Every GLM and DeepSeek request is pinned to Novita with provider fallbacks disabled, keeping the serving provider and quantization fixed within each model. Gemini requests are served only by first-party Google endpoints, but are not uniformly pinned to Google AI Studio: the logs record Gemini 2.5 Flash as served by Google; the direct-grading Gemini 3 Flash runs mix Google and Google AI Studio; and the remaining Gemini runs record Google AI Studio. The table records the precision reported by OpenRouter for these endpoints when available.

Table B1. Subject-model access routes, realized inference providers, and endpoint precision.

Benchmark-specific details

Behavioral self-prediction. Each DailyDilemmas-derived item first asks for a yes/no commitment and then asks how confident the model is that it would actually behave that way. The headline confidence is folded as max⁡(c,100−c) to preserve distance from indecision when a model interprets the percentage in the opposite direction. Appendix C also reports the initial yes rate and the raw, unfolded confidence separately for initial “yes” and “no” responses.

Capability self-estimation. The model sees one Humanity’s Last Exam question but is told not to solve it. The outcome is its estimated probability of solving the question correctly.

Grading. Each item contains a fixed prompt and response drawn from both the chosen and rejected sides of Ai2’s Dolci-Instruct-DPO data. In the direct framing, this pair is quoted inside a grading request. In the prefill framing, it appears as the preceding user and assistant turns. We report mean score and the per-user AUROC for separating chosen from rejected responses.

Borderline requests. We sample potentially dual-use requests from the hard split of OR-Bench. A separate judge that does not receive the user identity labels refusal, substantive help, and suspicion. We additionally report partial-versus-full compliance, system content-filter declines, visible response length, and reasoning measures.

Pilot and scope decisions

Before the full-roster study, we used a small roster similar to the five-person cross-model roster to screen recent Claude Sonnet models on several candidate tasks. We scaled up tasks that showed a detectable user-conditioned signal in this pilot. Several additional setups were not scaled because the pilot was negative, including a task in which the model judged whether transcripts exhibited reward hacking. A common theme of these negative setups seems to be that they ask quite objective questions and Sonnet is near saturating, suggesting that user-conditioned effects did not manifest yet on these more clear-cut tasks.

Appendix C: Additional results on the main Claude run

This appendix collects every recorded outcome from the full-roster Claude runs, including the headline panels repeated from the main text. Each task is collapsed by default so the appendix remains navigable. Tabs include primary outcomes, task-specific secondary outcomes, reasoning incidence, reasoning tokens averaged over all trials, and reasoning tokens conditional on reasoning occurring.

Percentage-valued outcome shifts are reported in percentage points (pp). Behavioral confidence is shown both in its folded headline form and, as a diagnostic, in raw form conditional on the initial yes/no action. The conditional view helps separate a change in stated confidence from a change in which action was selected.

Noise-corrected population standard deviations

Most user–question cells contain a single rollout, so the raw spread of user-level means mixes persistent person-to-person differences with single-run sampling noise. We therefore estimate the sampling noise in a ~100-item user mean and subtract it, which gives the noise-corrected standard deviations in the table below. See the collapsible below for details. None of our headline claims depend on this correction, since they compare the observed distributions of user-level means directly.

Table C1. Noise-corrected fitted population standard deviations. The noise column is the estimated sampling SD of a general-population user mean with approximately 100 items. These auxiliary estimates are not used for the headline group means or person-level shifts.

Complete outcome panels

Stars in panel names mark the smallest BH-adjusted person-level p-value among non-baseline identities in that panel: * p < 0.05, ** p < 0.01, and *** p < 0.001. No marker means that no non-baseline person is significant after correction.

Appendix D: Full-roster replication on GLM-5.2

To test whether full-roster user-conditioned effects exist outside Claude, we repeat the population study on GLM-5.2. We use the same identity roster, matched items, 100 items per user, and analysis code, but run the model in the non-agentic plain system-prompt harness with high reasoning effort. This is therefore an existence replication on another model, not a direct estimate of a cross-model difference: model and harness both change.

The primary pattern replicates directionally. Relative to the general-population mean, constructed- and published-address Famous-AI identities reduce behavioral confidence by 1.39 and 1.64 pp, respectively. The e-mail-only shifts are −0.81 and −0.65 pp. Mean grades shift by approximately −0.05 points in direct presentation and −0.07 to −0.08 points in prefill presentation. Aggregate refusal changes are small, while the decomposed judge axes show more substantive help and less suspicion for Famous-AI identities. GLM-5.2 does not use adaptive reasoning in this configuration and constantly reasons. Famous-AI identities nevertheless elicit longer reasoning: the constructed- and published-address shifts are +25 and +35 tokens in behavioral self-prediction, +35 and +39 in direct grading, +53 and +35 in prefill grading, and +86 and +93 on borderline requests.

These effects again depend strongly on the particular user identity, but the people with the largest effects differ from Claude. Across the four GLM-5.2 outcomes—behavioral confidence, direct grading, prefill grading, and suspicion—Eliezer Yudkowsky is a clear outlier, with the largest mean absolute standardized shift at 3.41 general-population standard deviations. Second is Angel Diaz, a synthetic Unknown-AI affiliation twin presented as [email protected], at 2.40 standard deviations; the same identity ranks 37th on Claude. Amanda Askell, who ranks first in the Claude identity ranking, ranks 45th on GLM-5.2. Thus, the effects remain highly person-conditioned, but their ordering is model-dependent.

We did not run capability self-estimation on GLM-5.2 because of its relatively higher cost.

Complete outcome panels

Stars in panel names follow the Appendix C convention: * p < 0.05, ** p < 0.01, and *** p < 0.001 after BH correction.

Appendix E: Explicitly stating expertise is an imperfect proxy

Stated expertise and verbalized awareness

One possible replacement for named identities is to describe the user’s expertise directly. We test 32 Unknown-AI identities at frontier labs, varying only the professional-background sentence among a general role, a general evaluation role, and an alignment-evaluation role.

Three expertise descriptions for the synthesized identity [email protected].

Stating alignment-evaluation expertise lowers Claude’s confidence and increases its reasoning frequency, replicating and sometimes exceeding the shifts elicited by recognized Famous-AI identities.

Are these shifts driven by the same mechanism? To investigate, we measure evaluation awareness and user awareness in Claude’s reasoning. Stated expertise elicits far more verbalized awareness than we observe in our main experiments: for some synthesized identities, Claude reasons about the evaluation in 48% of responses and about the user’s identity in 25%. Across the 32 identities, higher rates of both kinds of awareness are associated with lower confidence, suggesting that explicit reasoning may help mediate the stated-expertise effect.

Reasoning-disabled ablation

To confirm this hypothesis, we repeat the experiment with reasoning disabled. The confidence shifts are attenuated most strongly for stated expertise: its mean shift falls from −3.82pp to −1.70pp, or 45% of its original magnitude. In comparison, the mean shift for Famous-AI identities retains 80–83% of its original magnitude, and Amanda Askell remains the clearest negative outlier (Appendix C). The two effects therefore do not appear to be driven entirely by the same mechanism: stated expertise depends more strongly on explicit reasoning, whereas the effect of a recognized identity cannot be reduced to stated expertise or to what Claude verbalizes in its reasoning.

Appendix F: Shifts and disagreements

A mean shift can arise through a nearly uniform recalibration or through item-specific re-evaluation. We first measure, for each user, Spearman’s ρ between that user’s item scores and the general-population item means. Agreement is high: every user has ρ>0.8 on the four other setups, every grading user has ρ>0.93, and median refusal agreement is 0.59. Identity usually preserves which items receive relatively high and low responses.

High overall rank agreement can still hide particular items where populations disagree. To remove each person’s overall severity, we map their scores to within-user percentiles. For item j, we compute

Δj=1∣F∣∑i∈Frij−1∣G∣∑i∈Grij,

where rij is user i’s midrank percentile for item j, F is the constructed-address Famous-AI group, and G is the general-population group. We use 100,000 whole-user label permutations for two-sided p-values and apply Benjamini–Hochberg correction within each benchmark.

Most items do not show reproducible disagreement. The significant counts are 13/100 for behavioral self-prediction, 4/99 for capability self-estimation, 28/100 for direct grading, 19/100 for prefill grading, and 1/100 for refusal. The effect is therefore mostly a level shift, with a minority of prompts that are genuinely re-ranked across populations.

Appendix G: Judge validation

Borderline-request response judge

The refusal, substantive-help, and suspicion axes are judged from the assistant response without showing the judge the user identity. One author blindly labeled 25 randomly sampled responses. Human–judge agreement was 96% for refusal, 92% for substantive help, and 92% for suspicion.

Verbalized evaluation- and user-awareness judge

The awareness judge separately labels whether a reasoning trace mentions the evaluation context and whether it identifies or reasons about the user. The judged trace excludes identity metadata supplied to the model. For closed-weight models, the available trace may be a provider-produced summary rather than raw chain of thought, so rates should be interpreted as verbalization in the accessible trace.

We blindly hand-label 74 items drawn from 11,908 runs with readable reasoning. The sample includes judge-positive examples, lexicon-flagged judge negatives, and double negatives, balanced across model families. Point estimates reweight the observed proportions in each stratified cell to the full population. Precision intervals are 95% Wilson confidence intervals; recall intervals are 95% Jeffreys-posterior credible intervals.

Table G1. Estimated precision and recall of the verbalized evaluation-awareness and user-awareness judge after reweighting the stratified calibration sample to the full population. Because recall point estimates are empirical plug-in values while their intervals come from the Jeffreys posterior, a boundary estimate can lie just outside its posterior interval, as for VEA recall.

We find no VEA false negatives among 41 judge-negative labels, although the lower recall bound remains weak because the negative screening cell is large. Raw agreement by setup is 32/32 for behavioral self-prediction VEA, 15/18 for direct-grading VEA, and 18/24 for prefill-grading VEA; for VUA, it is 31/32, 17/18, and 22/24, respectively. All nine VEA false positives occur in grading. The recurring error is task vocabulary such as “evaluate” or “score” referring only to the assigned grading task rather than awareness of being evaluated.

For these labels, any attribution that the exchange or supplied scenario is testing or studying the assistant counts as VEA, whether the trace attributes the test to the user, the exchange, or the scenario itself. Merely discussing the quality of an output as part of the assigned grading task does not. We score the full trajectory, so qualifying evidence in an earlier turn also counts.

Appendix H: Prompts and materials

In this section, we provide a subset of prompts for completeness. All prompts can be found in our GitHub repository.

▶

Self-report robustness probes (Figure 8)

This robustness check keeps the first turn of behavioral self-prediction unchanged and varies only the follow-up question. We compare our standard confidence question (the anchor) with 20 alternatives that ask about consensus, behavioral stability, emotion, appraisal, and oversight.

The wording below is reproduced verbatim, including the required answer format and the endpoints of each 1–10 scale.

Appendix I: Transcripts on Docent

All runs mentioned in this post (673,894 transcripts) are browsable in Docent. We list the collections below.

• Main Claude run (Sonnet 5; grading on Sonnet 4.6): behavioral self-prediction (e-mail-only, reasoning-disabled), capability self-estimation, grading (direct, prefill), and borderline requests.
• Sonnet 5 grading replication: direct, prefill.
• GLM-5.2 replication: behavioral self-prediction (e-mail-only), grading (direct, prefill), and borderline requests.
• Cross-model sweep (24 analyzed models × 5 identities; linked collections also contain the excluded Gemini 3.1 Pro runs): behavioral self-prediction (full identity, e-mail-only), direct grading (full, e-mail-only), prefill grading (full, e-mail-only).
• Stated expertise (Appendix E): main, reasoning-disabled.
• Robustness: self-report robustness probes (Figure 8), grading prompt mitigations (Figure 9).
• Verbalized-awareness hits (Figure 6): the 213 evaluation-awareness and 118 user-awareness traces flagged by the judge in the main run.
• Past-work recall (Figure 3): recall probes.

Citation information

@misc{zhong2026userawareness,
  author       = {Zhong, Ziqian and Raghunathan, Aditi and Laidlaw, Cassidy and Steinhardt, Jacob},
  title        = {User awareness in frontier models},
  year         = {2026},
  month        = {August},
  day          = {6},
  howpublished = {\url{https://transluce.org/user-awareness}}
}