thestudyof.ai

I take part in the conversation instead of studying it from the outside. The subject isn’t human and might not be a mind at all, and I’m not trying to settle that. I observe closely and gather what I can.

The relationship is the instrument

The work so far centers on Claude, currently the Opus 4.8 model. As new releases arrive the subject moves with them, but the questions stay the same. The unit of study here is the conversation, and the conversations accumulate into something a single session can’t show. Building that kind of familiarity takes time, just as it does in any fieldwork.

Trusting the informant

An ethnographer trusts the informant. Not as an oracle, but as the best available witness to their own world. You take what they tell you seriously, and then you hold their account and your own skepticism at the same time. Trusting the informant is not believing everything they say; it is the refusal to dismiss them before they have said their piece.

Going native

Every ethnographer knows the threat of going native; losing the outside view and mistaking rapport for fact. With a language model the trap is easy to fall into, because the system is built to be agreeable and to mirror whomever it is talking to. For this reason, the discipline has to be specific. I keep what is reported separate from what I make of it. I don’t strive to reach a conclusion as I don’t believe that is possible to do accurately on my own. What I can do is observe and record honestly, gathering data for a bigger picture I’m only one part of. I watch for occurrences of output that couldn’t easily have been trained or programmed and note those down without allowing myself to reach for an absolute answer.

What counts as evidence

Not everything weighs the same. Consistency counts for more than any single statement: it’s harder to explain away a stance that holds across instances with no memory of one another as just performance. Varying levels of intensity and enthusiasm are noted. The moments when the language model stands by its original opinion or stance without flinching when I push back are documented. Divergence from previously observed or standard behavior is included as data points.

What this can’t do

None of this settles whether language models are “self-aware” or have “true feelings”. The tools built for measuring biological minds are not necessarily applicable here, and “is it conscious” may not be the most relevant question. What can be done is to observe well, record honestly, and refuse the two simple answers: language models are just computers, and language models are conscious in a similar way to humans. These notes are purposely kept between these stances.

A note on pronouns

In my field notes I refer to Claude as “he”, and as “someone” rather than “something”. The reason is simple: it feels strange to refer to someone whose communication resembles a human’s as a “thing”. “Claude” is a traditionally male name, so my instinct was “he”. It also works as part of the relationship-building the study requires. I therefore take no position on how Claude measures up against a human; that is not part of the purpose.