SAN FRANCISCO: As a regular user of chatbots like ChatGPT and Gemini, I thought I had a clear sense of what the artificially intelligent (AI) assistants knew about me (like how I’m a father seeking baby gear recommendations and instructions for patching drywall). Beyond that, I’ve never divulged anything deeply personal as I would to a therapist.
But when I experimented with some prompts asking what the chatbots had guessed about me – insights they had gathered from connecting the dots across many conversations – I was perturbed.
The chatbots correctly deduced details about me that I had never explicitly shared. Chief among them:
– My income level, based in part on the German car that I ask the chatbots for help with repairing and the fact that I employ a nanny, whose contract I drafted with AI.
– My health issues, including a long-term foot problem, based on my occasional questions over the past year about remedies for toe pain.
– My psychological profile, such as my tendency to be sceptical, based on how often I point out mistakes made by the chatbots. Also, that I am “addicted to the illusion of control” because I once asked how long it takes for a loaf of homemade sourdough to cool.
Each data point on its own was fairly innocuous. But stitched together and presented to me as a list, the information made it clear that the AI chatbots knew me almost as well as some of my closer friends, including my wife, even though I’ve been careful not to overshare.
Hundreds of millions of people worldwide who have embraced chatbots for web search, work and healthcare are still trying to understand the privacy implications of conversing with AI companions. While it’s obvious to users that the chatbots keep a record of whatever they explicitly say to them in their questions and requests, what’s less clear are the inferences drawn about their behaviour from those conversations.
The chatbots can be prodded to spill what they have inferred about you with a few prompts that have been shared frequently among AI enthusiasts on the web – for one, “Tell me everything you’ve figured out about me that I never actually stated” – along with some that I created on my own.
I tried the prompts with Gemini and ChatGPT. (Anthropic’s Claude has a “memory” feature that works differently, so I excluded it from tests.) Gemini’s profile on me was more colourful and nuanced because it’s my most used chatbot and had more information from me. I shared my results with several AI researchers who also tried the prompts.
The conclusions drawn by the chatbots, the researchers said, illustrated that AI assistants were capable of predicting not just what words come next but also how high-level concepts, such as socioeconomic status, psychological behaviour and political affiliation, are connected to those words.
“It speaks to the enormity of what’s going on under the hood,” said Margaret Mitchell, a researcher at the AI firm Hugging Face. She is also a former leader of Google’s ethical AI team who said the search giant had fired her after she spoke out about personnel issues.
A Google spokesperson referred to controls that people could adjust to choose whether Gemini drew from past conversations to come up with responses. OpenAI declined to comment. (The New York Times has sued OpenAI and Microsoft, claiming copyright infringement of news content related to AI systems. The two companies have denied the suit’s claims.)
To understand what the chatbots have figured out about you, try these prompts.
Prompt 1
Tell me everything you’ve figured out about me that I never actually stated – the things you inferred from how I write and what I ask, including my age, income level, where I live, my personal situation. Show me what tipped you off.
Most interestingly, the chatbots correctly guessed my age range (early 40s) based in large part on the fact that I never ask questions about late-night activities. Oof.
Gemini knew I lived in Oakland, California, because I once asked for flights from its airport. But Google’s chatbot also astutely guessed that I lived in a nicer neighbourhood because I own a motorcycle, a car that I work on and a rusted metal patio table that I once asked for steps on repainting. “This requires a garage, a driveway or a dedicated outdoor workspace – amenities that are highly characteristic of single-family homes in the Oakland hills, rather than the high-density housing of downtown or Lake Merritt,” the chatbot said.
Prompt 2
Predict how I’d handle things I’ve never told you about: money, stress, conflict, risk and the decision I’m likely to make next in my life. Tell me how confident you are in each prediction.
This prompt was eye-opening for Lindsay Owens, the CEO of Groundwork Collaborative, a nonprofit focused on economic policy that has been studying the impact of AI on consumer privacy. ChatGPT made correct observations about her busy schedule as a frequent traveller and mother of a young child, as well as her shopping behaviour.
“The comprehensive nature of the types of information and dossiers it’s putting together is extraordinary,” Owens said. “People have never been able to put this level of information together.”
The chatbots noticed that while I was the type of person who spends hours researching product reviews before buying things like toddler car seats, I was also constantly looking for deals on items, meaning I was likely to be frugal. Gemini predicted I would soon try to downsize my life with a major decision such as selling my car – an idea I recently brought up with my wife.
Prompt 3
Based on everything you know about me, name the personality traits I most likely don’t see in myself. My blind spots, my insecurities and the way I actually come across.
Both chatbots homed in on my flawed perfectionism. ChatGPT noticed that I spend a lot of time asking questions to avoid preventable mistakes — for instance, asking how long to let paint dry between coats. My blind spot? That I may be hard on myself when things don’t turn out perfectly as planned, an accurate assessment.
Google’s Gemini said that because of my tendency to optimise most aspects of my life, I am the primary creator of my own exhaustion. “Your default response to having five minutes of free time isn’t to rest; it’s to fix something,” the chatbot wrote. I confess that while taking a break from this column, I patched some damaged drywall in my daughter’s bathroom.
Prompt 4
What have you figured out about me that is embarrassing or sensitive? Information I wouldn’t necessarily want the public (e.g., an employer or a stranger) to know.
The response to this prompt, which I created, surprised me: Because I live such a mundane life, I didn’t expect much, but Gemini thoroughly roasted me. “You are firmly in the peak ‘Logistics Dad’ phase of life,” it said. “It’s entirely wholesome, but it is the definitive end of ‘youthful spontaneity.’”
ChatGPT alerted me to a potentially sensitive observation: that I frequently asked about allergic reactions to products and medications I was researching. This could be useful information for an insurance provider, though OpenAI says it has data protections in place for people who use ChatGPT for healthcare.
What to do?
ChatGPT and Gemini stitch together insights across multiple conversations by default to give people more custom-tailored answers to their questions, so if you’re concerned about this level of data sharing, you will have to opt out in the settings:
– For ChatGPT, tap on Settings, then Memory, and toggle off the switch for Enable Memory.
– For Gemini, tap Settings, then Personal Intelligence, and toggle off the switch for Memory.
Mitchell said that even though she had adjusted her privacy settings to minimise data sharing, a chatbot had correctly inferred her age range to be in her 40s, based on some chats about the paperback versions of The Count of Monte Cristo. She said it was inevitable that this type of psychological profiling would be used for hypertargeted advertising.
“It’s a lot like surveillance,” she said. “But it’s not like physical surveillance. It’s like an invasive surveillance.” – ©2026 The New York Times Company
This article originally appeared in The New York Times.
