How to avoid blind spots and use Deep Research effectively

Since OpenAI released it a few days ago, I’ve been testing ChatGPT Deep research on a several topics I’m knowledgeable about, and unlike most pundits, I’m not floored.

While it’s quite capable and definitely useful for an initial overview of an unfamiliar topic, it’s not radically better than o1 Pro with careful prompting, and it should come with a BIG WARNING:

It can COMPLETELY miss certain “crucial” (as it likes to say) aspects pertaining to the topic.

For example, one of the topics was presbyopia (“old eye”, the reason nearly anyone over 40 wears glasses for reading — 1.8 billion people worldwide). I asked Deep research to analyze the safest and most advanced methods for treating presbyopia, and it completely missed the most effective method, PRESBYOND (90%+ of patients see without glasses at any distance post-op, and would do it again).

When asked specifically about PRESBYOND and why it wasn’t included, it returned correct information, including its extremely high safety profile and patient satisfaction score, “multiple peer-reviewed outcome studies”, and explained why it wasn’t originally included (essentially, technicalities pertinent to a professional more than a consumer).

ChatGPT’s explanation for its glaring omission

I can only surmise that this reflects a US bias, since PRESBYOND is not FDA-approved, despite having been widely practiced in Europe, Asia and Latin America, for over two decades. However, if someone based their decision of laser vision correction based on ChatGPT Deep research, they’d end up with an inferior choice.

Moral of the story: if you don’t really know what you’re doing before Deep research, you still won’t REALLY know what you’re doing afterwards.

What I recommend for higher stakes research:

1. Try multiple models. Each has its own strengths and biases, and sometimes these biases can be controlled via the user interface. For example, DeepSeek is aimed at reasoning, and its answer to the same prompt (cee below) completely missed all laser options. Claude, Gemini Advanced Deep Research (yes, OpenAI stole the name), and Qwen shared the same US training data bias (even though Qwen is Chinese…) and didn’t mention PRESBYOND. Out of all the models I’ve tried, only Perplexity correctly mentioned PRESBYOND on the first try without any prompt engineering or special focus. Perplexity also lets you focus on academic vs. social sources, which is a key option for avoiding academic bias (as long as you don’t fall for the pseudoscience).
2. Iteratively improve the prompt as you learn from each model, rather than pasting the same prompt in all chats. This way your prompt will gradually become more comprehensive, as will your understanding of the topic.
   - Start by pasting your prompt into the least capable model, and not a research one. You’ll soon learn blind spots and aspects you’ve missed.
   - Then, edit your prompt, and pasted it into the least capable research model (at the time of this writing, Google Gemini Advanced)
   - Finally, polish your prompt with what you’ve learned, and paste it into the most advanced research model, and go reward yourself with a treat while it crunches the data to give you a comprehensive report.
   - Why not use the same model repeatedly? To avoid bias and introduce diversity, though if DeepSeek indeed was trained on OpenAI data, and given how most LLMs have been trained on most of the Internet, good diversity may be hard to find.
3. Run several variations of the prompt:
   - Adding target audiences may help — such as “Compile a report aimed at educated consumers” (vs. “medical professionals”, which may be an inherent bias).
   - Adding specific geographic restrictions may help where applicable. Alternatively, you can add “consider a worldwide view, not limited to the US”
4. Manage expectations. This isn’t an all-knowing tool. Be aware that not even scientists who publish review papers perform an exhaustive search. See for example the “Literature Search METHOD” section in this literature review that also missed PRESBYOND after looking through 1297 articles. And it’s not that PRESBYOND doesn’t have or isn’t mentioned in studies; it’s just less/not on the radar of American ophthalmology associations and websites, which were the main sources ChatGPT started with.
5. Understand that “Deep research” doesn’t include the deep web. The largest omission impacting the most people is that of customer reviews from sites like Google Maps (important if you search for service providers for expensive procedures), travel sites (good luck searching for a “quiet room” — none of the reviews on Airbnb/Booking/etc. have been crawled), or from chats. If you’re looking for niche information discussed on Discord, Slack or similar chats, that is also pretty much lost to search engines and web crawlers. For consumer product and services research, excluding reviews will bias the results towards widely publicized sources, part of which are self-published — a bias within a bias.

Here’s the conversation with Deep research on presbyopia treatment options. Interesting to watch the confirmation bias once I mentioned PRESBYOND:

https://chatgpt.com/share/67a361f4-94a8-8004-89dc-13f88d075f16