Manick Bhan
Founder CEO/CTO

URL Freshness in LLM-Generated Answers: Comparing Search-Enabled and Search-Disabled Citation Patterns

Published on: December 7, 2025

Executive Summary

This study evaluates how large language models (LLMs) reference external web content by analyzing the freshness of URLs cited across two major conditions:

1. Web Search Enabled
2. Web Search Disabled

We sampled 90,000 citations with web search enabled from OpenAI, Gemini, and Perplexity, and 60,000 citations with web search disabled from OpenAI and Gemini. Publication dates were successfully extracted for 10,329 URLs in the web-search-enabled dataset and 4,352 URLs in the web-search-disabled dataset.

In the web-search-enabled condition, all platforms show a clear recency bias, with most citations referencing content published within a few hundred days of the LLM response.

In the web-search-disabled condition, OpenAI still surfaces relatively recent content, though not as fresh as with search enabled. In contrast, Gemini shifts sharply toward older material and relies more on its most relevant training-time knowledge.

Methodology

Dataset Construction

1. Web Search Enabled Sample

• Total: 90,000 URLs
• Platforms: 30,000 each from OpenAI, Gemini, and Perplexity
• Time Range: October 28, 2025 – November 6, 2025

2. Web Search Disabled Sample

• Total: 60,000 URLs
• Platforms: 30,000 each from OpenAI and Gemini
• Time Range: October 01, 2025 – October 19, 2025

In total, these datasets consist of 180,000 sampled citations across both experimental conditions.

Publication Date Extraction

We removed all URLs that were homepages, and for each remaining cited URL we performed targeted web scraping to identify publication timestamp metadata. The extraction process checked trusted <meta> publication tags, such as article:published_time, og:published_time, datePublished, and similar fields.

Using this method, publication dates were successfully extracted for 21,412 URLs across both datasets.

Freshness Measurement

By comparing the publication dates of the cited URLs with the timestamps of their corresponding LLM responses, we measured how recently each referenced source was published relative to when it was cited.

Formula:
Freshness (days) = LLM Response Timestamp – URL Publication Date

Web Search Enabled

Homepage vs Non-Homepage / Content URLs – Web Search Enabled

The chart below shows the proportion of homepage vs non-homepage URLs cited by each LLM when web search is enabled.

From the chart, we see that when web search is enabled, Gemini (92.95%) and OpenAI (88.72%) overwhelmingly cite non-homepage URLs, indicating that these models tend to surface specific content pages rather than top-level domain homepages.

Domain Overlap Across LLM Platforms (Web Search Enabled)

To measure how similarly the models retrieve external sources, we computed domain-level overlap per query across pairs of LLMs.

This process was repeated for every query, producing three overlap distributions: OpenAI & Perplexity, Gemini & Perplexity, and OpenAI & Gemini.

Distribution of Extracted Publication Dates by Platform – Web Search Enabled

Out of the 30,000 URL citations analyzed for each platform, the chart below shows the number of non-homepage URLs from which a publication date could be extracted.

Distribution of Content Age Across All LLMs – Web Search Enabled

Overall distribution of days between publication and citation for all LLMs combined within the last 10 years.

Distribution of Content Age per LLM Platform – Web Search Enabled

Semantic Relationship Between Queries and Cited URL Content

To measure how relevant each citation was to the user’s query, we computed the semantic similarity between:

• the query text, and
• the scraped content from each cited URL.

Below is the average semantic relevance score for each platform, summarizing overall citation quality.

Web Search Disabled

Homepage vs Non-Homepage / Content URLs – Web Search Disabled

During this analysis, we observed that Gemini with web search disabled rarely returns URLs that point to specific webpages. Instead, it almost always returns general domain homepages.

Distribution of Extracted Publication Dates by Platform – Web Search Disabled

Out of the 30,000 Gemini samples in the web-search-disabled dataset, only 5,710 were non-homepage URLs. From these, our strict publication-date extraction logic was able to retrieve valid dates for just 132 URLs, since we only accept trusted metadata fields and avoid capturing unrelated or ambiguous dates.

Conclusion

Large language models demonstrate distinct patterns in how they cite web content, and the presence or absence of web search plays a defining role in the freshness and quality of their citations.