Testing, One, Two Three: The NYT Weighs Which Headlines Resonate with Readers

03.10.2017 | Christie Melby

An interesting article recently shared on Times Insider, The New York Times’ behind-the-scenes insight page, revealed that the paper has been running a test on which headlines are attracting more readers. Mark Bulik, a senior editor, unveiled the test is part of an effort to increase the paper’s readership.

The experiment is a relatively simple one, yet reveals the importance of a strong headline. It involves using a tool that allows them to simultaneously present two headlines for one story on the home page. Fifty percent of visitors see one headline and the other 50 percent see another. From there the paper measures click-through rates and, if there are significant differences between the two options, they select the winner to appear on the home page for all readers.

Below are several examples of test headlines The Times has run. Can you guess which ones received the most clicks?

Test One:

  1. $2 Billion Worth of Free Media for Trump
  2. Measuring Trump’s Media Dominance

Test Two:

  1. Soul-Searching in Baltimore, a Year After Freddie Gray’s Death
  2. Baltimore After Freddie Gray: The ‘Mind-Set Has Changed’

Test Three:

  1. Is Everything Wrestling?
  2. It’s Not Just Wresting That’s Fake. It’s The World.

The Results

In test one, headline No. 1 garnered 297 percent more readers. In test two, the second headline showed a 1,677 percent increase in readership. Lastly, with the third test, headline two increased readership in tenfold.

The Takeaways

It’s important to realize that while the article is the story, the headline is how a reader determines whether or not they will invest their time in reading the article. Based on the findings from this experiment, Bulik says the headlines with the most success are clear, powerful, and written in a conversational tone. As seen in the third test, incorporating something slightly controversial and that’s top-of-mind with readers, like the word “fake,” has proven effective.

While Bulik doesn’t state this in his article, based on our experience in the industry, people like numbers, which is why I would venture to guess headline one in test one saw more success. It helps people quantify the gravity of the topic. Anyone would look at $2 billion and think, that is a lot of money. That value will likely pique their curiosity and create a desire to understand how the journalist arrived at that number. However, when referring to measuring media dominance, it unclear what that will look like. Is it measuring the number of news article about Trump? Does it measure the time people spend reading article about him? Are the measurements at all significant? It leaves the reader with a lot of unanswered questions about the significance of the topic, rather than the topic itself.

This is an interesting approach, enabled by the age of digital media, to refining a critical element of the writing process. Bulik’s parting piece of advice, when developing a headline, two heads are better than one.

For more tips for creating a stellar headline, check out these articles:



Filed under: PR trends, Strategy

Leave a Reply

Communication Strategies that Move Product. Sell Services. Grow Brands.

We are experts at harnessing PR to achieve business results. We are quick-study strategists who know how to get the job done. Learn more »


We are big believers in the power of strategic public relations to meet business goals. You could even say we wrote the book on it. Check it out »