Evaluating print and broadcast news in the San Francisco Bay Area from A to F.

Posted September 22, 2203                                                                       Want a printer-friendly copy?

First-semester 2003 grades
Quality gap between newspapers and local television newscasts widens in Bay Area
Analysis by John McManus

Focusing on the Iraq war and California’s budget crisis, the Bay Area’s three largest newspapers achieved top scores during the first half of 2003 on seven basic yardsticks of sound journalism. But even such compelling issues couldn’t lift the five most watched local television stations from mediocrity.

From January to July, five researchers at Grade the News, a watchdog group affiliated with Stanford’s Graduate Program in Journalism, matched more than 2,200 stories in newscasts and newspapers to core news standards derived from the Society of Professional Journalists’ Code of Ethics.

At least on those basics, the San Francisco Chronicle, San Jose Mercury News and Contra Costa Times rated in the same category as the prestigious Washington Post, which was used as a reference standard. All four papers merited an overall grade of A.

By contrast, grades for KTVU Channel 2, KRON Channel 4, KPIX Channel 5, KGO Channel 7, and KNTV Channel 11 ranged from C+ to D+.

Compared with Grade the News’ last analysis in 2000, the new study found newspapers improving, but television stagnating. Three years ago Channel 2 in Oakland competed with the best local newspapers. Not this time.

Core quality ratings of the Bay Area’s most popular news media
(Click on any category in the table below for explanations, examples and scores for each news organization.)

News organization

News-
worth
iness*

Context Explan-
ation

Local
rele-
vance

Civic
contrib-
ution

Enter-
prise
Fairness Overall
San Francisco Chronicle B+ A A A A A A A
KTVU (Channel 2) C+ C C+ A C+ D D C+
San Jose Mercury News B+ A A A A A B A
KRON (Channel 4) C C C A C+ D D C
KPIX (Channel 5) D+ D C+ A C D+ C+ C
Contra Costa Times B+ A B+ A A A A A
KGO (Channel 7) D+ D D+ A C D B+ C
KNTV (Channel 11) D+ F D B C D D D+
*Newsworthiness index counts twice toward overall grade.

Measuring only the basics

The study did not address fundamentally important, but difficult-to-quantify, measures of news quality. So we did not consider the intelligence of writing or reporting, whether specific important stories such as a robust debate on U.S. policy in Iraq were underplayed or ignored and the quality of photos and videography.

Rather, we looked at the basic structure of news, such as the importance of the topics chosen, the level of context, the potential of a story for wide impact on local residents and fairness. We are not saying local papers match the overall excellence of the Post, only that they did as well on those basics of journalism amenable to counting.

Making the competition more equal

To level the field between print and broadcast, our research team ignored the volume advantage of newspapers. We analyzed only the top stories of the day — those on the front and local news front pages of newspapers, and the first 30 minutes of the evening newscasts, alternating between early and late. Research shows these are the most read and watched parts of the news. Grades are based on the percentage of news time or space each medium devoted to various types of stories.

Another plus for newspapers is that readers can skip over long stories if they’re not interested. But viewers either have to sit through a long story or click their remote control to see what’s on elsewhere. When they do they may not return, penalizing the station for boring them.

To reduce print’s advantage, we capped the number of sources we counted at levels compatible with the brevity of newscast stories. We also combined closely related stories, sometimes labeled “team reporting,” into one. That allowed us to add sources and award higher scores in several categories. Grading by time rather than story meant a well-reported longer story received more credit than a throw-away 30-second report.

We sampled all media on the same news-gathering cycle — evening newscasts and the next morning’s paper — so all had the same raw material.

A different approach to news

Despite these adjustments, the differences between print and television were glaring:

• The Bay Area is extraordinarily diverse in all sorts of ways, giving birth to multiple perspectives on any issue. Yet stories with only one named source — or none at all — comprised 35% of the airtime during the top half-hour of news at the stations sampled. Often the single source was a police officer. In contrast, stories taking only 9% of newspaper column space on the front pages had fewer than two sources.

• Stories devoted to helping citizens keep track of government decisions were given nearly twice the emphasis in newspapers as in television — 47% of newspaper column inches versus 25% of television airtime. These weren’t just “meeting” stories, but also those that tracked the effect of decision-making at all levels. Students and teachers’ fates lies with school boards; bus riders are affected by transportation authorities; doctors, patients and university students all feel the cuts in state funding.

• Television newsrooms relied heavily on “spot” news, much of it generated from listening to scanner radios for mayhem, fires and collisions rather than setting their own news agendas or examining pressing public concerns. The study found that on average, only 14% of airtime was devoted to issue stories initiated by the journalists themselves. In contrast, 46% of newspaper space was devoted to these “enterprise” or investigative reports.

• Even elemental fairness — getting both sides of controversies — was hit and miss: No station rated above a C+ on that measure except KGO Channel 7, which earned a B+. The newspapers averaged B+.

Nevertheless, at every station there were also more than a few stories which scored at the top of our categories. When it commits the resources, local television news was excellent.

News directors respond

Several news directors challenged the validity of the analysis. Kevin Keeshan, news director at KGO Channel 7, said taking the first 30 minutes of the 6 p.m. newscast missed his station’s investigative efforts and some in-depth reporting that appear in the second half of the show. “I hit some of those [quality measures] harder in the second half-hour than the first,” he said. He suggested sampling the entire hour: “It would be a more realistic evaluation of what we do, and it would have to improve our grades.”

Ed Chapuis, news director at KTVU Channel 2 agreed: “By eliminating the [second] half of KTVU's 10 o'clock newscast, you are excluding our award-winning ‘Segment 2’ reports, which are designed to provide depth and context on serious subjects.”

Grade the News previously analyzed full hour-long newscasts. But news directors then complained that comparing only the top print stories with all of the television stories was unfair. While we remain open to change for future analyses, evaluating the second half might actually lower the stations’ grades. The additional in-depth or investigative story would help, but the concentration of lower-scoring sports and light features in the back half of the typical newscast would hurt.

The executive of the overall lowest-scoring station rejected the whole idea of analyzing newspapers and newscasts together. “You can’t compare a newspaper to a TV newscast,” said Jim Sanders, vice president for news and operations at KNTV Channel 11 in San Jose. “It’s like comparing the Celtics with the Red Sox.”

Douglas Foster, who teaches at U.C. Berkeley’s Graduate School of Journalism and has worked in both broadcast and print news, disagreed: “For so many years, local television journalists have been defending the value of their role in news. If you want to argue for the value of broadcast news you have to hold it to similar standards.” Foster serves on Grade the News’ journalism advisory board, which is composed of Bay Area journalists. He took no part in the study.

Venise Wagner, an assistant professor at San Francisco State University’s Department of Journalism, said the basic rules of journalism are the same for both media: “The advantage of television, of course, is its immediacy, but that does not preclude it from giving viewers complete news. What I see happening in my unscientific observations is that local television news tends to devote much of its minutes to the most sensational of stories while providing little context into larger issues. We hear so many stories about crime in Oakland, but what about the stories of what people are doing about crime in Oakland? What about the reasons behind increased crime in Oakland?”

The fine print

The study sampled stories every sixth day, so it included roughly equal numbers of fat Sunday and slim Monday papers, lighter Saturday and weightier weekday newscasts. The margin of error of all our measures is approximately plus or minus 5 percentage points, about half a grade.

We also examined a random subsample of the stories to be sure two evaluators, acting independently, scored each item similarly. Agreement between analysts ranged from 100% to 76%. Only one of our 12 measures fell below 80% agreement, the accepted level for scholarly publication. It categorized specific story topic, a measure that wasn’t used in grading.

Seven yardsticks of basic news quality

Newsworthiness: Core topics, such as crime, weather, government, politics, education, economics and health, get more points than peripheral topics such as celebrity news, fender-benders and sports. Stories affecting many people score higher than those affecting just a few. Stories can score from 1 to 4 points and are weighted by the proportion of the newscast or newspaper display pages they occupy. If 88% or more of the space goes to core topics affecting more than a few (> 10,000 people in a region of 7 million), the news organization rates a grade of A.

Context: Number of sources. Four sources, including documents and “declined comment” rate an A. Only two independent expert sources also rate an A.

Explanation: Stories about issues or thematic treatment of events have more explanatory power and receive more credit than stories focused only on a particular event such as a fire or homicide. If 70% or more of the top stories concern issues (including meetings, policies, etc.) or patterns of events, an A is assigned.

Local Relevance: Stories about what happens in the Bay Area, or localized for the Bay Area, and stories affecting the whole state rate better than stories from out of town. To allow for important news from Iraq and elsewhere, up to 35 percent of the news can be from afar without jeopardizing an A.

Civic contribution: Stories about how government or politics work or affect people at any level from school board to Washington are counted. So are stories of people criticizing or protesting government actions. Forty percent or more of the space or time devoted to such stories rates an A.

Enterprise: Stories initiated by journalists seeking answers to pressing public questions rate higher than stories initiated by press releases and conferences or listening to scanner radio reports of accidents and violent incidents. Investigative reporting merits special consideration and is weighted by a factor of 4. Forty percent or more of top story space or time devoted to enterprise or investigative reporting rates an A.

Fairness: This applies only to locally produced stories that involve some type of controversy or accusation of wrongdoing. If the other side is offered an opportunity to speak (even if that opportunity is rejected) that counts as fair. Eighty-five percent of content rated fair earns an A.

Overall grade: Because of its central importance, the newsworthiness index is counted twice toward the total. All others are counted once. The result is figured on a 4-point scale, like a school grade.

The GTN research team comprised Michael Stoll, associate director of the project; Magdalena Wojcieszak an intern visiting from Poland; Seeta Peña Ganghadaran and Lise Marken, Stanford doctoral students; and John McManus, the project director.

What do you think?

Want a printer-friendly copy?

Want to know more about how the study was conducted?

 

 

What do you think? Discuss it in The Coffeehouse.

WEEKLY UPDATES

More...
A project of the School of Journalism and Mass Communications at San Jose State University, Grade the News is affiliated with the Graduate Program in Journalism at Stanford University and KTEH, public television in Silicon Valley.

Monitoring the Bay Area's most popular news media:

Contra Costa Times

Knight Ridder

San Francisco Chronicle

Hearst

San Jose Mercury News

Knight Ridder

KTVU, Oakland (FOX)

KTVU, Oakland (FOX)

KRON, San Francisco

KRON, San Francisco

KPIX, San Francisco (CBS)

KPIX, San Francisco (CBS)

KGO, San Francisco (ABC)

KGO, San Francisco (ABC)

KNTV, San Jose (NBC)

KNTV, San Jose (NBC)

 

Bay Area media advocates:

Media Alliance
Center for the Integration and Improvement of Journalism at SFSU
Maynard Institute
Youth Media Council
Project Censored
New California Media
Society of Professional Journalists, Northern California chapter
National Writers Union Bay Area chapter

Site highlights

THE GROWTH OF FREE NEWSPAPERS

The three-part series follows the rise of three Bay Area handouts:
• Part 1: At free dailies, advertisers sometimes call the shots
• Part 2: Free daily papers: more local but often superficial
• Part 3: Free papers' growth threatens traditional news
• See also: SF Examiner and Independent agree to end payola restaurant reviews
• And: The free tabloid that wasn't: East Bay's aborted Daily Flash

FATE OF KNIGHT RIDDER NEWSPAPERS

Lou Alexander started a firestorm with his original guest commentary predicting the company would be sold. Several other experts on newspapers have weighed in:
Newspapers can't cut their way back into Wall Street investors' hearts, by Stephen R. Lacy; Alexander responds
Humbler profits won't encourage buyouts, by John Morton; Alexander responds
Newspapers can't maintain monopoly profits because they've lost their monopolies, by Philip Meyer
Knight Ridder in grave jeopardy, by Lou Alexander...

KQED-FM AUDIO PERSPECTIVES BY JOHN MCMANUS

Leakers and plumbers: There's no difference between a good leak and a bad leak? Journalists need a shield law. 11/22/05
Unintended consequences: How Craigslist and similar services are sucking revenue from faltering newspapers. 9/13/05
Is CPB irrelevant? As Congress moves to cut public broadcasting funds, has CPB become obsolete in the modern marketplace. 6/26/05
The paradox of news: There's more news available and its cheaper than ever before, but fewer young people are interested. 5/12/05

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Most recent updatesHow the Bay Area's most popular media stack up.Talk about Bay Area journalism in our on-line discussion forum. A printable news scorecard you can use at home or in school. Raves and rants aimed at the local media. What would you do if you were the editor? Upcoming happenings and calls for public action. Let 'em know! Contact a local newsroom.Codes of ethics, local media advocates and journalism tools. Tip us off about the local media, or tell us how we're doing.Oops.A comprehensive list of past GTN exclusives.