The CIA spies the future of journalism

Who would have thought that the Central Intelligence Agency has something to teach the news business?

A bit of serendipitous surfing landed me at Library Clips, the Australian knowledge-management blog of John Tropea, and in particular at his recent post "How do wikis and blogs fit together"—a topic which is not much different from the question of how wikis can be integrated into a news operation, a subject I've touched upon before, and which Matt Thompson is exploring at Mizzou.

John quotes a 2005 paper by D. Calvin Andrus of the CIA, "The Wiki and the Blog: Toward a Complex Adaptive Intelligence Community" (free downloadable PDF via link). The paper proposed replacing the agency's cumbersome, hierarchical, inflexible and ineffective processing of raw data into accepted conclusions of fact with a self-organizing "complex adaptive system" consisting of:

• A repository of raw data ("like unrefined ore") gathered from the agency's network of sources.
• Blogs in which individuals can express and share opinions within the agency (which would be a big "paradigm shift," according to Andrus.
• A wikipedia in which material archived in the repository and discussed and interpreted in the blogs is organized into an authoritative agency knowledge base of agreed-upon facts, tools, techniques and policies.
• Search tools: "Part of the agility required in today's high speed security environment is to be able to quickly find information."
• Feedback tools, with which agency users can discover quick answers to questions such as what parts of the wiki are changing rapidly (an indication of new information being developed), where there are knowledge gaps, what areas of the system are getting the most attention, and so on. "As important as information sharing is to the success of the solution, it is even more important to know who is sharing what information."

It turns out the agency took Andrus's advice and has implemented much of this system in the form of its Intellipedia. (See also the Wikipedia entry on this.)

Does this suggest a way for news enterprises to organize and publish their information? Indeed. Here's the model (assuming a local or regional news organization):

• A repository of raw data stored electronically—census data, election results, municipal reports, agendas, minutes, financial data, press releases, court records, real estate sales information, traffic studies, environmental impact statements, and data from reporters, stringers, and other contributors: notes, audio, video, stills. In other words, all the detritus that usually stagnates in, under and atop desks and file cabinets in newsrooms.
• "News" (the equivalent of the CIA's internal blogs): published "stories" as well as blog posts, columns, editorials, letters to the editor, reader comments, etc., which are generated based on information in the repository, or on stories and blog posts by others.
  
• A local wikipedia constantly being built and refined from the blog-equivalents, in which local issues as well as people-in-the-news, businesses, government entities, non-profits, geographic features, buildings, streets, etc. all have pages that are collectively built and curated using the knowledge gathered into the repository and refined in the blog-equivalents.
• Search: a self-explanatory feature facilitating access to all of this material
• Feedback tools: these will help editors and journalists see how material is accessed, shared and disseminated by its users.

But importantly, I would add one more key feature that's perhaps not as important in a closed system such as the CIA, but will be critical for news enterprises interested in garnering a loyal readership without gray hair:

• Social networking: As famously uttered by a college student in a focus group and repeated throughout the journalism blogosphere, "if the news is that important, it will find me." From the linked N. Y. Times story: "In the days after Mr. Obama’s speech on race last week, for example, links to the transcript and the video were the most popular items posted on Facebook. On The New York Times’s Web site, the transcript of the speech ranked consistently higher on the most e-mailed list than the articles written about the speech." As Steve Outing has suggested in a column worth a detailed read, this suggests not only that we must redefine the news organization and the news itself, but that news organizations must connect with readers and sources through networks like Facebook and Twitter, incorporating unaffiliated news feeds like Google News and Topix, pushing content out via Flickr, YouTube and Twitter, and facilitating user-generated news feeds that may incorporate everything from wire-service feeds to micro-personal news from the reader's social network connections.
  

Social networking also suggests avenues for those (like me) who are still interested in how all this gets paid for. Publishing stuff online in order to generate "eyeballs" and then selling ads to folks interested in reaching those eyeballs is pretty much yesterday's model. (Actually, it's a previous-centuries model that may have lost its final bit of momentum on 9/11—an event "important enough to find" just about everyone long before showing up anywhere in print, and one that created new imperatives for people to reach out and connect with each other.) A community-centered news enterprise that exploits the power of social networking will tend to generate "friends" rather than "eyeballs," and friends—customers with loyalty—are what advertisers are looking for.

Universalizing the copy desk

Turmoil in the generally placid waters of copyediting! Singleton lets drop that he might outsource their work to India! Weeping, wailing, gnashing of teeth! The American Copy Editors Society (ACES) responds (on front page of site, not permalinkable):

Sending copy editing overseas is a sure way to kill a paper's credibility. Maybe not immediately, and maybe not in one dramatic gesture, but every time that desk 9,000 miles away overlooks some nuance that local readers will spot, the newspaper's credibility dies a little. Oak Lawn, Oak Cliff, Oak Park, Oak Hill? What's the diff?

....And let's not even start on the tragicomic workflow disaster an overseas desk would cause. Just think of the last time you had to deal with an overseas customer service call center.

I'm sorry, but I happen to think the world’s offended copy editors are wrong for suggesting, in effect, that those dumb folks in India don’t know English well enough, and won't know their particular backyard, resulting in a ton of stupid mistakes.

The unions’ argument against Singleton (in their formal response) is plainly weak; it boils down to: There might be “errors a local person would have caught.” What else is new? There are plenty of errors already.

But beside that, if you look at the stories in any major Indian newspaper online, you won’t find scads of grammar errors, spelling errors, or stilted language. And for the most part, at the various regionalized copy desks already within MediaNews Group today, they don’t make mistakes about street names and such that are a couple of counties away, because Google has been invented (along with the telephone), to doublecheck those things. Distant copy desks can learn the local particulars pretty quickly and would obviously build themselves a reference manual as well. The California MNG papers are already doing ad production in India, apparently with no surge in errors, and the outfit Singleton probably has in mind, Mindworks, has been at this for a while very very successfully for a number of other major publications.

A better idea than all of this, however, is wiki-based journalism, which I seem to be posting about frequently (click "wikipedia" tag below this post). As part of this, stories could go online before getting excruciatingly edited, and would then go through several cycles of re-editing and annotation, Wikipedia-style, as envisioned by Japan's Information Architects. Copy editing in this model is universalized, as it is at Wikipedia.

Of course, copy editors won’t like that either, because it means that to some extent, Jane Public becomes your copy editor. But what's wrong with that? If Jane Public is already spotting mistakes (or “missed local nuances”) all over the paper today, while there are still copy editors warming American seats, why not let her just fix those mistakes, so reporters and editors can focus on chasing down the rest of the story? In fact, if enough Jacks and Janes out there can be engaged in the process of not only copy editing but researching, reporting and improving stories and news wiki topics, news content could be enormously enriched.

Valley Advocate does wiki, sort of

I picked up a copy of the Valley Advocate (that's the Pioneer, or Connecticut River, Valley in Massachusetts) the other day (something I do maybe three times a year), and noticed the front page headline: "Who's a WIKI: Help the Valley Advocate begin gathering data for the first Valley political wiki." Holy cow, I thought, they're starting a local wiki! They even express the hope that this will eventually be "a large and growing database of information about local people and issues in the news." Issues! Large and growing! A localpedia that's more than a directory of pizza joints and brew pubs!

No such luck, however. There's a two-page spread inside, with short bios of a bunch of politicians, and an introduction to the effect that they'd like readers to "fill in the gaps." So I go online and click on a pol I happen to know, Rep. John Olver. Can I edit what they've got? No. Can I add any information of my own? No. Can I do the wiki part of wikipedia? No. Well, actually, I can add a comment.

Sorry, Valley Advocate, that's not a local wiki. You want a local wiki? Use the open source software you can get from MediaWiki, or something similar, have every reporter, stringer and editor start contributing content, and open the gates to the public to add and edit. But, you were right about "local people and issues in the news"—the right way to build a localpedia is to base it around people and current issues, not restaurants and historic landmarks. (See multiple previous posts on this.)

I'll just have to keep looking for someone to kick this off right. Or do it myself.
................................................................................................................................................

Extra: Must-read of the day is "The Elite Newspaper of the Future" by Philip Meyer in AJR. He makes some excellent points about why the end-game is here and now for newspapers, and suggests they might survive in some way by focusing on the "leadership audience," a narrow subset of the broad audience they've always aimed for. Here's Meyer:

If they should peel back to some core function, newspapers would still have to worry about the Internet and its unbeatable capacity for narrowcasting. The newspapers that survive will probably do so with some kind of hybrid content: analysis, interpretation and investigative reporting in a print product that appears less than daily, combined with constant updating and reader interaction on the Web....But it won't be a worthwhile possibility unless the news-paper endgame concentrates on retaining newspapers' core of trust and responsibility. The mass audience is drifting away, and resources should be focused on the leadership audience. If existing newspapers don't do it, new competitors will enter their markets and do it for them.

Indeed, they will.

Newspaper as Wiki, by Information Architects

Oliver Reichenstein heads up an outfit in Japan, with an outpost in Switzerland, called Information Architects (iA). Matt Gerber is iA's information designer. They've done a lot of innovative branding work, and a lot of innovative thinking about the future of news, and especially about "newspapers as wikis. " Their entire "about us" reads:

Brands are interfaces. Successful brands perform and evolve in a controlled, interactive process with their audience. Performance empowers the brand.

iA plans, builds, and manages interactive brands by continuously measuring and optimizing the performance of their user interfaces.

iA’s offices in Tokyo and Zürich serve clients from Japan, Germany and Switzerland.

Follow their thinking about news and newspapers in this sequence:

From March 2007, "The Future of News: How to Survive the New Media Shift," an endorsement of the wikipedia approach to news publishing (which I've endorsed, urged and elaborated on previously):

The making of a news article must become a public process. This will greatly increase the quality and the trust in published text. The integration of social media into the writing process is a greatest opportunity for writers and readers since the invention of the press; the more intelligent the readership of a newspaper the more intelligent the product will become.

Wikipedia proves that it works. The only question is: Which newspaper will allow their readers to participate in the writing an editorial process first. Participating means: You let them influence the article from the beginning to the end—and beyond. Future newspapers will allow their readers to influence the article before and after it is published. The only way to do this is by turning the newspaper web sites into transparent public editorial tools.

Also in March 2007, iA posted "10 Newspaper Myths Deconstructed," including:

Myth 10: Newspapers need to become social networks
Fact 10: Newspapers need to become Wikis

Newspaper readers are not friends. Good readers are supposed to be critical about each other and not pat each other’s backs. There is a much more obvious technological model for newspapers.

I believe that newspapers need to make the development of a story transparent, they need to cross link and cross reference articles, they need to allow users to write articles and collectively correct each other’s mistakes and they need to become at least as accurate as the Encyclopedia Britannica. Got it? Yes, newspapers should use Wiki technology. There is a lot to improve on the front end, but the content generation and optimization model is an almost perfect fit.

Then in April 2007, hinting that they had a customer interested in the wiki notion, they posted "Washington Post Redesign as a Wiki, " to answer questions beginning with "What does a newspaper as a wiki look like?" (The customer was not the Post.) Have a look at the graphics illustrating their thoughts.

Shortly thereafter appeared "Newspaper Wiki: Schematics," a Q&A on how iA sees the "newspaper as wiki" working. It includes a graphic illustrating a "wiki-based editing process,"

Finally, in June 2008, iA unveiled Newsnetz, which "joins the forces of major Swiss newspapers and a series of local Swiss papers to become the country’s leading news domain in terms of reach, traffic, editorial size, and production." With Newsnetz, they've put into practice their development of the "newspaper as a wiki." Again, look at the post for extensive graphics showing the sites before and after, and how the design was done. The sites themselves are now live, they're Tages Anzeiger, Berner Zeitung, and BazOnline. (Keep in mind that at this point, I believe the wikification is behind the scenes in the content management system, and apparently has not yet reached the stage where readers can join the process. But perhaps a reader of German can explore those sites better than me and explain how the reader might interface with them, wiki-like.)

There's a lot more to digest at iA, including many comments from visitors and responses from iA to the comments. Well worth spending some time with.

Why not Wikify?—Updated 1

It turns out there is at least one other voice crying in the wilderness: "Wikify, wikify!" (See my recent post entitled "Why not Wikify?"

Matt Thompson, who blogs personally at Snarkmarket, is currently on a one-year fellowship at the Reynolds Journalism Institute of the University of Missouri, where he is working on a research project entitled "Wikipedia-ing the News." (via Fimoculous) From his original proposal:

To paint a broad picture: Imagine if the work of the hundreds of reporters dispatched daily to cover a city didn’t merely fade into an obscure archive, but added day after day to the work that came before it. An online news site in the era of Wikipedia would be a living archive, adaptable to suit any context, growing to encompass all aspects of life in a community. Entries would be deeply and meaningfully interlinked to other entries, elegantly situating every news event in multiple larger contexts. The “latest news” on the site could be a kind of changelog, reflecting new additions or edits in the system. The site would be a news commons atop which other narrative presentations of the news — stories, blogs, videos, games — could sit.

He said it better than I did, but a couple of my paragraphs offer seamless elaboration:

And then, dare I suggest, why not allow readers to edit the wiki as well, with the reporter who "owns" the issue keeping an eye on it to prevent abuse? It works at Wikipedia itself, why would it not work at a local newspaper wiki?

Once established, readers could add new topics, including pages on the history and amenities of the area, on elected and appointed local officials and legislators, on businesses and non-profit organizations, on schools and churches, on clubs, bands, rivers, mountains, parks and what-have-you. Every new news story might come with a series of wiki references to allow readers to explore the backgrounds of not only the issues but also the personalities, buildings and organizations involved.

See also Matt's "elevator pitch," or capsule summary of what he's talking about.

As Matt points out, this whole notion was suggested in the American Press Institute's February 2008 report, "NewspaperNext 2.0—Making the Leap Beyond 'Newspaper Companies'" (PDF, page 13). I had read their idea and should have mentioned it, but at least as described in their report, API's notion of a "localpedia" sounded more descriptive than dynamic. It did not incorporate the "living archive" aspect Matt and I have envisioned, in which the local wiki follows all ongoing news stories and the "latest news" becomes the equivalent of a "changelog," which reflects the stories most recent incremental changes.

Matt's cubicle neighbor at Reynolds is my friend Bill Densmore, who is there working on another project worthy of extended mention (soon!), the Information Valet Project. I'll be keeping tabs on both Matt's and Bill's progress with interest!

Why not Wikify?

Most news organizations, both online-only and traditional media with web sites, are keenly scanning the technological horizon for the latest trends to incorporate into their sites. As noted yesterday, Twittering is now in the news site toolkit, joining a host of features adopted earlier—including discussion boards, commenting, video and audio clips, blogs, personalized home pages, RSS feeds and site search, along with all manner of ad presentation techniques from pop-ups to contextual ads to Flash.

So, why haven't newsrooms noticed Wikipedia? And why haven't they considered what might be done in newsrooms with the freely-available, open source, MediaWiki software that drives Wikipedia and many other wiki sites?

Here's how a reporter approaches a story at a local paper. She gathers information about what's happening by going to the event, covering the meeting, and/or interviewing some sources. Then, back at the desk, she usually rummages through a bunch of files and notes, ransacks the morgue where clip files are kept, and/or searches through whatever electronic archive system the paper may possess, in order to add background to the story. This is important because most stories are part of a larger, ongoing local issue that may have years of background readers need to be continually reminded of, and even a fire or murder can benefit from some background on the building or the victim culled from the newspaper's records.

The thing is, this process must be repeated whenever there's a new development, and especially when there's a new reporter on the beat, which is frequently, these days. Similarly, readers interested in researching an issue need to delve into the paper's archives online or at the public library and potentially go through many stories to get the picture. Wouldn't it make sense to build all of the back story into a wiki on the topic, and to make it the responsibility of the reporter to update the wiki whenever something new happens? And once the wiki is created, why not make it available online, linked in the printed and online versions of the story, so a reader can get a summary of all the background the paper possesses, not just whatever the reporter considers relevant to the current story.

And then, dare I suggest, why not allow readers to edit the wiki as well, with the reporter who "owns" the issue keeping an eye on it to prevent abuse? It works at Wikipedia itself, why would it not work at a local newspaper wiki?

Once established, readers could add new topics, including pages on the history and amenities of the area, on elected and appointed local officials and legislators, on businesses and non-profit organizations, on schools and churches, on clubs, bands, rivers, mountains, parks and what-have-you. Every new news story might come with a series of wiki references to allow readers to explore the backgrounds of not only the issues but also the personalities, buildings and organizations involved.

If anyone is doing something even remotely like this, I'd like to know. I'm aware only of one stab taken toward this in Brattleboro, VT, where a "Brattleboro Community Brain Trust" in wiki format is slowly accumulating content. Very slowly. It's mostly historical stuff and it's not updated with current events. The latest news on it seems to be Brattleboro's resolution against a war with Iran, passed in March 2008, which is published without background or other context. The site's timeline for Vermont Yankee, a nuclear plant that's the hottest issue in town, ends in 2007, even though there have been almost daily developments this year in the plant's effort to be relicensed for another 20 years of operation.

Elsewhere, Wikispot is trying to foment the creation of local wikis, not in association with news media (have they tried, though?). The most successful of these appears to be in Davis, CA. You'll find a lot of information about Davis there, but the most recent news on it is dated June 23, 2008. And it features no effort to wikify issues rather than individual news events.

Keith Buchanan, a McClatchy staffer, posted a brief thought this summer about newspaper-sponsored local wikis on the McClatchy blog, but the idea seems to have landed with a very soft thud there. Wikipedia itself has a page on local wikis around the world, which demonstrates that not many of them have gained any traction, and none of them are sponsored by a news organization. The definition there of "city wiki" does not even include the idea of using the wiki to track ongoing issues and other news stories.

It's often said that newspapers are the first draft of history. (Today, Twitter is, but that was yesterday's news.) With wiki technology, newspapers can write the next draft, and continue to update and refine it, assisted by their readers. And it can help them "monetize" the Web, by getting more mileage out of the content they create, attracting more site users, generating more page views, and selling more ad exposures.