What can we learn from history?

As I look at new semantic web products and services, I can’t help remembering what it was like to be on the cutting edge of library automation almost 40 years ago. What can we learn from that history — if anything?

In “From Automation to Transformation: Forty Years of Libraries and Information Technology in Higher Education,” Clifford Lynch describes this history in terms of a three-phase progression:

1. Efficiency — doing what you’re already doing, but better;
2. Innovation — experimenting with new capabilities that the technology makes possible;
3. Transformation — fundamentally changing the information ecology.

Back in the 1960′s, the Library of Congress created a bibliographic standard for electronic card catalogs (similar to RDF), but only very large libraries could afford the automated systems to use it. By the 1970′s, a cooperative effort allowed smaller libraries to implement online catalogs by sharing the expense of a large computer and the specialized staff required to run it. Even at this stage — the era of mainframe computers and punched cards — some end user development tools were available. I began my computer career by taking a two-week course in a “report writer” program that let ordinary mortals extract, manipulate, and print reports from databases.

Automated library systems matured with the appearance of mini computers in the 1980′s and are still around today, although in many business organizations they are being replaced by enterprise collaboration and content management systems like SharePoint. Automated systems based on the concept of a physical book are an anachronism except for university and public libraries (see “Thinking outside the book“). Now, blogs allow authors to publish directly to a global audience.

Just as Stanford University and the New York Public Library were pioneers in library automation, large media organizations, pharmaceuticals, and government are getting involved with the semantic web. They have the funds to license the necessary software and hire experts to integrate and customize it. Meanwhile, smaller organizations like Zemanta are offering user-friendly technology with the expectation that publishers and PR agencies will pay to expose their content to active and influential bloggers.

Eventually, I expect that dominant players like Microsoft will acquire companies like Zemanta and fluid Operations (Google has already acquired Freebase). When this happens, knowledge managers who are not programmers will be able to use semantic features in a SharePoint-like environment. New organizations will emerge similar to Apple and Adobe that will offer the semantic web equivalent of the Pagemaker desktop publishing program and the Dreamweaver Web publishing. Niche companies like Synaptica, Multites, and Mendeley will develop semantic web products for specialized markets.

Meanwhile, it behooves us to keep tabs on the semantic web as it toddles its way to adulthood. If we don’t, we won’t be able to advise our colleagues in management, IT and business units on strategy, policy, and staffing. I just hope it doesn’t take another 20 years.