DIGITIZING AS A PRESERVATION STRATEGY
Digitizing as a preservation strategy has taken many forms since the early 1990’s when scanning become affordable. Preservation of traditional materials was until then, roughly described, conservation activities and storage.
A major problem of the time, which we are still stuck with, was brittle paper. In Sweden a research and development-project on paper conservation started in 1987. The aim of the project was to find which mass deacidification method to use to stop the deterioration of modern acidic paper. This project was abandoned - and I literally mean abandoned - by its stakeholders as soon as they got wind of digitization. The well run, and internationally renowned project was not allowed to be finished. Digitization was seemingly the solution to all their problems. Digitization was the preservation strategy of the future.
When scanners started to be economically available, even for smaller cultural institutes it was not uncommon to hear that every librarian should have a scanner by their computer and scan books and documents whenever time was given. Behind that idea was mainly the aspect of dissemination.
Institions struggling with the task of preserving photographic collections envisaged scanning as the solution to end all problems. Even at that time when storage capacity was a nano-fraction of what we are used to today, the prospect of scanning every photograph in general and then just get rid of it, or at the best stow it away, was almost to good to be true. And of course it wasn’t.
A few years later the same people emphasized the need for skilled scanning personnel and today the major problem for museum curators and their colleagues at archives and libraries, dealing with photographic collections, is how to preserve the digital image files.
Digitization as a preservation strategy should mainly be looked upon as dissemination technology. Actually the only things that must be digitized to be preserved are analogue audio and video records, since they continuously decay both in the carrier of the information layer and the layer itself. This category of records needs special strategies. The delicate task of digitizing the information from sometimes very fragile carriers leads inevitably towards a continuous economical commitment. Traditional materials give us more options for preservation strategies.
The optimal preservation strategy of any item is probably to lock it away in a nitrogen-filled chamber or deep freeze it, and the optimal access is access without any restriction. Neither would work since both activities are dependant on the success of the other.
Digitization gives us the possibility to improve both aspects. Digitizing an item will improve access to the information, making it possible to restrict access to original, directing most researchers towards using the substitute. This means less strain on originals even though some researchers would still want to study the original.
If one starts to digitize and make information available on the web it is also a possibility that there will be an increase in demand for the originals. Thus it is important to include planning an access strategy as well as a security strategy along with the digitization strategy.
Reasons for digitizing In 2007 an analysis was made of the reasons for digitisation in the Nordic countries Denmark, Finland, Norway and Sweden. It was concluded that the main reason for digitizing was to improve access. Protecting original documents from wear was the second reason but was mainly stressed by the national archives in respective country. In all
Nordic countries digitization has been focused on material that when scanned can be accessed on the web in its entirety. Since genealogical research is very popular mostly genealogical records have been digitized, with the exception of Finland, which has focused on the needs of scientific research and education.
The decision process on what to digitize differs from each country, ranging from very centralised decisions to locally made.
Genealogical records have been digitized since around 2000 at the National Archives center for digitization, Media Conversion Center which started in 1991 as a microfilm service. Scanning was introduced later and 2003 became the only activity. Now about 80 persons are working two-shift with scanning only. Since then over 33 million scans have been produced for the sole purpose of web-access of genealogical records.
In 1996 the National Land Survey of Sweden started to scan all their maps at MKC. This totals so far 5,6 million records and maps, of which 1,6 millions are maps in A1-size or bigger.
All the National Land Survey-material will be stored at the National Archives and the seven regional archives. The digitized copies are the
ones, which will be used on a daily basis and the originals preserved for research.
This is a huge amount of digital information.
To make things even worse, 18 months ago the previous government quite suddenly funded extensive digitization within the cultural sector. The sum was 50 million Euros, almost 80 million dollars to be spent during two years. This now has resulted in even more large quantities of image files.
To have a reliable long term storage strategy for digital information the Swedish National Archives invested four years ago in an HSM-system (hierarchical storage management system), based around a tape cassette robot. This system was up and running in the beginning of 2004. It is as system with high reliability of securing data and prepared for migrating data to the next generation of hardware and software.
When the process of evaluating and deciding for the next step to take last year it was found that all previous
predictions on how much digital information would grow failed. Instead of expected 200 Tb only about 120 Tb had yet been collected one year ago. Of this only one 80th or 1,5 TB was born digital. But it was noted that when the deliveries of the above mentioned digitized material along with even more genealogical material from the Mormons in United States, the expected amount of image information would end up in the region of 2,3 Pb.
Digitally born records do not appear to be a major problem. The problems come when we start to digitize.
An aggressive development of prices of long term storage systems has put the National Archives in a position where funding won’t be sufficient. The strategy of extensive scanning slowly but inevitably has become a problem.
Thus it has been decided that in the future only born digital records will be stored within the HSM-system and in-house scanned images will be stored on tapes stored on shelves. A check up strategy is being planned but this means we’ve put our self in a position where we deliberately risk loss of data, since the shelving-strategy doesn’t provide fully reliable
long-term storage. This accepted risk is based on the possibility for us to scan again if data is lost but scanning cost a lot. To scan material to fill an LTO 4 digital tape of 800 Gb cost us 30 000,- USD.
The process to create the digital files is expensive and the invested funding and labour may turn out to be wasted if planning for the future is ignored and no structural funding for maintenance is secured. Then the digital information may be lost forever in a black hole.
The kinetics of a typical digitization project has a lot in common with the life cycle of a star.
Like an interstellar medium good ideas are always around. In the initial stages a lot of energy is drawn together and transferred into next phase, of planning and securing project funding. Then the project is underway, things begin to take shape, and digitization is started.
When all information is digitized and organized and made accessibel it grows into a Super Giant, bright and strong and visible to the whole world. Then new projects begin to be developed, other interests get in the way, and the project begins to be neglected and starts to collapse. The producers eventually decide to terminate it - and that’s the end of
another good project.
Not everything necessarily dies with the project: the most important information may well survive and some of the old, good ideas will meet new good ideas to form new projects.
A life cycle with a relatively short life expectancy may be perfectly acceptable for minor, less costly, projects that are of interest only for a limited period of time but for larger projects it may be too costly not to plan for long-term storage. In such cases the choice between starting the project or not in fact should depend on the willingness to plan for the future. A project with long-term survival ambitions entails a long-term financial commitment.
Luckily there seem to be possibilities for a lesser-cost strategy based on open source software. The Sub-committee on Technology is currently making plans for a realization of the ideas in Kevin Bradley’s report Towards an Open Source Repository and Preservation System, published by MOWP last year. I assume you will hear more about this later this morning.
As an alternative to digital long term storage there is actually the possibility of analogue long term storage of digital information.
Properly processed and stored microfilm may well last many centuries. The technology of computer output microfilm - COM, has now been improved to a level when it can be used for high resolution image files even in colour. Plotting digital information on film was initially developed for storing 1-bit images and vector drawings, later also for simulated grey scale images. The film can be plotted with meta-data, which will be used to search each image when the film has been scanned and the information is within the digital domain again. The COM-technology is very useful for long term storage of flat image files, which may be very large in bytes, and not needed to be digitally available all the time. But it is also possible to store any digital information on film as twodimensional bar code, even though this is not yet fully developed.
The Swedish National Archives did an assessment of the state of the art of COM last year and we found it stood up to our demands on traditional microfilm. Thus we will invest in a COM station, consisting of plotter, developing machine and scanner. Apart from storing scanned images of records we will change from producing microfilm the traditional way with cameras to use scanners and scan to film together with meta-data.
I in my opinion title Digitization as a preservation strategy should rather be re-phrased to Digitization as a tool in preservation strategy. The case with new technology, as exemplified in the beginning of this presentation, is we tend to embrace it too quickly. I would like to say history shows us a lot of examples where parts of new and old technologies merge, combining the best elements of the two, and sometimes we may even experience synergetic effects from this process. Digitization is a very helpful tool among others in preservation.
Jonas Palm February 2008