Copyright (c) 2001-2003 Leon Bottou, Yann Le Cun, Patrick Haffner, Copyright (c) 2001 AT&T Corp., and Lizardtech, Inc. This is free documentation; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. The GNU General Public License's references to "object code" and "executables" are to be interpreted as the output of any document formatti...
NAMEDjVu - DjVu and DjVuLibre.
Although the Internet has given us a worldwide infrastructure on which to build the universal library, much of the world knowledge, history, and literature is still trapped on paper in the basements of the world's traditional libraries. Many libraries and content owners are in the process of digitizing their collections. While many such efforts involve the painstaking process of converting paper documents to computer-friendly form, such as
Despite the quickly improving speed of network connections and computers, the number of scanned document images accessible on the Web today is relatively small. There are several reasons for this.
The first reason is the relatively high cost of scanning anything else but unbound sheets in black and white. This problem is slowly going away with the appearance of fast and low-cost color scanners with sheet feeders.
The second reason is that long-established image compression standards and file formats have proved inadequate for distributing scanned documents at high resolution, particularly color documents. Not only are the file sizes and download times impractical, the decoding and rendering times are also prohibitive. A typical magazine page scanned in color at 100 dpi in
The third reason is that digital documents are more than just a collection of individual page images. Pages in a scanned documents have a natural serial order. Special provision must be made to ensure that flipping pages be instantaneous and effortless so as to maintain a good user experience. Even more important, most existing document formats force users to download the entire document first before displaying a chosen page. However, users often want to jump to individual pages of the document without waiting for the entire document to download. Efficient browsing requires efficient random page access, fast sequential page flipping, and quick rendering. This can be achieved with a combination of advanced compression, pre-fetching, pre-decoding, caching, and progressive rendering. DjVu decomposes each page into multiple components (text, backgrounds, images, libraries of common shapes...) that may be shared by several pages and downloaded on demand. All these requirements call for a very sophisticated but parsimonious control mechanism to handle on-demand downloading, pre-fetching, decoding, caching, and progressive rendering of the page images. What is being considered here is not just a document image compression technique, but a whole platform for document delivery.
DjVu is an image compression technique, a document format, and a software
platform for delivering documents images over the Internet that fulfills the
DJVU IMAGE COMPRESSION
The DjVu image compression is based on three technologies:
DjVuPhotoDjVuPhoto, also known as
DjVuBitonalDjVuBitonal, also known as
DjVuDocumentDjVuDocument is a compression technique specifically designed for color digital documents images containing both pictures and text, such as a page of a magazine. DjVuDocument represents images into separately compressed layers. The foreground layer is usually compressed with DjVu Bitonal and contains the text and drawings. The background layer is usually compressed with DjVuPhoto and contains the background texture and the pictures at lower resolution.
DJVU DOCUMENT DELIVERY PLATFORM
The DjVu technology is designed from the ground up to support the efficient delivery of digital documents over the Internet. It provides various ways to deal with multi-page documents, and various ways to enrich the content with hyper-links, meta-data, searchable text, etc.
MIME typesThe DjVu format has an official MIME type of image/vnd.djvu, which is the preferred content-type to be given by http servers for DjVu files. Unofficial mime types used historically are image/x.djvu and image/x-djvu, which may still be encountered. Ideally, clients should be configured to handle all three. (For web server configuration help, see www.djvuzone.org/support/tutorial/chapter-authoring1.html.)
Bundled multi-page documentsBundled multi-page DjVu document uses a single file to represent the entire document. This single file contains all the pages as well as ancillary information (e.g. the page directory, data shared by several pages, thumbnails, etc.). Using a single file format is very convenient for storing documents or for sending email attachments.
When you type the
Indirect multi-page documentsIndirect multi-page DjVu documents solve both problems. An indirect multi-page DjVu document is composed of several files. The main file is named the index file. You can browse a document using the
AnnotationsEvery DjVu image optionally includes so-called annotation chunks. The annotation chunk is often used to define hyper-links to other document pages or to arbitrary web pages. Annotation chunks can also be used for other purposes such as setting the initial viewing mode of a page, defining highlighted zones, or storing arbitrary meta-data about the page or the document.
Hidden textEvery DjVu image optionally includes a hidden text layer that associated graphical features with the corresponding text. The hidden text layer is usually generated by running an Optical Character Recognition software. This textual information provides for indexing DjVu documents and copying/pasting text from DjVu page images.
ThumbnailsDjVu documents sometimes contain pre-computed page thumbnails.
OutlineDjVu documents sometimes contain a navigation chunk containing an outline, that is, a hierarchical table of contents with pointers to the corresponding document pages.
DJVUZONE AND DJVULIBRE
The DjVu technology was initially created by a few researchers in AT&T Labs between 1995 and 1999. Lizardtech, Inc. ( www.lizardtech.com ) then obtained a commercial license from AT&T and continued the development. They have now a variety of solutions for producing and distributing documents using the DjVu technology.
The DjVuZone web site ( www.djvuzone.org ) is managed by the few AT&T Labs researchers who created the DjVu technology in the first place. We promote the DjVu technology by providing an independent source of information about DjVu.
Understanding how little room there is for a proprietary document format, Lizardtech released the DjVu Reference Library under the
These two releases form the basis of our initial DjVuLibre software. We modified the build system to comply with the expectations of the open source community. Various bugs and portability issues have been fixed. We also tried to make it simpler to use and install, while preserving the essential structure of the Lizardtech releases.
The DjVuLibre software contains the following components:
- A general purpose compression command line program. Many internal DjVu data structures are compressed using this technique.
- A DjVuPhoto command line encoder. This state-of-the-art wavelet compressor produces DjVuPhoto images from PPM or JPEG images.
- A DjVuBitonal command line encoder. This soft-pattern-matching compressor produces DjVuBitonal images from PBM images. It can encode images without loss, or introduce small changes in order to improve the compression ratio. The lossless encoding mode is competitive with that of the Lizardtech commercial encoders.
- A DjVuDocument command line encoder for images with few colors. This encoder is well suited to compressing images with a small number of distinct colors (e.g. screen-shots). The dominant color is encoded by the background layer. The other colors are encoded by the foreground layer.
- A DjVuDocument command line encoder for separated images. This encoder takes a file containing pre-segmented foreground and background images and produces a DjVuDocument image.
A command line decoder for DjVu images. This program produces a
PNMimage representing any segment of any page of a DjVu document at any resolution.
- A stand-alone viewer for DjVu images. This sophisticated viewer displays DjVu documents. It implements document navigation as well as fast zooming and panning.
- A web browser plugin for viewing DjVu images. This small plugin allows for viewing DjVu documents from web browsers. It internally uses djview to perform the actual work.
- A command line tool for converting DjVu documents into PostScript .
- A command line tool for manipulating bundled multi-page DjVu documents. This program is often used to collect individual pages and produce a bundled document.
- A command line tool for converting bundled documents to indirect documents and conversely.
- A powerful command line tool for manipulating multi-page documents, creating or editing annotation chunks, creating or editing hidden text layers, pre-computing thumbnail images, and more...
- A command line tool to extract the hidden text from DjVu documents.
- A command line tool for inspecting DjVu files and displaying their internal structure.
- A command line tool for dis-assembling DjVu image files.
- A command line tool for assembling DjVu image files.
CGIprogram for generating indirect multi-page DjVu documents on the fly.
- djvutoxml(1), djvuxmlparser(1)
Command line tools to edit DjVu metadata as XML files.
DJVU ENCODERS AND ANY2DJVU
DjVuLibre comes with a variety of specialized encoders, c44(1) for photographic images, cjb2(1) for bitonal images, and cpaldjvu(1) for images with few distinct colors. Although these encoders perform well in their specialized domain, they cannot handle complex tasks involving segmentation and multipage encoding.
The Lizardtech commercial products (see www.lizardtech.com/solutions/document) can perform these complex encoding tasks
Another solution is provided by the compression server at (any2djvu.djvuzone.org). This machine uses pre-lizardtech prototype encoders from AT&T Labs and performs almost as well as the commercial Lizardtech encoders. Please note that the Any2DjVu compression server comes with no guarantee, that nothing is done to ensure that your documents will remain confidential, and that there is only one computer working for the whole planet.
Numerous people have contributed to the DjVu source code during the last five years. Please submit a sourceforge bug report to update the following list.
Regis M. Chaplin,
Paul G. Howard,
Yann Le Cun,
Joseph M. Orost,