unsq (1)

Leading comments

$Id: sq.1,v 1.12 2005/04/14 14:38:23 geoff Exp $

Copyright 1992, 1993, 1999, 2001, Geoff Kuenning, Claremont, CA
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:

1. Redistributions of source code must retain the above copyright
   notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
   notice, this l...

(The comments found at the beginning of the groff file "man1/unsq.1".)

NAME

sq, unsq - squeeze or unsqueeze a sorted word list

SYNOPSIS

sq < infile > outfile

unsq < infile > outfile

DESCRIPTION

sq compresses a sorted list of words (a dictionary). For example:

sort -u /usr/share/dict/words | sq | gzip -9 > words.sq.gz

will compress dict by about a factor of 5.

unsq uncompress the output of sq. For example:

gunzip < words.sq.gz | unsq | sort -f -o words

will uncompress a dictionary compressed with sq. The squeezing is achieved by eliminating common prefixes, and replacing them with a single character which encodes the number of characters shared with the preceding word. The prefix size is encoded as a single printable character: 0-9 represent 0-9, A-Z represent 10-35, and a-z represent 36-61.

BUGS

sq and unsq can only handle words of up to 256 characters. The input must be sorted, and duplicates must be suppressed.

AUTHOR

Mike Wexler

SEE ALSO

compress(1), sort(1).