initial commit
This commit is contained in:
commit
60052b2f16
GPG Key ID:
91BE884FBA4B591A
32
MANIFEST
Normal file
32
MANIFEST
Normal file
@ -0,0 +1,32 @@
|
|||||||
|
1 test-file
|
||||||
|
2 MANIFEST
|
||||||
|
D books/
|
||||||
|
D books/tools/
|
||||||
|
3 bootstrap
|
||||||
|
4 bootstrap2
|
||||||
|
5 sortpages
|
||||||
|
6 Makefile
|
||||||
|
7 heap.c
|
||||||
|
8 heap.h
|
||||||
|
9 mempool.c
|
||||||
|
10 mempool.h
|
||||||
|
11 util.c
|
||||||
|
12 util.h
|
||||||
|
13 repair.c
|
||||||
|
14 subst.c
|
||||||
|
15 subst.h
|
||||||
|
16 unmunge.c
|
||||||
|
17 munge.c
|
||||||
|
18 yapp.doc
|
||||||
|
19 yapp
|
||||||
|
20 psgen
|
||||||
|
21 makemanifest
|
||||||
|
D books/ps/
|
||||||
|
22 prolog.ps
|
||||||
|
23 charmap.ps
|
||||||
|
D books/example/
|
||||||
|
24 Makefile
|
||||||
|
25 .cvsignore
|
||||||
|
26 filelist
|
||||||
|
27 footer.ps
|
||||||
|
28 us-constitution.gz
|
||||||
477
README
Normal file
477
README
Normal file
@ -0,0 +1,477 @@
|
|||||||
|
PREFACE
|
||||||
|
-------
|
||||||
|
|
||||||
|
This book grew out of a project to publish source code for cryptographic
|
||||||
|
software, namely PGP (Pretty Good Privacy), a software package for the
|
||||||
|
encryption of electronic mail and computer files. PGP is the most widely
|
||||||
|
used software in the world for email encryption. Pretty Good Privacy, Inc
|
||||||
|
(or "PGP") has published the source code of PGP for peer review, a long-
|
||||||
|
standing tradition in the history of PGP. The first time a fully implemented
|
||||||
|
cryptographic software package was published in its entirety in book form
|
||||||
|
was "PGP Source Code and Internals," by Philip Zimmermann, published by The
|
||||||
|
MIT Press, 1995, ISBN 0-262-24039-4.
|
||||||
|
|
||||||
|
Peer review of the source code is important to get users to trust the
|
||||||
|
software, since any weaknesses can be detected by knowledgeable experts who
|
||||||
|
make the effort to review the code. But peer review cannot be completely
|
||||||
|
effective unless the experts conducting the review can compile and test the
|
||||||
|
software, and verify that it is the same as the software products that are
|
||||||
|
published electronically. To facilitate that, PGP publishes its source code
|
||||||
|
in printed form that can be scanned into a computer via OCR (optical
|
||||||
|
character recognition) technology.
|
||||||
|
|
||||||
|
Why not publish the source code in electronic form? As you may know,
|
||||||
|
cryptographic software is subject to U.S. export control laws and
|
||||||
|
regulations. The new 1997 Commerce Department Export Administration
|
||||||
|
Regulations (EAR) explicitly provide that "A printed book or other printed
|
||||||
|
material setting forth encryption source code is not itself subject to the
|
||||||
|
EAR." (see 15 C.F.R. §734.3(b)(2)). PGP, in an overabundance of caution,
|
||||||
|
has only made available its source code in a form that is not subject to
|
||||||
|
those regulations. So, books containing cryptographic source code may be
|
||||||
|
published, and after they are published they may be exported, but only
|
||||||
|
while they are still in printed form.
|
||||||
|
|
||||||
|
Electronic commerce on the Internet cannot fully be successful without
|
||||||
|
strong cryptography. Cryptography is important for protecting our privacy,
|
||||||
|
civil liberties, and the security of our personal and business transactions
|
||||||
|
in the information age. The widespread deployment of strong cryptography
|
||||||
|
can help us regain some of the privacy and security that we have lost due
|
||||||
|
to information technology. Further, strong cryptography (in the form of
|
||||||
|
PGP) has already proven itself to be a valuable tool for the protection of
|
||||||
|
human rights in oppressive countries around the world, by keeping those
|
||||||
|
governments from reading the communications of human rights workers.
|
||||||
|
|
||||||
|
This book of tools contains no cryptographic software of any kind, nor does
|
||||||
|
it call, connect, nor integrate in any way with cryptographic software. But
|
||||||
|
it does contain tools that make it easy to publish source code in book form.
|
||||||
|
And it makes it easy to scan such source code in with OCR software rapidly
|
||||||
|
and accurately.
|
||||||
|
|
||||||
|
Philip Zimmermann
|
||||||
|
prz@acm.org
|
||||||
|
|
||||||
|
November 1997
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
INTRODUCTION
|
||||||
|
------------
|
||||||
|
|
||||||
|
This book contains tools for printing computer source code on paper in
|
||||||
|
human-readable form and reconstructing it exactly using automated tools.
|
||||||
|
While standard OCR software can recover most of the graphic characters,
|
||||||
|
non-printing characters like tabs, spaces, newlines and form feeds cause
|
||||||
|
problems.
|
||||||
|
|
||||||
|
In fact, these tools can print any ASCII text file; it's just that the
|
||||||
|
attention these tools pay to spacing is particularly valuable for computer
|
||||||
|
source code. The two-dimensional indentation structure of source code is
|
||||||
|
very important to its comprehensibility. In some cases, distinctions
|
||||||
|
between non-printing characters are critical: the standard make utility
|
||||||
|
will not accept spaces where it expects to see a tab character.
|
||||||
|
|
||||||
|
Producing a byte-for-byte identical copy of the original is also valuable
|
||||||
|
for authentication, as you can verify a checksum.
|
||||||
|
|
||||||
|
There are five problems we have addressed:
|
||||||
|
|
||||||
|
1. Getting good OCR accuracy.
|
||||||
|
2. Preserving whitespace.
|
||||||
|
3. Preserving lines longer than can be printed on the page.
|
||||||
|
4. Dealing with data that isn't human-readable.
|
||||||
|
5. Detecting and correcting any residual errors.
|
||||||
|
|
||||||
|
The first problem is partly addressed by using a font designed for OCR
|
||||||
|
purposes, OCR-B. OCR-A is a very ugly font that contains only the digits 0
|
||||||
|
through 9 and a few special punctuation symbols. OCR-B is a very readable
|
||||||
|
monospaced font that contains a full ASCII set, and has been popular as a
|
||||||
|
font on line printers for years because it distinguishes ambiguous
|
||||||
|
characters and is clear even if fuzzy or distorted.
|
||||||
|
|
||||||
|
The most unusual thing about the OCR-B font is the way that it prints a
|
||||||
|
lower-case letter 1, with a small hook on the bottom, something like an
|
||||||
|
upper-case L. This is to distinguish it from the numeral 1. We also made
|
||||||
|
some modifications to the font, to print the numeral 0 with a slash, and
|
||||||
|
to print the vertical bar in a broken form. Both of these are such common
|
||||||
|
variants that they should not present any intelligibility barrier. Finally,
|
||||||
|
we print the underscore character in a distinct manner that is hopefully
|
||||||
|
not visually distracting, but is clearly distinguishable from the minus
|
||||||
|
sign even in the absence of a baseline reference.
|
||||||
|
|
||||||
|
The most significant part of getting good OCR accuracy is, however, using
|
||||||
|
the OCR tools well. We've done a lot of testing and experimentation and
|
||||||
|
present here a lot of information on what works and what doesn't.
|
||||||
|
|
||||||
|
To preserve whitespace, we added some special symbols to display spaces,
|
||||||
|
tabs, and form feeds. A space is printed as a small triangular dot
|
||||||
|
character, while a hollow rightward-pointing triangle (followed by blank
|
||||||
|
spaces to the right tab stop) signifies a tab. A form feed is printed as
|
||||||
|
a yen symbol, and the printed line is broken after the form feed.
|
||||||
|
|
||||||
|
Making the dot triangular instead of square helps distinguish it from a
|
||||||
|
period. To reduce the clutter on the page and make the text more readable,
|
||||||
|
the space character is only printed as a small dot if it follows a blank
|
||||||
|
on the page (a tab or another space), or comes immediately before the end
|
||||||
|
of the line. Thus, the reader (human or software) must be able to
|
||||||
|
distinguish one space from no spaces, but can find multiple spaces by
|
||||||
|
counting the dots (and adding one).
|
||||||
|
|
||||||
|
The format is designed so that 80 characters, plus checksums, can be
|
||||||
|
printed on one line of an 8.5x11" (or A4) page, the still-common punched
|
||||||
|
card line length. Longer lines are managed with the simple technique of
|
||||||
|
appending a big ugly black blob to the first part of the line indicating
|
||||||
|
that the next printed line should be concatenated with the current one
|
||||||
|
with no intervening newline. Hopefully, its use is infrequent.
|
||||||
|
|
||||||
|
While ASCII text is by far the most popular form, some source code is not
|
||||||
|
readable in the usual way. It may be an audio clip, a graphic image bitmap,
|
||||||
|
or something else that is manipulated with a specialized editing tool. For
|
||||||
|
printing purposes, these tools just print any such files as a long string
|
||||||
|
of gibberish in a 64-character set designed to be easy to OCR unambiguously.
|
||||||
|
Although the tools recognize such binary data and apply extra consistency
|
||||||
|
checks, that can be considered a separate step.
|
||||||
|
|
||||||
|
Finally, the problem of residual errors arises. OCR software is not perfect,
|
||||||
|
and uses a variety of heuristics and spelling-check dictionaries to clean up
|
||||||
|
any residual errors in human-language text. This isn't reliable enough for
|
||||||
|
source code, so we have added per-page and per-line checksums to the printed
|
||||||
|
material, and a series of tools to use those checksums to correct any
|
||||||
|
remaining errors and convert the scanned text into a series of files again.
|
||||||
|
|
||||||
|
This "munged" form is what you see in most of the body of this book. We
|
||||||
|
think it does a good job of presenting source code in a way that can be read
|
||||||
|
easily by both humans and computers.
|
||||||
|
|
||||||
|
The tools are command-line oriented and a bit clunky. This has a purpose
|
||||||
|
beyond laziness on the authors' parts: it keeps them small. Keeping them
|
||||||
|
small makes the "bootstrapping" part of scanning this book easier, since you
|
||||||
|
don't have the tools to help you with that.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
SCANNING
|
||||||
|
--------
|
||||||
|
|
||||||
|
Our tests were done with OmniPage 7.0 on a Power Macintosh 8500/120 and an
|
||||||
|
HP ScanJet 4c scanner with an automatic document feeder. The first part of
|
||||||
|
this is heavily OmniPage-specific, as that appears to be the most widely
|
||||||
|
available OCR software.
|
||||||
|
|
||||||
|
The tools here were developed under Linux, and should be generally portable
|
||||||
|
to any Unix platform. Since this book is about printing and scanning source
|
||||||
|
code, we assume the readers have enough programming background to know how
|
||||||
|
to build a program from a Makefile, understand the hazards of CR, LF or CRLF
|
||||||
|
line endings, and such minor details without explicit mention.
|
||||||
|
|
||||||
|
The first step to getting OrnniPage 7 to work well is to set it up with
|
||||||
|
options to disable all of its more advanced features for preserving font
|
||||||
|
changes and formatting. Look in the Seffings menu.
|
||||||
|
|
||||||
|
· Create a Zone Contents File with all of ASCII in it, plus the extra
|
||||||
|
bullet, currency, yen and pilcrow symbols. Name it "Source Code".
|
||||||
|
· Create a Source Code style set. Within it, create a Source Code zone style
|
||||||
|
and make it the default.
|
||||||
|
· Set the font to something fixed-width, like Courier.
|
||||||
|
· Set a fixed font size (10 point) and plain text, left-aligned.
|
||||||
|
· Set the tab character to a space.
|
||||||
|
· Set the text flow to hard line returns.
|
||||||
|
· Set the margins to their widest.
|
||||||
|
· The font mapping options are irrelevant.
|
||||||
|
|
||||||
|
Go to the settings panel and:
|
||||||
|
|
||||||
|
· Under Scanner, set the brightness to manual. With careful setting of the
|
||||||
|
threshold, this generates much better results than either the automatic
|
||||||
|
threshold or the 3D OCR. Around 144 has been a good setting for us; you
|
||||||
|
may want to start there.
|
||||||
|
· Under OCR, you'll build a training file to use later, but turn off
|
||||||
|
automatic page orientation and select your Source Code style set in the
|
||||||
|
Output Options. Also set a reasonable reject character. (For test, we
|
||||||
|
used the pi symbol, which came across from the Macintosh as a weird
|
||||||
|
sequence, but you can use anything as long as you make the appropriate
|
||||||
|
definition in subst.c.)
|
||||||
|
|
||||||
|
Do an initial scan of a few pages and create a manual zone encompassing
|
||||||
|
all of the text. Leave some margin for page misalignment, and leave space
|
||||||
|
on the sides for the left-right shift caused by the book binding being in
|
||||||
|
different places on odd and even pages.
|
||||||
|
|
||||||
|
Set the Zone Contents and the Style set to the Source Code settings. After
|
||||||
|
setting the Style Set, the Zone Style should be automatically set correctly
|
||||||
|
(since you set Source Code as the default).
|
||||||
|
|
||||||
|
Then save the Zone Template, and in the pop-up menu under the Zone step on
|
||||||
|
the main toolbar you can now select it.
|
||||||
|
|
||||||
|
Now we're ready to get characters recognized. The first results will be
|
||||||
|
terrible, with lots of red (unrecognizable) and green (suspicious) text in
|
||||||
|
the recognized window. Some tweaking will improve this enormously.
|
||||||
|
|
||||||
|
The first step is setting a good black threshold. Auto brightness sets the
|
||||||
|
threshold too low, making the character outlines bleed and picking up a lot
|
||||||
|
of glitches on mostly-blank pages. Try training OCR on the few pages you've
|
||||||
|
scanned and look at the representative characters. Adjust the threshold so
|
||||||
|
the strokes are clear and distinct, neither so thin they are broken nor so
|
||||||
|
think they smear into each other. The character that bleeds worst is
|
||||||
|
lowercase w, while the underscore and tab symbols have the thinnest lines
|
||||||
|
that need worry.
|
||||||
|
|
||||||
|
You'll have to re-scan (you can just click the AUTO button) until you get
|
||||||
|
satisfactory results.
|
||||||
|
|
||||||
|
The next step is training. You should scan a significant number of pages
|
||||||
|
and teach OmniPage about any characters it has difficulty with. There are
|
||||||
|
several characters which have been printed in unusual ways which you must
|
||||||
|
teach OmniPage about before it can recognize them reliably. We also have
|
||||||
|
some characters that are unique, which the tools expect to be mapped to
|
||||||
|
specific Latin-1 characters to be processed.
|
||||||
|
|
||||||
|
They characters most in need of training are as follows:
|
||||||
|
|
||||||
|
· Zero is printed 'slashed.'
|
||||||
|
· Lowercase L has a curled tail to distinguish it clearly from other
|
||||||
|
vertical characters like 1 and I.
|
||||||
|
· The or-bar or pipe symbol '|' is printed "broken" with a gap in the
|
||||||
|
middle to distinguish it similarly.
|
||||||
|
· The underscore character has little "serifs" on the end to distinguish
|
||||||
|
it from a minus sign. We also raised it a just a tad higher than the
|
||||||
|
normal underscore character, which was too low in the character cell to
|
||||||
|
be reliably seen by OmniPage.
|
||||||
|
· Tabs are printed as a hollow right-pointing triangle, followed by blanks
|
||||||
|
to the correct alignment position. If not trained enough, OmniPage
|
||||||
|
guesses this is a capital D. You should train OmniPage to recognize this
|
||||||
|
symbol as a currency symbol (Latin-1 244).
|
||||||
|
· Any spaces in the original that follow a space, or a blank on the printed
|
||||||
|
page, are printed as a tiny black triangle. You should train OmniPage to
|
||||||
|
recognize this as a center dot or bullet (Latin-1 267). We didn't use a
|
||||||
|
standard center dot because OmniPage confused it with a period.
|
||||||
|
· Any form feeds in the original are printed as a yen currency symbol
|
||||||
|
(Latin-1 245).
|
||||||
|
· Lines over 80 columns long are broken after 79 columns by appending a big
|
||||||
|
ugly black block. You should train OmniPage to recognize this as a
|
||||||
|
pilcrow (paragraph symbol, Latin-1 266). We did this because after
|
||||||
|
deciding something black and visible was suitable, we found out the font
|
||||||
|
we used doesn't have a pilcrow in it.
|
||||||
|
|
||||||
|
The zero and the tab character, because of their frequency, deserve special
|
||||||
|
attention.
|
||||||
|
|
||||||
|
In addition, look for any unrecognized characters (in red) and retrain those
|
||||||
|
pages. If you get an unrecognized character, that character needs training,
|
||||||
|
but Caere says that "good examples" are best to train on, so if the training
|
||||||
|
doesn't recognize a slightly fuzzy K, and there's a nice crisp K available
|
||||||
|
to train on, use that.
|
||||||
|
|
||||||
|
Other things that need training:
|
||||||
|
|
||||||
|
· ~ (tilde), ^ (caret), ` (backquote) and ' (quote). These get dropped
|
||||||
|
frequently unless you train them.
|
||||||
|
· i, j and; (semicolon). These get mixed up.
|
||||||
|
· 3 and S. These also get mixed up.
|
||||||
|
· Q can fail to be recognized.
|
||||||
|
· C and [ can be confused.
|
||||||
|
· c/C, o/O, p/P, s/S, u/U, v/V, w/W, y/Y and z/Z are often confused. This
|
||||||
|
can be helped by some training.
|
||||||
|
· r gets confused with c and n. I don't understand c, but it happens.
|
||||||
|
· f gets confused with i.
|
||||||
|
|
||||||
|
The OCR training pages have lots of useful examples of troublesome
|
||||||
|
characters. Scan a few pages of material, training each page, then scan a
|
||||||
|
few dozen pages and look for recognition problems. Look for what OmniPage
|
||||||
|
reports as troublesome, and when you have the repair program working, use
|
||||||
|
it to find and report further errors. Train a few pages particularly dense
|
||||||
|
in problems and append the troublesome characters to the training file, the
|
||||||
|
re-recognize the lot.
|
||||||
|
|
||||||
|
Double-check your training file for case errors. It's easy to miss the shift
|
||||||
|
key in the middle of a lot of training and will result in terrible results
|
||||||
|
even though OmniPage won't report anything amiss. We have spent a while
|
||||||
|
wondering why OmniPage wasn't recognizing capital S or capital W, only to
|
||||||
|
find that OmniPage was just doing what it was trained to do.
|
||||||
|
|
||||||
|
We have heard some reports that OmniPage has problems with large training
|
||||||
|
files. We have observed OmniPage suffering repeatable internal errors
|
||||||
|
sometimes after massive training additions, but they were cured by deleting
|
||||||
|
a few training images. Appending more training images to the training file
|
||||||
|
did not cause the problem to re-appear.
|
||||||
|
|
||||||
|
Repairing the OCR results
|
||||||
|
|
||||||
|
If the only copy of the tools you have is printed in this book, see the next
|
||||||
|
chapter on bootstrapping at this point. Here, we assume that you have the
|
||||||
|
tools and they work.
|
||||||
|
|
||||||
|
When you have some reasonable OCR results, delete any directory pages. With
|
||||||
|
no checksum information, they just confuse the postprocessing tools. (The
|
||||||
|
tools will just stop with an error when they get to the "uncorrectable"
|
||||||
|
directory name and you'll have to delete it then, so it's not fatal if you
|
||||||
|
forget.) Copy the data to a machine that you have the repair and unmunge
|
||||||
|
utilities on.
|
||||||
|
|
||||||
|
The repair utility attempts automatic table-driven correction of common
|
||||||
|
scanning errors. You have to recompile it to change the tables, but are
|
||||||
|
encouraged to if you find a common problem that it does not correct reliably.
|
||||||
|
If it gets stuck, it will deposit you into your favorite editor on or
|
||||||
|
slightly after the offending line. (The file you will be editing is the
|
||||||
|
unprocessed portion of the input.) After you correct the problem and quit
|
||||||
|
the editor, repair will resume.
|
||||||
|
|
||||||
|
"Your favorite editor" is taken from the $VISUAL and $EDITOR environment
|
||||||
|
variables, or the -e option to repair.
|
||||||
|
|
||||||
|
The repair utility never alters the original input file. It will produce
|
||||||
|
corrected output for file in file.out, and when it has to stop, it writes
|
||||||
|
any remaining uncorrected input back out to file.in (via a temporary
|
||||||
|
file.dump) and lets you edit this file. If you re-run repair on file and
|
||||||
|
file.in exists, repair will restart from there, so you may safely quit and
|
||||||
|
re-run repair as often as you like. (But if you change the input file, you
|
||||||
|
need to delete the .in file for repair to notice the change.)
|
||||||
|
|
||||||
|
Statistics on repair's work are printed to file.log. This is an excellent
|
||||||
|
place to look to see if any characters require more training.
|
||||||
|
|
||||||
|
As it works, repair prints the line it is working on. If you see it make a
|
||||||
|
mistake or get stuck, you can interrupt it (control-C or whatever is
|
||||||
|
appropriate), and it will immediately drop into the editor. If you interrupt
|
||||||
|
it a second time, it will exit rather than invoking the editor. If the
|
||||||
|
editor returns a non-zero result code (fails), repair will also stop. (E.g.
|
||||||
|
:cq in vim.)
|
||||||
|
|
||||||
|
One thing that repair fixes without the least trouble is the number of
|
||||||
|
spaces expected after a printing tab character. It's such an omnipresent OCR
|
||||||
|
software error that repair doesn't even log it as a correction.
|
||||||
|
|
||||||
|
In some cases, repair can miscorrect a line and go on to the next line,
|
||||||
|
possibly even more than once, finally giving up a few lines below the actual
|
||||||
|
error. If you are having trouble spotting the error, one helpful trick is to
|
||||||
|
exit the editor and let repair try to fix the page again, but interrupt it
|
||||||
|
while it is still working on the first line, before it has found the
|
||||||
|
miscorrection.
|
||||||
|
|
||||||
|
The Nasty Lines
|
||||||
|
|
||||||
|
Some lines of code, particularly those containing long runs of underscore or
|
||||||
|
minus characters, are particularly difficult to scan reliably. The repair
|
||||||
|
program has a special "nasty lines" feature to deal with this. If a file
|
||||||
|
named "nastylines" (or as specified by the -l option) exists, they are
|
||||||
|
checksummed and are considered as total replacements for any input line with
|
||||||
|
the same checksum. So, for example, if you place a blank line in the
|
||||||
|
nastylines file, any scanner noise on blank lines will be ignored.
|
||||||
|
|
||||||
|
The "nastylines" file is re-read every time repair restarts after an edit,
|
||||||
|
so you can add more lines as the program runs. (The error-correction patterns
|
||||||
|
should be done this way, too, but that'll have to wait for the next release.)
|
||||||
|
|
||||||
|
Sortpages
|
||||||
|
|
||||||
|
If, in the course of scanning, the pages have been split up or have gotten
|
||||||
|
out of order, a perl script called sortpages can restore them to the proper
|
||||||
|
order. It can merge multiple input files, discard duplicates, and warns about
|
||||||
|
any missing pages it encounters. This script requires that the pages have
|
||||||
|
been repaired, so that the page headers can be read reliably. The repair
|
||||||
|
program does not care about the order it works on pages in; it examines each
|
||||||
|
page independently. Unmunge, however, does need the pages in order.
|
||||||
|
|
||||||
|
Unmunging
|
||||||
|
|
||||||
|
After repair has finished its work, the unmunge program strips out the
|
||||||
|
checksums and, based on the page headers, divides the data up among various
|
||||||
|
files. Its first argument is the file to unpack. The optional second argument
|
||||||
|
is a manifest file that lists all of the files and the directories they go
|
||||||
|
in. Supplying this (an excellent idea) lets unmunge recreate a directory
|
||||||
|
hierarchy and warn about missing files.
|
||||||
|
|
||||||
|
When you have unmunged everything and reconstructed the original source code,
|
||||||
|
you are done. Unmunge verifies all of the checksums independently of repair,
|
||||||
|
as a sanity check, and you can have high confidence that the files are
|
||||||
|
exactly the same as the originals that were printed.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
BOOTSTRAPPING
|
||||||
|
-------------
|
||||||
|
|
||||||
|
There's a problem using the postprocessing tools to correct OCR errors, when
|
||||||
|
the code being OCRed is the tools themselves. We've tried to provide a
|
||||||
|
reasonably easy way to get the system up and running starting from nothing
|
||||||
|
but a copy of OmniPage.
|
||||||
|
|
||||||
|
You could just scan all of the tools in, correct any errors by hand, delete
|
||||||
|
the error-checking information in a text editor, and compile them. But
|
||||||
|
finding all the errors by hand is painful in a body of code that large.
|
||||||
|
With the aid of perl (version 5), which provides a lot of power in very
|
||||||
|
little code, we have provided some utilities to make this process easier.
|
||||||
|
|
||||||
|
The first-stage bootstrap is a one-page perl script designed to be as small
|
||||||
|
and simple as possible, because you'll have to hand-correct it. It can verify
|
||||||
|
the checksums on each line, and drop you into the editor on any lines where
|
||||||
|
an error has occurred. It also knows how to strip out the visible spaces and
|
||||||
|
tabs, how to correct spacing errors after visible tab characters, and how to
|
||||||
|
invoke an editor on the erroneous line.
|
||||||
|
|
||||||
|
Scan in the first-stage bootstrap as carefully as possible, using OmniPage's
|
||||||
|
warnings to guide you to any errors, and either use a text editor or the
|
||||||
|
one-line perl command at the top of the file to remove the checksums and
|
||||||
|
convert any funny printed characters to whitespace form.
|
||||||
|
|
||||||
|
The first thing to do is try running it on itself, and correct any errors you
|
||||||
|
find this way. Note that the script writes its output to the file named in
|
||||||
|
the page header, so you should name your hand-corrected version differently
|
||||||
|
(or put it in a different directory) to avoid having it overwritten.
|
||||||
|
|
||||||
|
The second-stage bootstrap is a much denser one-pager, with better error
|
||||||
|
detection; it can detect missing lines and missing pages, and takes an
|
||||||
|
optional second argument of a manifest file which it can use to put files
|
||||||
|
in their proper directories. It's not strictly necessary, but it's only one
|
||||||
|
more (dense) page and you can check it against itself and the original
|
||||||
|
bootstrap.
|
||||||
|
|
||||||
|
Both of the botstrap utilities can correct tab spacing errors in the OCR
|
||||||
|
output. Although this doesn't matter in most source code, it is included
|
||||||
|
in the checksums.
|
||||||
|
|
||||||
|
Once you have reached this point, you can scan in the C code for repair and
|
||||||
|
unmunge. The C unmunge is actually less friendly than the bootstrap
|
||||||
|
utilities, because it is only intended to work with the output of repair.
|
||||||
|
It is, however, much faster, since computing CRCs a bit at a time in an
|
||||||
|
interpreted language is painfully slow for large amounts of data. It can
|
||||||
|
also deal with binary files printed in radix-64.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
PRINTING
|
||||||
|
--------
|
||||||
|
|
||||||
|
Despite the title of this book, this process of producing a book is not well
|
||||||
|
documented, since it's been evolving up to the moment of publication. There,
|
||||||
|
is, however, a very useful working example of how to produce a book
|
||||||
|
(strikingly similar to this book) in the example directory, all controlled
|
||||||
|
by a Makefile.
|
||||||
|
|
||||||
|
Briefly, a master perl script called psgen takes three parameters: a file
|
||||||
|
list, a page numbers file to write to, and a volume number (which should
|
||||||
|
always be 1 for a one-volume book). It runs the listed files through the
|
||||||
|
munge utility, wraps them in some simple PostScript, and prepends a prolog
|
||||||
|
that defines the special characters and PostScript functions needed by the
|
||||||
|
text.
|
||||||
|
|
||||||
|
The file list also includes per-file flags. The most important is the
|
||||||
|
text/binary marker. Text files can also have a tab width specified, although
|
||||||
|
munge knows how to read Emacs-style tab width settings from the end of a
|
||||||
|
source file.
|
||||||
|
|
||||||
|
The prolog is assembled from various other files and defines by psgen using
|
||||||
|
a simple preprocessor called yapp (Yet Another Preprocessor). This process
|
||||||
|
includes some book-specific information like the page footer.
|
||||||
|
|
||||||
|
Producing the final PostScript requires the necessary non-standard fonts
|
||||||
|
(Futura for the footers and OCRB for the code) and the psutils package,
|
||||||
|
which provides the includeres utility used to embed the fonts in the
|
||||||
|
PostScript file. The fonts should go in the books/ps directory, as
|
||||||
|
"Futura.pfa" and the like.
|
||||||
|
|
||||||
|
The pagenums file can be used to produce a table of contents. For this book,
|
||||||
|
we generated the front matter (such as this chapter) separately, told psgen
|
||||||
|
to start on the next page after this, and concatenated the resultant
|
||||||
|
PostScript files for printing. The only trick was making the page footers
|
||||||
|
look identical.
|
||||||
3
example/.cvsignore
Normal file
3
example/.cvsignore
Normal file
@ -0,0 +1,3 @@
|
|||||||
|
pagenums
|
||||||
|
MANIFEST
|
||||||
|
code.ps
|
||||||
23
example/Makefile
Normal file
23
example/Makefile
Normal file
@ -0,0 +1,23 @@
|
|||||||
|
BOOKROOT=..
|
||||||
|
TOOLSDIR=$(BOOKROOT)/tools
|
||||||
|
PSDIR=$(BOOKROOT)/ps
|
||||||
|
YAPP=$(TOOLSDIR)/yapp
|
||||||
|
MAKEMANIFEST=$(TOOLSDIR)/makemanifest
|
||||||
|
PSGEN=BOOKROOT=$(BOOKROOT) $(TOOLSDIR)/psgen
|
||||||
|
INCLUDERES=(cd $(PSDIR); includeres)
|
||||||
|
|
||||||
|
code.ps pagenums: filelist footer.ps MANIFEST books
|
||||||
|
$(PSGEN) -P2 -l3 -DfooterFile=footer.ps filelist pagenums 1 \
|
||||||
|
| $(INCLUDERES) > code.ps
|
||||||
|
|
||||||
|
books:
|
||||||
|
ln -s $(BOOKROOT) books
|
||||||
|
|
||||||
|
MANIFEST: filelist
|
||||||
|
$(MAKEMANIFEST) $< > $@
|
||||||
|
|
||||||
|
clean:
|
||||||
|
rm -f `cat .cvsignore`
|
||||||
|
|
||||||
|
gv%: %.ps
|
||||||
|
gv $<
|
||||||
32
example/filelist
Normal file
32
example/filelist
Normal file
@ -0,0 +1,32 @@
|
|||||||
|
V 1 8
|
||||||
|
T MANIFEST
|
||||||
|
D books/
|
||||||
|
D books/tools/
|
||||||
|
T books/tools/bootstrap
|
||||||
|
T books/tools/bootstrap2
|
||||||
|
T4 books/tools/sortpages
|
||||||
|
T books/tools/Makefile
|
||||||
|
T books/tools/heap.c
|
||||||
|
T books/tools/heap.h
|
||||||
|
T books/tools/mempool.c
|
||||||
|
T books/tools/mempool.h
|
||||||
|
T books/tools/util.c
|
||||||
|
T books/tools/util.h
|
||||||
|
T books/tools/repair.c
|
||||||
|
T books/tools/subst.c
|
||||||
|
T books/tools/subst.h
|
||||||
|
T books/tools/unmunge.c
|
||||||
|
T books/tools/munge.c
|
||||||
|
T books/tools/yapp.doc
|
||||||
|
T4 books/tools/yapp
|
||||||
|
T4 books/tools/psgen
|
||||||
|
T4 books/tools/makemanifest
|
||||||
|
D books/ps/
|
||||||
|
T books/ps/prolog.ps
|
||||||
|
T books/ps/charmap.ps
|
||||||
|
D books/example/
|
||||||
|
T books/example/Makefile
|
||||||
|
T books/example/.cvsignore
|
||||||
|
T books/example/filelist
|
||||||
|
T books/example/footer.ps
|
||||||
|
B books/example/us-constitution.gz
|
||||||
5
example/footer.ps
Normal file
5
example/footer.ps
Normal file
@ -0,0 +1,5 @@
|
|||||||
|
% A program to print the page footer, using the magic P function,
|
||||||
|
% which takes a string and a font.
|
||||||
|
(Tools for Publishing Source Code via OCR ) /Futura P
|
||||||
|
(\343) /Symbol P % Copyright symbol
|
||||||
|
( 1997 Pretty Good Privacy, Inc.) /Futura P
|
||||||
BIN
example/us-constitution.gz
Normal file
BIN
example/us-constitution.gz
Normal file
Binary file not shown.
68
ps/charmap.ps
Normal file
68
ps/charmap.ps
Normal file
@ -0,0 +1,68 @@
|
|||||||
|
%%BeginResource: procset Latin1-vec 0 0
|
||||||
|
/Latin1-vec [
|
||||||
|
/.notdef /.notdef /.notdef /.notdef
|
||||||
|
/.notdef /.notdef /.notdef /.notdef
|
||||||
|
/.notdef /.notdef /.notdef /.notdef
|
||||||
|
/.notdef /.notdef /.notdef /.notdef
|
||||||
|
/.notdef /.notdef /.notdef /.notdef
|
||||||
|
/.notdef /.notdef /.notdef /.notdef
|
||||||
|
/.notdef /.notdef /.notdef /.notdef
|
||||||
|
/.notdef /.notdef /.notdef /.notdef
|
||||||
|
/space /exclam /quotedbl /numbersign
|
||||||
|
/dollar /percent /ampersand /${rightQuoteGlyph}
|
||||||
|
/parenleft /parenright /asterisk /plus
|
||||||
|
/comma /hyphen /period /slash
|
||||||
|
/${zeroGlyph} /one /two /three
|
||||||
|
/four /five /six /seven
|
||||||
|
/eight /nine /colon /semicolon
|
||||||
|
/less /equal /greater /question
|
||||||
|
/at /A /B /C
|
||||||
|
/D /E /F /G
|
||||||
|
/H /I /J /K
|
||||||
|
/L /M /N /O
|
||||||
|
/P /Q /R /S
|
||||||
|
/T /U /V /W
|
||||||
|
/X /Y /Z /bracketleft
|
||||||
|
/backslash /bracketright /asciicircum /${underscoreGlyph}
|
||||||
|
/${leftQuoteGlyph} /a /b /c
|
||||||
|
/d /e /f /g
|
||||||
|
/h /i /j /k
|
||||||
|
/l /m /n /o
|
||||||
|
/p /q /r /s
|
||||||
|
/t /u /v /w
|
||||||
|
/x /y /z /braceleft
|
||||||
|
/${barGlyph} /braceright /tilde /.notdef
|
||||||
|
/.notdef /.notdef /.notdef /.notdef
|
||||||
|
/.notdef /.notdef /.notdef /.notdef
|
||||||
|
/.notdef /.notdef /.notdef /.notdef
|
||||||
|
/.notdef /.notdef /.notdef /.notdef
|
||||||
|
/.notdef /.notdef /.notdef /.notdef
|
||||||
|
/.notdef /.notdef /.notdef /.notdef
|
||||||
|
/.notdef /.notdef /.notdef /.notdef
|
||||||
|
/.notdef /.notdef /.notdef /.notdef
|
||||||
|
/space /exclamdown /cent /sterling
|
||||||
|
/${tabGlyph} /yen /brokenbar /section
|
||||||
|
/dieresis /copyright /ordfeminine /guillemotleft
|
||||||
|
/logicalnot /hyphen /registered /macron
|
||||||
|
/degree /plusminus /twosuperior /threesuperior
|
||||||
|
/acute /mu /${pilcrowGlyph} /${bulletGlyph}
|
||||||
|
/cedilla /dotlessi /ordmasculine /guillemotright
|
||||||
|
/onequarter /onehalf /threequarters /questiondown
|
||||||
|
/Agrave /Aacute /Acircumflex /Atilde
|
||||||
|
/Adieresis /Aring /AE /Ccedilla
|
||||||
|
/Egrave /Eacute /Ecircumflex /Edieresis
|
||||||
|
/Igrave /Iacute /Icircumflex /Idieresis
|
||||||
|
/Eth /Ntilde /Ograve /Oacute
|
||||||
|
/Ocircumflex /Otilde /Odieresis /multiply
|
||||||
|
/Oslash /Ugrave /Uacute /Ucircumflex
|
||||||
|
/Udieresis /Yacute /Thorn /germandbls
|
||||||
|
/agrave /aacute /acircumflex /atilde
|
||||||
|
/adieresis /aring /ae /ccedilla
|
||||||
|
/egrave /eacute /ecircumflex /edieresis
|
||||||
|
/igrave /iacute /icircumflex /idieresis
|
||||||
|
/eth /ntilde /ograve /oacute
|
||||||
|
/ocircumflex /otilde /odieresis /divide
|
||||||
|
/oslash /ugrave /uacute /ucircumflex
|
||||||
|
/udieresis /yacute /thorn /ydieresis
|
||||||
|
]def
|
||||||
|
%%EndResource
|
||||||
306
ps/prolog.ps
Normal file
306
ps/prolog.ps
Normal file
@ -0,0 +1,306 @@
|
|||||||
|
##set pageNumFont="Futura"
|
||||||
|
##set dirNameFont="Futura-Heavy"
|
||||||
|
##set fontsNeeded="${font} Symbol Futura Futura-Heavy"
|
||||||
|
##set includeFontComments=<<"END"
|
||||||
|
%%IncludeResource: font ${font}
|
||||||
|
%%IncludeResource: font Symbol
|
||||||
|
%%IncludeResource: font Futura
|
||||||
|
%%IncludeResource: font Futura-Heavy
|
||||||
|
END
|
||||||
|
##if ${font} eq Courier
|
||||||
|
##set charShrinkFactor=0.93
|
||||||
|
##set zeroGlyph=Oslash
|
||||||
|
##set underscoreGlyph=underscore
|
||||||
|
##set bulletGlyph=bullet
|
||||||
|
##set tabGlyph=currency
|
||||||
|
##set leftQuoteGlyph=quoteleft
|
||||||
|
##set rightQuoteGlyph=quoteright
|
||||||
|
##set pilcrowGlyph=paragraph
|
||||||
|
##set barGlyph=bar
|
||||||
|
##else
|
||||||
|
##set charShrinkFactor=1
|
||||||
|
##set zeroGlyph=Oslash
|
||||||
|
##set underscoreGlyph=underscore2
|
||||||
|
##set bulletGlyph=bullet2
|
||||||
|
##set tabGlyph=tabsym
|
||||||
|
##set leftQuoteGlyph=grave
|
||||||
|
##set rightQuoteGlyph=quoteright ### was "acute"
|
||||||
|
##set pilcrowGlyph=erase
|
||||||
|
##set barGlyph=orsym
|
||||||
|
##set do_custom_chars=1
|
||||||
|
##endif
|
||||||
|
%!PS-Adobe-3.0
|
||||||
|
%%Orientation: Portrait
|
||||||
|
%%Pages: (atend)
|
||||||
|
%%DocumentNeededResources: font ${fontsNeeded}
|
||||||
|
%%DocumentMedia: Letter 612 792 74 white ()
|
||||||
|
%%EndComments
|
||||||
|
%%BeginDefaults
|
||||||
|
%%PageMedia: Letter
|
||||||
|
%%PageResources: font ${fontsNeeded}
|
||||||
|
%%EndDefaults
|
||||||
|
%%BeginProlog
|
||||||
|
%%BeginResource: procset Custom-Preamble 0 0
|
||||||
|
%
|
||||||
|
% Document definitions
|
||||||
|
% (Upper case to avoid collisions)
|
||||||
|
%
|
||||||
|
|
||||||
|
% 8.5x11 paper is 612x792 points, but 24 points near the edge or so
|
||||||
|
% shouldn't be used.
|
||||||
|
/Topmargin 770 def
|
||||||
|
/Leftmargin 30 def
|
||||||
|
/Rightmargin 612 Leftmargin sub def
|
||||||
|
/Botmargin 22 def
|
||||||
|
/Bindoffset 40 def
|
||||||
|
|
||||||
|
/Lineskip -10 def
|
||||||
|
% How much to shrink characters by?
|
||||||
|
/Factor ${charShrinkFactor} def
|
||||||
|
/Fontsize 9.5 Factor mul def
|
||||||
|
% (1000 units is std height, so Courier at 6/10 aspect ratio is 600.
|
||||||
|
% Widen to make up for scaling loss.
|
||||||
|
/Charwidth
|
||||||
|
Rightmargin Leftmargin sub Bindoffset sub 87 div Fontsize div 1000 mul
|
||||||
|
def
|
||||||
|
|
||||||
|
% Print a header (expects page number on stack)
|
||||||
|
/OddPageStart
|
||||||
|
{ save exch /MyFont findfont Fontsize scalefont setfont
|
||||||
|
/CurrentLeft Leftmargin Bindoffset add def
|
||||||
|
/CurrentRight Rightmargin def
|
||||||
|
CurrentLeft Topmargin moveto } def
|
||||||
|
|
||||||
|
/EvenPageStart
|
||||||
|
{ save exch /MyFont findfont Fontsize scalefont setfont
|
||||||
|
/CurrentLeft Leftmargin def
|
||||||
|
/CurrentRight Rightmargin Bindoffset sub def
|
||||||
|
CurrentLeft Topmargin moveto } def
|
||||||
|
|
||||||
|
% /MyFont findfont [Fontsize 0 0 Fontsize 0 0] makefont setfont
|
||||||
|
|
||||||
|
% Print the name of the directory in a large font
|
||||||
|
/DirPage
|
||||||
|
{
|
||||||
|
/${dirNameFont} findfont 14 scalefont setfont
|
||||||
|
0 -10 rmoveto (Directory) show
|
||||||
|
CurrentLeft 30 add currentpoint exch pop 20 sub moveto show
|
||||||
|
} def
|
||||||
|
|
||||||
|
% Advance a line
|
||||||
|
/L {show CurrentLeft currentpoint exch pop Lineskip add moveto} bind def
|
||||||
|
|
||||||
|
% Print the "inside" footer line using P (string font => )
|
||||||
|
% We do some magic involving redefining P to first measure the
|
||||||
|
% width of this string and then print it, so you must use it
|
||||||
|
% to do all printing.
|
||||||
|
/Foot {
|
||||||
|
##ifdef footerFile
|
||||||
|
##include "${footerFile}"
|
||||||
|
##endif
|
||||||
|
} def
|
||||||
|
|
||||||
|
% /P is defined in the Setup section
|
||||||
|
|
||||||
|
% Print an odd footer
|
||||||
|
/OddPageEnd
|
||||||
|
{ CurrentLeft Botmargin moveto CurrentRight Botmargin lineto
|
||||||
|
1 setlinewidth stroke
|
||||||
|
CurrentLeft Botmargin 10 sub moveto
|
||||||
|
Foot
|
||||||
|
10 string cvs dup stringwidth
|
||||||
|
pop CurrentRight exch sub currentpoint exch pop moveto
|
||||||
|
/${pageNumFont} P
|
||||||
|
showpage
|
||||||
|
restore
|
||||||
|
} def
|
||||||
|
|
||||||
|
% Print an even footer
|
||||||
|
/EvenPageEnd
|
||||||
|
{ CurrentLeft Botmargin moveto CurrentRight Botmargin lineto
|
||||||
|
1 setlinewidth stroke
|
||||||
|
Leftmargin Botmargin 10 sub moveto
|
||||||
|
/${pageNumFont} P
|
||||||
|
CurrentRight FootWidth sub currentpoint exch pop moveto
|
||||||
|
Foot
|
||||||
|
showpage
|
||||||
|
restore
|
||||||
|
} def
|
||||||
|
|
||||||
|
##ifdef do_custom_chars
|
||||||
|
% A 1000-point OCRB discunderline consists of:
|
||||||
|
% 111.45 -173.688 moveto
|
||||||
|
% 609.356 -173.688 lineto
|
||||||
|
% 609.356 -70.9227 lineto
|
||||||
|
% 111.45 -70.9227 lineto
|
||||||
|
% closepath
|
||||||
|
% 720.0 -0.0 moveto
|
||||||
|
% Line thickness is
|
||||||
|
% 102.7653 pts.
|
||||||
|
|
||||||
|
% This would suggest the following values:
|
||||||
|
/underleft 111.45 def
|
||||||
|
/underright 609.356 def
|
||||||
|
/underthick 102.7643 def
|
||||||
|
/underup underthick def
|
||||||
|
/underdown 0 def
|
||||||
|
/underserif 25 def
|
||||||
|
|
||||||
|
% These look better in GhostScript, but not on a real Adobe rasterizer
|
||||||
|
%/underright 600 def
|
||||||
|
%/underleft 100 def
|
||||||
|
%/underthick 75 def
|
||||||
|
|
||||||
|
171
|
||||||
|
211
|
||||||
|
36081
|
||||||
|
% The default bullet character is
|
||||||
|
% 254.0 341.0 moveto
|
||||||
|
% 254.0 170.0 lineto
|
||||||
|
% 465.0 170.0 lineto
|
||||||
|
% 465.0 341.0 lineto
|
||||||
|
% closepath
|
||||||
|
% Our modified version is based on:
|
||||||
|
/bullwid 204 def
|
||||||
|
/bullht 176.75 def
|
||||||
|
/bullleft 254 341 add bullwid sub 2 div def
|
||||||
|
/bullright 254 341 add bullwid add 2 div def
|
||||||
|
/bullbot 254 def
|
||||||
|
/bulltop bullbot bullht add def
|
||||||
|
|
||||||
|
% And a custom-created tab symbol
|
||||||
|
/tableft 250 def
|
||||||
|
/tabright 550 def
|
||||||
|
/tabtop 550 def
|
||||||
|
/tabbot 50 def
|
||||||
|
/tablinewidth 35 def
|
||||||
|
|
||||||
|
% Let's try a vertical bar
|
||||||
|
% OCRB defines (|)
|
||||||
|
% 411.062 -173.688 moveto
|
||||||
|
% 411.062 741.043 lineto
|
||||||
|
% 308.297 741.043 lineto
|
||||||
|
% 308.297 -173.688 lineto
|
||||||
|
% closepath
|
||||||
|
% 720.0 -0.0 moveto
|
||||||
|
/orleft 308.297 def
|
||||||
|
/orright 411.062 def
|
||||||
|
/orbot -173.688 def
|
||||||
|
/ortop 741.043 def
|
||||||
|
/orbreak 150 def % Width of break
|
||||||
|
/orbbot ortop orbot add orbreak sub 2 div def % Bottom of break
|
||||||
|
/orbtop ortop orbot add orbreak add 2 div def % Top of break
|
||||||
|
##endif
|
||||||
|
|
||||||
|
% newfontname encoding-vec fontname -> - make a new encoded font
|
||||||
|
/MF2 {
|
||||||
|
% Make a dict for the new font, with room for the /Metrics
|
||||||
|
findfont dup length 1 add dict begin
|
||||||
|
% Copy everything except the FID entry
|
||||||
|
{1 index /FID eq {pop pop} {def} ifelse} forall
|
||||||
|
% Set the encoding vector
|
||||||
|
/Encoding exch def
|
||||||
|
|
||||||
|
##ifdef do_custom_chars
|
||||||
|
% Create a new expanded CharStrings dictionary
|
||||||
|
CharStrings dup length 5 add dict
|
||||||
|
begin { def } forall
|
||||||
|
% Create a custom underscore character
|
||||||
|
/underscore2 {
|
||||||
|
pop
|
||||||
|
//Charwidth 0 % width, bounding box follows
|
||||||
|
//underleft //underdown neg //underright //underthick //underup add
|
||||||
|
setcachedevice
|
||||||
|
//underleft //underthick //underup add moveto
|
||||||
|
//underleft //underserif add //underthick //underup add lineto
|
||||||
|
//underleft //underserif add //underthick lineto
|
||||||
|
//underright //underserif sub //underthick lineto
|
||||||
|
//underright //underserif sub //underthick //underup add lineto
|
||||||
|
//underright //underthick //underup add lineto
|
||||||
|
//underright //underdown neg lineto
|
||||||
|
//underright //underserif sub //underdown neg lineto
|
||||||
|
//underright //underserif sub 0 lineto
|
||||||
|
//underleft //underserif add 0 lineto
|
||||||
|
//underleft //underserif add //underdown neg lineto
|
||||||
|
//underleft //underdown neg lineto
|
||||||
|
closepath fill
|
||||||
|
} bind def
|
||||||
|
% Create a custom bullet character.
|
||||||
|
/bullet2 {
|
||||||
|
pop
|
||||||
|
//Charwidth 0 % width, bounding box follows
|
||||||
|
//bullleft //bullbot //bullright //bulltop
|
||||||
|
setcachedevice
|
||||||
|
//bullleft //bullbot moveto
|
||||||
|
//bullleft bullright add 2 div bulltop lineto
|
||||||
|
//bullright //bullbot lineto
|
||||||
|
closepath fill
|
||||||
|
} bind def
|
||||||
|
% Create a custom tab character.
|
||||||
|
/tabsym {
|
||||||
|
pop
|
||||||
|
//Charwidth 0 % width, bounding box follows
|
||||||
|
//tableft //tablinewidth sub //tabbot //tablinewidth sub
|
||||||
|
//tabright //tablinewidth add //tabtop //tablinewidth add
|
||||||
|
setcachedevice
|
||||||
|
//tablinewidth setlinewidth
|
||||||
|
true setstrokeadjust
|
||||||
|
0 setlinejoin
|
||||||
|
//tableft //tabbot moveto
|
||||||
|
//tabright //tabtop //tabbot add 2 div lineto
|
||||||
|
//tableft //tabtop lineto
|
||||||
|
closepath stroke
|
||||||
|
} bind def
|
||||||
|
/orsym {
|
||||||
|
pop
|
||||||
|
//Charwidth 0 % width, bounding box follows
|
||||||
|
//orleft //orbot //orright //ortop
|
||||||
|
setcachedevice
|
||||||
|
//orleft //orbot moveto
|
||||||
|
//orleft //orbbot lineto
|
||||||
|
//orright //orbbot lineto
|
||||||
|
//orright //orbot lineto
|
||||||
|
closepath
|
||||||
|
//orleft //ortop moveto
|
||||||
|
//orleft //orbtop lineto
|
||||||
|
//orright //orbtop lineto
|
||||||
|
//orright //ortop lineto
|
||||||
|
closepath fill
|
||||||
|
} bind def
|
||||||
|
/CharStrings currentdict end def
|
||||||
|
##endif
|
||||||
|
|
||||||
|
% Create a new dict to be the /Metrics values
|
||||||
|
CharStrings dup length dict
|
||||||
|
% Now fill in the metrics dict with the desired width
|
||||||
|
begin { pop Charwidth def } forall /Metrics currentdict end def
|
||||||
|
% End of definitions
|
||||||
|
currentdict end
|
||||||
|
% Define the font
|
||||||
|
definefont pop
|
||||||
|
} bind def
|
||||||
|
|
||||||
|
% Check PostScript language level.
|
||||||
|
/gs_languagelevel /languagelevel where { pop languagelevel } { 1 } ifelse def
|
||||||
|
|
||||||
|
%%EndResource
|
||||||
|
##include "charmap.ps"
|
||||||
|
${includeFontComments}
|
||||||
|
%%EndProlog
|
||||||
|
|
||||||
|
|
||||||
|
%%BeginSetup
|
||||||
|
|
||||||
|
/MyFont Latin1-vec /${font} MF2
|
||||||
|
/#copies 1 def
|
||||||
|
|
||||||
|
% Compute the width of the /Foot string, by defining P to
|
||||||
|
% add up the x-width of the characters.
|
||||||
|
/P { findfont 9 scalefont setfont stringwidth pop add } def
|
||||||
|
/FootWidth 0 Foot def
|
||||||
|
% Redefine P to print, as usual
|
||||||
|
/P { findfont 9 scalefont setfont show } def
|
||||||
|
%%BeginResource: procset foo 0 0
|
||||||
|
% This is an example
|
||||||
|
%%EndResource
|
||||||
|
%%EndSetup
|
||||||
30
tools/Makefile
Normal file
30
tools/Makefile
Normal file
@ -0,0 +1,30 @@
|
|||||||
|
all: unmunge repair munge
|
||||||
|
|
||||||
|
OPT = -g -O -W -Wall
|
||||||
|
COMMON_OBJS = util.o
|
||||||
|
|
||||||
|
UNMUNGE_OBJS = $(COMMON_OBJS) unmunge.o
|
||||||
|
MUNGE_OBJS = $(COMMON_OBJS) munge.o
|
||||||
|
REPAIR_OBJS = $(COMMON_OBJS) heap.o mempool.o subst.o repair.o
|
||||||
|
|
||||||
|
unmunge: $(UNMUNGE_OBJS)
|
||||||
|
$(CC) $(OPT) -o $@ $(UNMUNGE_OBJS)
|
||||||
|
|
||||||
|
munge: $(MUNGE_OBJS)
|
||||||
|
$(CC) $(OPT) -o $@ $(MUNGE_OBJS)
|
||||||
|
|
||||||
|
repair: $(REPAIR_OBJS)
|
||||||
|
$(CC) $(OPT) -o $@ $(REPAIR_OBJS)
|
||||||
|
|
||||||
|
.c.o:
|
||||||
|
$(CC) $(OPT) -o $@ -c $<
|
||||||
|
|
||||||
|
clean:
|
||||||
|
-rm -f *.o munge unmunge repair core *.core
|
||||||
|
|
||||||
|
unmunge.o: util.h
|
||||||
|
munge.o: util.h
|
||||||
|
repair.o: heap.h mempool.h util.h subst.h
|
||||||
|
heap.o: heap.h
|
||||||
|
mempool.o: mempool.h
|
||||||
|
subst.o: subst.h
|
||||||
68
tools/bootstrap
Normal file
68
tools/bootstrap
Normal file
@ -0,0 +1,68 @@
|
|||||||
|
#!/usr/bin/perl -s
|
||||||
|
#
|
||||||
|
# bootstrap -- Simpler version of unmunge for bootstrapping
|
||||||
|
#
|
||||||
|
# Unmunge this file using:
|
||||||
|
# perl -ne 'if (s/^ *[^-\s]\S{4,6} ?//) { s/[\244\245\267]/ /g; print; }'
|
||||||
|
#
|
||||||
|
# $Id: bootstrap,v 1.15 1997/11/14 03:52:53 mhw Exp $
|
||||||
|
|
||||||
|
sub Fatal { print STDERR @_; exit(1); }
|
||||||
|
sub Max { my ($a, $b) = @_; ($a > $b) ? $a : $b; }
|
||||||
|
sub TabSkip { $tabWidth - 1 - (length($_[0]) % $tabWidth); }
|
||||||
|
|
||||||
|
($tab,$yen,$pilc,$cdot,$tmp1,$tmp2)=("\244","\245","\266","\267","\377","\376");
|
||||||
|
$editor = $ENV{'VISUAL'} || $ENV{'EDITOR'} || 'vi';
|
||||||
|
$inFile = $ARGV[0];
|
||||||
|
doFile: {
|
||||||
|
open(IN, "<$inFile") || die;
|
||||||
|
for ($lineNum = 1; ($_ = <IN>); $lineNum++) {
|
||||||
|
s/^\s+//; s/\s+$//; # Strip leading and trailing spaces
|
||||||
|
next if (/^$/); # Ignore blank lines
|
||||||
|
($prefix, $seenCRCStr, $dummy, $_) = /^(\S{2})(\S{4})( (.*))?/;
|
||||||
|
|
||||||
|
# Correct the number of spaces after each tab
|
||||||
|
while (s/$tab( *)/$tmp1 . ($tmp2 x &Max(length($1), &TabSkip($`)))/e) {}
|
||||||
|
s/ ( +)/" " . ($cdot x length($1))/eg; # Correct center dots
|
||||||
|
s/$tmp1/$tab/g; s/$tmp2/ /g; # Restore tabs and spaces from correction
|
||||||
|
s/\s*$/\n/; # Strip trailing spaces, and add a newline
|
||||||
|
|
||||||
|
$crc = $seenCRC = 0; # Calculate CRC
|
||||||
|
for ($data = $_; $data ne ""; $data = substr($data, 1)) {
|
||||||
|
$crc ^= ord($data);
|
||||||
|
for (1..8) {
|
||||||
|
$crc = ($crc >> 1) ^ (($crc & 1) ? 0x8408 : 0);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if ($crc != hex($seenCRCStr)) { # CRC mismatch
|
||||||
|
close(IN); close(OUT);
|
||||||
|
unlink(@filesCreated);
|
||||||
|
@filesCreated = ();
|
||||||
|
@oldStat = stat($inFile);
|
||||||
|
system($editor, "+$lineNum", $inFile);
|
||||||
|
@newStat = stat($inFile);
|
||||||
|
redo doFile if ($oldStat[9] != $newStat[9]); # Check mod date
|
||||||
|
&Fatal("Line $lineNum invalid: $_");
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($prefix eq '--') { # Process header line
|
||||||
|
($code, $pageNum, $file) = /^(\S{19}) Page (\d+) of (.*)/;
|
||||||
|
$tabWidth = hex(substr($code, 11, 1));
|
||||||
|
if ($file ne $lastFile) {
|
||||||
|
print "$file\n";
|
||||||
|
&Fatal("$file: already exists\n") if (!$f && (-e $file));
|
||||||
|
close(OUT);
|
||||||
|
open(OUT, ">$file") || &Fatal("$file: $!\n");
|
||||||
|
push(@filesCreated, ($lastFile = $file));
|
||||||
|
}
|
||||||
|
} else { # Unmunge normal line
|
||||||
|
s/$tab( *)/"\t".(" " x (length($1) - &TabSkip($`)))/eg;
|
||||||
|
s/$yen\n/\f/; # Handle form feeds
|
||||||
|
s/$pilc\n//; # Handle continuation lines
|
||||||
|
s/$cdot/ /g; # Center dots -> spaces
|
||||||
|
|
||||||
|
print OUT;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
close(IN); close(OUT);
|
||||||
|
}
|
||||||
72
tools/bootstrap2
Normal file
72
tools/bootstrap2
Normal file
@ -0,0 +1,72 @@
|
|||||||
|
#!/usr/bin/perl -s
|
||||||
|
#
|
||||||
|
# bootstrap2 -- Second stage bootstrapper, a version of unmunge
|
||||||
|
#
|
||||||
|
# $Id: bootstrap2,v 1.4 1997/11/14 03:52:54 mhw Exp $
|
||||||
|
|
||||||
|
sub Cleanup { close(IN); close(OUT); unlink(@files); @files = (); }
|
||||||
|
sub Fatal { &Cleanup(); print STDERR @_; exit(1); }
|
||||||
|
sub TabSkip { $tabWidth - 1 - (length($_[0]) % $tabWidth); }
|
||||||
|
sub TabFix { my ($needed, $actual) = (&TabSkip($_[0]), length($_[1]));
|
||||||
|
$tmp1 . ($tmp2 x $needed) . (" " x ($actual - $needed)); }
|
||||||
|
sub HumanEdit { my ($file, $line, @message) = ($inFile, @_); &Cleanup();
|
||||||
|
@old = stat($file); system($editor, "+$line", $file); @new = stat($file);
|
||||||
|
redo doFile if ($old[9] != $new[9]); # Check mod date
|
||||||
|
&Fatal("Line $line, ", @message); }
|
||||||
|
|
||||||
|
($tab,$yen,$pilc,$cdot,$tmp1,$tmp2)=("\244","\245","\266","\267","\377","\376");
|
||||||
|
$editor = $ENV{'VISUAL'} || $ENV{'EDITOR'} || 'vi';
|
||||||
|
($inFile, $manifest, @rest) = @ARGV;
|
||||||
|
if ($manifest ne "") { # Read manifest file
|
||||||
|
open(MANIFEST, "<$manifest") || &Fatal("$manifest: $!\n");
|
||||||
|
while (<MANIFEST>) { $dir = $1 if /^D\s+(.*)$/;
|
||||||
|
$index[$1] = $dir . $2 if /^(\d+)\s+(.*)$/; }
|
||||||
|
}
|
||||||
|
doFile: {
|
||||||
|
$seenPCRC = $pcrc1 = 0; $lastFlags = 1; $lastFileNum = 0;
|
||||||
|
open(IN, "<$inFile") || &Fatal("$inFile: $!\n");
|
||||||
|
for ($line = 1; ($_ = <IN>); $line++) {
|
||||||
|
s/^\s+//; s/\s+$//; # Strip leading and trailing spaces
|
||||||
|
next if (/^$/); # Ignore blank lines
|
||||||
|
($prefix, $seenCRCStr, $dummy, $_) = /^(\S{2})(\S{4})( (.*))?/;
|
||||||
|
while (s/$tab( *)/&TabFix($`, $1)/eo) {} # Correct spaces after tabs
|
||||||
|
s/($tmp2| )( +)/$1 . ($cdot x length($2))/ego; # Correct center dots
|
||||||
|
s/$tmp1/$tab/go; s/$tmp2/ /go; # Restore tabs/spaces from correction
|
||||||
|
s/\s*$/\n/; # Strip trailing spaces, and add a newline
|
||||||
|
|
||||||
|
$crc = 0; $pcrc = $pcrc1; # Calculate CRCs
|
||||||
|
for ($data = $_; $data ne ""; $data = substr($data, 1)) {
|
||||||
|
$crc ^= ord($data); $pcrc1 ^= ord($data);
|
||||||
|
for (1..8) { $crc = ($crc >> 1) ^ (($crc & 1) ? 0x8408 : 0);
|
||||||
|
$pcrc1 = ($pcrc1 >> 1) ^ (($pcrc1 & 1) ? 0xedb88320 : 0); }
|
||||||
|
}
|
||||||
|
($seenPLCRC, $seenCRC) = map { hex($_) } ($prefix, $seenCRCStr);
|
||||||
|
&HumanEdit($line, "CRC failed: $_") if $crc != $seenCRC;
|
||||||
|
if ($prefix eq '--') { # Process header line
|
||||||
|
&HumanEdit($line - 1, "Page CRC failed") if $pcrc != $seenPCRC;
|
||||||
|
($humanHdr, $pageNum, $file) = /^\S{19} (Page (\d+) of (.*))/;
|
||||||
|
($vers, $flags, $seenPCRC, $tabWidth, $prodNum, $fileNum) =
|
||||||
|
map { hex($_) } /^(\S)(\S\S)(\S{8})(\S)(\S{3})(\S{4})/;
|
||||||
|
if ($fileNum != $lastFileNum) {
|
||||||
|
print STDERR "MISSING files\n" if $fileNum != $lastFileNum + 1;
|
||||||
|
&Fatal("Missing pages\n") if $pageNum != 1 || !($lastFlags & 1);
|
||||||
|
if ($manifest ne "") {
|
||||||
|
($_ = $index[$fileNum]) =~ m%([^/]*)$%;
|
||||||
|
&Fatal("Manifest mismatch\n") if ($file ne $1);
|
||||||
|
($file = $_) =~ s|/+|mkdir($`, 0777), "/"|eg; # mkdir -p
|
||||||
|
}
|
||||||
|
&Fatal("$file: already exists\n") if (!$f && (-e $file));
|
||||||
|
close(OUT); open(OUT, ">$file") || &Fatal("$file: $!\n");
|
||||||
|
push(@files, $file); print "$fileNum $file\n";
|
||||||
|
} else {
|
||||||
|
&Fatal("MISSING pages\n") if ($pageNum != $lastPageNum + 1);
|
||||||
|
}
|
||||||
|
($lastFlags,$lastFileNum,$lastPageNum) = ($flags,$fileNum,$pageNum);
|
||||||
|
$pcrc1 = 0;
|
||||||
|
} else { # Unmunge normal line
|
||||||
|
&HumanEdit($line, "CRC failed: $_") if ($pcrc1 >> 24) != $seenPLCRC;
|
||||||
|
s/$tab( *)/"\t".(" " x (length($1) - &TabSkip($`)))/ego;
|
||||||
|
s/$yen\n/\f/o; s/$pilc\n//o; s/$cdot/ /go; print OUT;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
144
tools/heap.c
Normal file
144
tools/heap.c
Normal file
@ -0,0 +1,144 @@
|
|||||||
|
/*
|
||||||
|
* heap.c -- Simple priority queue. Takes pointers to cost values
|
||||||
|
* (presumably the first field in a larger structure) and returns
|
||||||
|
* them in increasing order of cost.
|
||||||
|
*
|
||||||
|
* Copyright (C) 1997 Pretty Good Privacy, Inc.
|
||||||
|
*
|
||||||
|
* Written by Colin Plumb and Mark H. Weaver
|
||||||
|
*
|
||||||
|
* $Id: heap.c,v 1.2 1997/07/05 02:55:23 colin Exp $
|
||||||
|
*/
|
||||||
|
|
||||||
|
#include <stdio.h> /* For fprintf(stderr, "Out of memory") */
|
||||||
|
#include <stdlib.h> /* For malloc() & co. */
|
||||||
|
|
||||||
|
#include "heap.h"
|
||||||
|
|
||||||
|
#define HeapParent(i) ((i) / 2)
|
||||||
|
#define HeapLeftChild(i) ((i) * 2)
|
||||||
|
#define HeapRightChild(i) ((i) * 2 + 1)
|
||||||
|
#define HeapElem(h, i) (h)->elems[i]
|
||||||
|
#define HeapMinElem(h) HeapElem(h, 1)
|
||||||
|
#define HeapElemCost(e) (*(e))
|
||||||
|
#define HeapCost(h, i) HeapElemCost(HeapElem(h, i))
|
||||||
|
#define HeapSize(h) ((h)->numElems)
|
||||||
|
|
||||||
|
static void
|
||||||
|
SiftDown(Heap const *heap, HeapCost *e)
|
||||||
|
{
|
||||||
|
HeapIndex size = HeapSize(heap), parent = 1, child;
|
||||||
|
HeapCost cparent = HeapElemCost(e), cchild;
|
||||||
|
|
||||||
|
for (;;) {
|
||||||
|
child = 2*parent;
|
||||||
|
if (child > size)
|
||||||
|
break;
|
||||||
|
cchild = HeapCost(heap, child);
|
||||||
|
if (child < size && cchild > HeapCost(heap, child+1)) {
|
||||||
|
cchild = HeapCost(heap, child+1);
|
||||||
|
child++;
|
||||||
|
}
|
||||||
|
if (cparent <= cchild)
|
||||||
|
break; /* Stop sifting down */
|
||||||
|
HeapElem(heap, parent) = HeapElem(heap, child);
|
||||||
|
parent = child;
|
||||||
|
}
|
||||||
|
HeapElem(heap, parent) = e;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Debug tool: verify heap property */
|
||||||
|
void
|
||||||
|
HeapVerify(Heap *heap)
|
||||||
|
{
|
||||||
|
HeapIndex i;
|
||||||
|
|
||||||
|
for (i = 2; i <= HeapSize(heap); i++)
|
||||||
|
if (HeapCost(heap, i) < HeapCost(heap, HeapParent(i)))
|
||||||
|
fprintf(stderr, "DEBUG: VerifyHeap failed at elem %d\n", i);
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Remove and return the minimum cost from the heap. */
|
||||||
|
HeapCost *
|
||||||
|
HeapGetMin(Heap *heap)
|
||||||
|
{
|
||||||
|
HeapIndex lastElem = HeapSize(heap);
|
||||||
|
HeapCost *retval;
|
||||||
|
|
||||||
|
if (!lastElem)
|
||||||
|
return NULL;
|
||||||
|
retval = HeapMinElem(heap);
|
||||||
|
HeapSize(heap) = lastElem-1;
|
||||||
|
SiftDown(heap, HeapElem(heap, lastElem));
|
||||||
|
return retval;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Helper - set heap size, reallocating if needed */
|
||||||
|
static void
|
||||||
|
HeapResize(Heap *heap, HeapIndex newNumElems)
|
||||||
|
{
|
||||||
|
if (newNumElems >= heap->elemsAllocated) {
|
||||||
|
HeapIndex newAllocSize = heap->elemsAllocated * 2;
|
||||||
|
|
||||||
|
if (newAllocSize <= newNumElems)
|
||||||
|
newAllocSize = newNumElems + 1;
|
||||||
|
heap->elems = (HeapCost **)realloc((void *)heap->elems,
|
||||||
|
sizeof(*heap->elems) * newAllocSize);
|
||||||
|
if (heap->elems == NULL) {
|
||||||
|
fprintf(stderr, "Fatal error: Out of memory growing heap\n");
|
||||||
|
exit(1);
|
||||||
|
}
|
||||||
|
heap->elemsAllocated = newAllocSize;
|
||||||
|
}
|
||||||
|
heap->numElems = newNumElems;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Add an element to the heap */
|
||||||
|
void
|
||||||
|
HeapInsert(Heap *heap, HeapCost *newElem)
|
||||||
|
{
|
||||||
|
HeapIndex parent, i = ++HeapSize(heap);
|
||||||
|
HeapCost cost = HeapElemCost(newElem);
|
||||||
|
|
||||||
|
HeapResize(heap, i);
|
||||||
|
/* Sift up until parent = 0 */
|
||||||
|
while ((parent = HeapParent(i)) && HeapCost(heap, parent) > cost) {
|
||||||
|
HeapElem(heap, i) = HeapElem(heap, parent);
|
||||||
|
i = parent;
|
||||||
|
}
|
||||||
|
heap->elems[i] = newElem;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Initialize a new heap */
|
||||||
|
void
|
||||||
|
HeapInit(Heap *heap, HeapIndex initSize)
|
||||||
|
{
|
||||||
|
initSize++; /* Add one for temporary element */
|
||||||
|
if (initSize < 1)
|
||||||
|
initSize = 1;
|
||||||
|
heap->elems = (HeapCost **)malloc(initSize * sizeof(*heap->elems));
|
||||||
|
if (heap->elems == NULL) {
|
||||||
|
fprintf(stderr, "Fatal error: Out of memory creating heap\n");
|
||||||
|
exit(1);
|
||||||
|
}
|
||||||
|
heap->elemsAllocated = initSize;
|
||||||
|
heap->numElems = 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Free up a heap's resources. */
|
||||||
|
void
|
||||||
|
HeapDestroy(Heap *heap)
|
||||||
|
{
|
||||||
|
free((void *)heap->elems);
|
||||||
|
heap->elemsAllocated = 0;
|
||||||
|
heap->numElems = 0;
|
||||||
|
heap->elems = NULL;
|
||||||
|
}
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Local Variables:
|
||||||
|
* tab-width: 4
|
||||||
|
* End:
|
||||||
|
* vi: ts=4 sw=4
|
||||||
|
* vim: si
|
||||||
|
*/
|
||||||
43
tools/heap.h
Normal file
43
tools/heap.h
Normal file
@ -0,0 +1,43 @@
|
|||||||
|
/*
|
||||||
|
* heap.h -- Simple priority queue. Takes pointers to cost values
|
||||||
|
* (presumably the first field in a larger structure) and returns
|
||||||
|
* them in increasing order of cost.
|
||||||
|
*
|
||||||
|
* Copyright (C) 1997 Pretty Good Privacy, Inc.
|
||||||
|
*
|
||||||
|
* Written by Colin Plumb and Mark H. Weaver
|
||||||
|
*
|
||||||
|
* $Id: heap.h,v 1.6 1997/10/31 04:22:46 mhw Exp $
|
||||||
|
*/
|
||||||
|
|
||||||
|
#ifndef HEAP_H
|
||||||
|
#define HEAP_H 1
|
||||||
|
|
||||||
|
#include <stdio.h>
|
||||||
|
#include <stdlib.h>
|
||||||
|
#include <limits.h>
|
||||||
|
|
||||||
|
typedef int HeapCost;
|
||||||
|
#define COST_INFINITY INT_MAX
|
||||||
|
typedef unsigned HeapIndex;
|
||||||
|
|
||||||
|
typedef struct Heap {
|
||||||
|
HeapCost **elems;
|
||||||
|
HeapIndex numElems, elemsAllocated;
|
||||||
|
} Heap;
|
||||||
|
|
||||||
|
void HeapInit(Heap *heap, HeapIndex initSize);
|
||||||
|
void HeapDestroy(Heap *heap);
|
||||||
|
void HeapInsert(Heap *heap, HeapCost *newElem);
|
||||||
|
HeapCost *HeapGetMin(Heap *heap);
|
||||||
|
void HeapVerify(Heap *heap);
|
||||||
|
|
||||||
|
#endif
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Local Variables:
|
||||||
|
* tab-width: 4
|
||||||
|
* End:
|
||||||
|
* vi: ts=4 sw=4
|
||||||
|
* vim: si
|
||||||
|
*/
|
||||||
31
tools/makemanifest
Normal file
31
tools/makemanifest
Normal file
@ -0,0 +1,31 @@
|
|||||||
|
#!/usr/bin/perl
|
||||||
|
|
||||||
|
$fileNum = 0;
|
||||||
|
while(<>)
|
||||||
|
{
|
||||||
|
/^([VDTB])(\S*)\s+(.*)/ || die("Bad filelist, line $.");
|
||||||
|
($type, $options, $name) = ($1, $2, $3);
|
||||||
|
|
||||||
|
if ($type eq "D")
|
||||||
|
{
|
||||||
|
$dir = $name;
|
||||||
|
print "D $dir\n";
|
||||||
|
}
|
||||||
|
elsif ($type eq "V")
|
||||||
|
{
|
||||||
|
# Do nothing
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
$fileNum++;
|
||||||
|
$tail = $name;
|
||||||
|
$tail =~ s|^.*/||;
|
||||||
|
die("Bad filelist, line $.") if $name ne $dir . $tail;
|
||||||
|
print "$fileNum $tail\n";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#
|
||||||
|
# vi: ai ts=4
|
||||||
|
# vim: si
|
||||||
|
#
|
||||||
137
tools/mempool.c
Normal file
137
tools/mempool.c
Normal file
@ -0,0 +1,137 @@
|
|||||||
|
/*
|
||||||
|
* mempool.c - Pooled memory allocation, similar to GNU obstacks.
|
||||||
|
*
|
||||||
|
* $Id: mempool.c,v 1.5 1997/11/13 23:53:08 colin Exp $
|
||||||
|
*/
|
||||||
|
#include <assert.h>
|
||||||
|
#include <stdio.h>
|
||||||
|
#include <string.h>
|
||||||
|
#include <stdlib.h> /* For malloc() & free() */
|
||||||
|
|
||||||
|
#include "mempool.h"
|
||||||
|
|
||||||
|
/*
|
||||||
|
* The memory pool allocation functions
|
||||||
|
*
|
||||||
|
* These are based on a linked list of memory blocks, usually of uniform
|
||||||
|
* size. New memory is allocated from the tail of the current block,
|
||||||
|
* until that is inadequate, then a new block is allocated.
|
||||||
|
* The entire pool can be freed at once by calling memPoolFree().
|
||||||
|
*/
|
||||||
|
struct PoolBuf {
|
||||||
|
struct PoolBuf *next;
|
||||||
|
unsigned size;
|
||||||
|
/* Data follows */
|
||||||
|
};
|
||||||
|
|
||||||
|
/* The prototype empty pool, including the default allocation size. */
|
||||||
|
static struct MemPool EmptyPool = { 0, 0, 0, 4096, 0 , 0, 0};
|
||||||
|
|
||||||
|
/* Initialize the pool for first use */
|
||||||
|
void
|
||||||
|
memPoolInit(struct MemPool *pool)
|
||||||
|
{
|
||||||
|
*pool = EmptyPool;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Set the pool's purge function */
|
||||||
|
void
|
||||||
|
memPoolSetPurge(struct MemPool *pool, int (*purge)(void *), void *arg)
|
||||||
|
{
|
||||||
|
pool->purge = purge;
|
||||||
|
pool->purgearg = arg;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Free all the memory in the pool */
|
||||||
|
void
|
||||||
|
memPoolEmpty(struct MemPool *pool)
|
||||||
|
{
|
||||||
|
struct PoolBuf *buf;
|
||||||
|
|
||||||
|
while ((buf = pool->head) != 0) {
|
||||||
|
pool->head = buf->next;
|
||||||
|
free(buf);
|
||||||
|
}
|
||||||
|
pool->freespace = 0;
|
||||||
|
pool->totalsize = 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Restore a pool to a marked position, freeing subsequently allocated
|
||||||
|
* memory.
|
||||||
|
*/
|
||||||
|
void
|
||||||
|
memPoolCutBack(struct MemPool *pool, struct MemPool const *cutback)
|
||||||
|
{
|
||||||
|
struct PoolBuf *buf;
|
||||||
|
|
||||||
|
assert(pool);
|
||||||
|
assert(cutback);
|
||||||
|
assert(pool->totalsize >= cutback->totalsize);
|
||||||
|
|
||||||
|
while((buf = pool->head) != cutback->head) {
|
||||||
|
pool->head = buf->next;
|
||||||
|
free(buf);
|
||||||
|
}
|
||||||
|
*pool = *cutback;
|
||||||
|
}
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Allocate a chunk of memory for a structure. Alignment is assumed to be
|
||||||
|
* a power of 2. It could be generalized, if that ever becomes relevant.
|
||||||
|
* Note that alignment is from the beginning of an allocated chunk, which
|
||||||
|
* is guaranteed by ANSI to be as aligned as can possibly matter.
|
||||||
|
*/
|
||||||
|
void *
|
||||||
|
memPoolAlloc(struct MemPool *pool, unsigned len, unsigned alignment)
|
||||||
|
{
|
||||||
|
char *p;
|
||||||
|
unsigned t;
|
||||||
|
|
||||||
|
/* Where to allocate next object */
|
||||||
|
p = pool->freeptr;
|
||||||
|
/* How far it is from the beginning of the chunk. */
|
||||||
|
t = p - (char *)pool->head;
|
||||||
|
/* How much to round up freeptr to make alignment */
|
||||||
|
t = -t & --alignment;
|
||||||
|
|
||||||
|
/* Okay, does it fit? */
|
||||||
|
if (pool->freespace >= len+t) {
|
||||||
|
pool->freespace -= len+t;
|
||||||
|
p += t;
|
||||||
|
pool->freeptr = p + len;
|
||||||
|
return p;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* It does not fit in the current chunk. Go for a bigger chunk. */
|
||||||
|
|
||||||
|
/* First, figure out how much to skip at the beginning of the chunk */
|
||||||
|
alignment &= -(unsigned)sizeof(struct PoolBuf);
|
||||||
|
alignment += sizeof(struct PoolBuf);
|
||||||
|
/* Then, figure out a chunk size that will fit */
|
||||||
|
t = pool->chunksize;
|
||||||
|
assert(t);
|
||||||
|
while (len + alignment > t)
|
||||||
|
t *= 2;
|
||||||
|
while ((p = malloc(t)) == 0) {
|
||||||
|
/* If that didn't work, try purging or smaller allocations */
|
||||||
|
if (!pool->purge || !pool->purge(pool->purgearg)) {
|
||||||
|
t /= 2;
|
||||||
|
if (len + alignment > t)
|
||||||
|
fputs("Out of memory!\n", stderr);
|
||||||
|
exit (1); /* Failed */
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Update the various pointers. */
|
||||||
|
pool->totalsize += t;
|
||||||
|
((struct PoolBuf *)p)->next = pool->head;
|
||||||
|
((struct PoolBuf *)p)->size = t;
|
||||||
|
pool->head = (struct PoolBuf *)p;
|
||||||
|
pool->freespace = t - len - alignment;
|
||||||
|
p += alignment;
|
||||||
|
pool->freeptr = p + len;
|
||||||
|
|
||||||
|
return p;
|
||||||
|
}
|
||||||
36
tools/mempool.h
Normal file
36
tools/mempool.h
Normal file
@ -0,0 +1,36 @@
|
|||||||
|
/* $Id: mempool.h,v 1.2 1997/11/13 23:53:09 colin Exp $ */
|
||||||
|
|
||||||
|
#ifndef MEMPOOL_H
|
||||||
|
#define MEMPOOL_H
|
||||||
|
|
||||||
|
typedef struct MemPool {
|
||||||
|
struct PoolBuf *head;
|
||||||
|
char *freeptr;
|
||||||
|
unsigned freespace;
|
||||||
|
unsigned chunksize; /* Default starting point */
|
||||||
|
unsigned long totalsize;
|
||||||
|
int (*purge)(void *); /* Return non-zero to retry alloc */
|
||||||
|
void *purgearg;
|
||||||
|
} MemPool;
|
||||||
|
|
||||||
|
/* A global pool for miscellaneous stuff. */
|
||||||
|
extern struct MemPool MiscPool;
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Nice clean interfaces
|
||||||
|
*/
|
||||||
|
void memPoolInit(struct MemPool *pool);
|
||||||
|
void memPoolSetPurge(struct MemPool *pool, int (*purge)(void *), void *arg);
|
||||||
|
void memPoolEmpty(struct MemPool *pool);
|
||||||
|
void memPoolCutBack(struct MemPool *dest, struct MemPool const *cutback);
|
||||||
|
void *memPoolAlloc(struct MemPool *pool, unsigned len, unsigned alignment);
|
||||||
|
#ifdef DEADCODE
|
||||||
|
char const *memPoolStore(struct MemPool *pool, char const *str);
|
||||||
|
#endif
|
||||||
|
|
||||||
|
/* Lookie here! An ASNI-compliant alignment finder! */
|
||||||
|
#define alignof(type) (sizeof(struct{type _x; char _y;}) - sizeof(type))
|
||||||
|
|
||||||
|
#define memPoolNew(pool, type) memPoolAlloc(pool, sizeof(type), alignof(type))
|
||||||
|
|
||||||
|
#endif /* MEMPOOL_H */
|
||||||
543
tools/munge.c
Normal file
543
tools/munge.c
Normal file
@ -0,0 +1,543 @@
|
|||||||
|
/*
|
||||||
|
* munge.c -- Program to convert a text file into "munged" form,
|
||||||
|
* suitable for reconstruction from printed form. Tabs are
|
||||||
|
* made visible and checksums are added to each line and each
|
||||||
|
* page to protect against transcription errors.
|
||||||
|
*
|
||||||
|
* Copyright (C) 1997 Pretty Good Privacy, Inc.
|
||||||
|
*
|
||||||
|
* Designed by Colin Plumb, Mark H. Weaver, and Philip R. Zimmermann
|
||||||
|
* Written by Mark H. Weaver
|
||||||
|
*
|
||||||
|
* $Id: munge.c,v 1.32 1997/11/12 23:28:53 mhw Exp $
|
||||||
|
*/
|
||||||
|
|
||||||
|
#include <stdio.h>
|
||||||
|
#include <errno.h>
|
||||||
|
#include <string.h>
|
||||||
|
#include <ctype.h>
|
||||||
|
#include <stdlib.h>
|
||||||
|
|
||||||
|
#include "util.h"
|
||||||
|
|
||||||
|
/*
|
||||||
|
* The file is divided into pages, and the format of each page is
|
||||||
|
*
|
||||||
|
--f414 000b2dc79af40010002 Page 1 of munge.c
|
||||||
|
|
||||||
|
bc38e5 /*
|
||||||
|
40a838 * munge.c -- Program to convert a text file into munged form
|
||||||
|
647222 *
|
||||||
|
193f28 * Copyright (C) 1997 Pretty Good Privacy, Inc.
|
||||||
|
827222 *
|
||||||
|
699025 * Designed by Colin Plumb, Mark H. Weaver, and Philip R. Zimmermann
|
||||||
|
0d050c * Written by Mark H. Weaver
|
||||||
|
*
|
||||||
|
* Where the first 2 columns are the high 8 bits (in hex) of a running
|
||||||
|
* CRC-32 of the page (the string "--", unlikely to be confused with
|
||||||
|
* any digits, indicates a page header line) and the next 4 columns
|
||||||
|
* are a CRC-16 of the rest of the line. Then a space (not counted in
|
||||||
|
* the CRC), and the line of text. Tabs are printed as the currency
|
||||||
|
* symbol (ISO Latin 1 character 164) followed by the appropriate number
|
||||||
|
* of spaces, and any form feeds are printed as a yen symbol (Latin 1 165).
|
||||||
|
* The CRC is computed on the transformed line, including the trailing
|
||||||
|
* newline. No trailing whitespace is permitted.
|
||||||
|
*
|
||||||
|
* The header line contains a (hex) number of the form 0ffcccccccctpppnnnn,
|
||||||
|
* where the digit 0 is a version number, ff are flags, ccccccc is the CRC-32
|
||||||
|
* of the page, t is the tab size (usually 4 or 8; 0 for binary files that
|
||||||
|
* are sent in radix-64), ppp is the product number (usually 1, different
|
||||||
|
* for different books), and nnnn is the file number (sequential from 1).
|
||||||
|
*
|
||||||
|
* This is followed by " Page %u of " and the file name.
|
||||||
|
*/
|
||||||
|
|
||||||
|
typedef struct MungeState
|
||||||
|
{
|
||||||
|
EncodeFormat const * fmt;
|
||||||
|
EncodeFormat const * hFmt;
|
||||||
|
int binaryMode, tabWidth;
|
||||||
|
long origLineNumber;
|
||||||
|
long productNumber, fileNumber, pageNumber, lineNumber;
|
||||||
|
unsigned long fileOffset;
|
||||||
|
CRC pageCRC;
|
||||||
|
char const * fileName;
|
||||||
|
char const * fileNameTail;
|
||||||
|
char * pageBuffer; /* Buffer large enough to hold one page */
|
||||||
|
char * pagePos; /* Current position in pageBuffer */
|
||||||
|
word16 hdrFlags;
|
||||||
|
FILE * file;
|
||||||
|
FILE * out;
|
||||||
|
} MungeState;
|
||||||
|
|
||||||
|
|
||||||
|
void ChecksumLine(EncodeFormat const *fmt, char const *line, size_t length,
|
||||||
|
char *prefix, CRC *pageCRC)
|
||||||
|
{
|
||||||
|
CRC lineCRC;
|
||||||
|
CRC runCRCPart = 0;
|
||||||
|
|
||||||
|
lineCRC = CalculateCRC(fmt->lineCRC, 0, (byte const *)line, length);
|
||||||
|
if (pageCRC != NULL)
|
||||||
|
{
|
||||||
|
*pageCRC = CalculateCRC(fmt->pageCRC, *pageCRC,
|
||||||
|
(byte const *)line, length);
|
||||||
|
runCRCPart = RunningCRCFromPageCRC(fmt, *pageCRC);
|
||||||
|
}
|
||||||
|
|
||||||
|
prefix += EncodeCheckDigits(fmt, runCRCPart, fmt->runningCRCBits, prefix);
|
||||||
|
prefix += EncodeCheckDigits(fmt, lineCRC, fmt->lineCRC->bits, prefix);
|
||||||
|
|
||||||
|
*prefix++ = ' '; /* Write a space over the null byte */
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Returns 1 for convenience */
|
||||||
|
int PrintFileError(MungeState *state, char const *message)
|
||||||
|
{
|
||||||
|
fprintf(stderr, "%s in %s %s %lu\n", message, state->fileName,
|
||||||
|
state->binaryMode ? "offset" : "line",
|
||||||
|
state->binaryMode ? state->fileOffset : state->origLineNumber);
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
int MungeLine(MungeState *state, char *buffer, int length,
|
||||||
|
char *line, int *bufferUsed)
|
||||||
|
{
|
||||||
|
int i = 0, j = 0, jOld = 0;
|
||||||
|
char ch;
|
||||||
|
|
||||||
|
for (i = 0; i < length && j < LINE_LENGTH; i++)
|
||||||
|
{
|
||||||
|
jOld = j;
|
||||||
|
ch = buffer[i];
|
||||||
|
if (ch == '\t')
|
||||||
|
{
|
||||||
|
line[j++] = TAB_CHAR;
|
||||||
|
if (state->tabWidth < 1)
|
||||||
|
return PrintFileError(state,
|
||||||
|
"ERROR: Tab found in radix64 stream");
|
||||||
|
else
|
||||||
|
while (j % state->tabWidth && j < LINE_LENGTH)
|
||||||
|
line[j++] = TAB_PAD_CHAR;
|
||||||
|
}
|
||||||
|
else if (ch == '\n')
|
||||||
|
{
|
||||||
|
if (i + 1 < length)
|
||||||
|
return PrintFileError(state,
|
||||||
|
"UNEXPECTED ERROR: fgets read past newline!?");
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
else if (ch == '\f')
|
||||||
|
{
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
else if (ch == ' ' && (j <= 0 || line[j-1] == ' ' ||
|
||||||
|
line[j-1] == SPACE_CHAR ||
|
||||||
|
i+1 >= length || buffer[i+1] == '\n'))
|
||||||
|
{
|
||||||
|
line[j++] = SPACE_CHAR;
|
||||||
|
}
|
||||||
|
else if (ch >= ' ' && ch <= '~')
|
||||||
|
line[j++] = ch;
|
||||||
|
else
|
||||||
|
return PrintFileError(state, "ERROR: Non-ASCII char");
|
||||||
|
}
|
||||||
|
|
||||||
|
if (i < length && buffer[i] == '\n')
|
||||||
|
{
|
||||||
|
i++;
|
||||||
|
state->origLineNumber++;
|
||||||
|
}
|
||||||
|
else if (i < length && buffer[i] == '\f' && j < LINE_LENGTH)
|
||||||
|
{
|
||||||
|
i++;
|
||||||
|
line[j++] = FORMFEED_CHAR;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
/* If there's no newline, we need to add the continuation marker */
|
||||||
|
if (i > 0 && j >= LINE_LENGTH)
|
||||||
|
{
|
||||||
|
/* Remove the last character if we're out of room */
|
||||||
|
i--;
|
||||||
|
j = jOld;
|
||||||
|
}
|
||||||
|
line[j++] = CONTIN_CHAR;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Strip trailing spaces */
|
||||||
|
while (j > 0 && isspace((unsigned char)line[j - 1]))
|
||||||
|
j--;
|
||||||
|
|
||||||
|
if (j > LINE_LENGTH) /* This should never happen */
|
||||||
|
return PrintFileError(state, "ERROR: Internal error, line too long");
|
||||||
|
|
||||||
|
/* Add trailing newline and NULL */
|
||||||
|
line[j++] = '\n';
|
||||||
|
line[j++] = '\0';
|
||||||
|
|
||||||
|
/* Return number of chars used from buffer */
|
||||||
|
*bufferUsed = i;
|
||||||
|
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
static void
|
||||||
|
Encode3(byte const src[3], char dest[4])
|
||||||
|
{
|
||||||
|
dest[0] = radix64Digits[ (src[0]>>2 & 0x3f)];
|
||||||
|
dest[1] = radix64Digits[(src[0]<<4 & 0x30) | (src[1]>>4 & 0x0f)];
|
||||||
|
dest[2] = radix64Digits[(src[1]<<2 & 0x3c) | (src[2]>>6 & 0x03)];
|
||||||
|
dest[3] = radix64Digits[(src[2] & 0x3f)];
|
||||||
|
}
|
||||||
|
|
||||||
|
static int
|
||||||
|
EncodeLine(byte const *src, int srcLen, char *dest)
|
||||||
|
{
|
||||||
|
char * destp = dest;
|
||||||
|
byte tempSrc[3];
|
||||||
|
|
||||||
|
for (; srcLen >= 3; srcLen -= 3)
|
||||||
|
{
|
||||||
|
Encode3(src, destp);
|
||||||
|
src += 3; destp += 4;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (srcLen > 0)
|
||||||
|
{
|
||||||
|
memset(tempSrc, 0, sizeof(tempSrc));
|
||||||
|
memcpy(tempSrc, src, srcLen);
|
||||||
|
Encode3(src, destp);
|
||||||
|
src += 3; destp += 4; srcLen -= 3;
|
||||||
|
while (srcLen < 0)
|
||||||
|
destp[srcLen++] = RADIX64_END_CHAR;
|
||||||
|
}
|
||||||
|
|
||||||
|
return destp - dest;
|
||||||
|
}
|
||||||
|
|
||||||
|
static int
|
||||||
|
MungeBinaryLine(MungeState *state, byte const *buffer, int length, char *line)
|
||||||
|
{
|
||||||
|
char binLine[128];
|
||||||
|
int binLength; /* Destination length */
|
||||||
|
int used;
|
||||||
|
|
||||||
|
binLength = EncodeLine(buffer, length, binLine);
|
||||||
|
|
||||||
|
/* Append newline */
|
||||||
|
binLine[binLength++] = '\n';
|
||||||
|
binLine[binLength] = '\0';
|
||||||
|
|
||||||
|
return MungeLine(state, binLine, binLength, line, &used);
|
||||||
|
}
|
||||||
|
|
||||||
|
int MaybePageBreak(MungeState *state)
|
||||||
|
{
|
||||||
|
EncodeFormat const * fmt = state->fmt;
|
||||||
|
EncodeFormat const * hFmt = state->hFmt;
|
||||||
|
|
||||||
|
if (state->lineNumber >= LINES_PER_PAGE)
|
||||||
|
{
|
||||||
|
char line[512];
|
||||||
|
char * lineData = line + PREFIX_LENGTH;
|
||||||
|
char * p = lineData;
|
||||||
|
|
||||||
|
p += EncodeCheckDigits(hFmt, 0, HDR_VERSION_BITS, p);
|
||||||
|
p += EncodeCheckDigits(hFmt, state->hdrFlags, HDR_FLAG_BITS, p);
|
||||||
|
p += EncodeCheckDigits(hFmt, state->pageCRC, fmt->pageCRC->bits, p);
|
||||||
|
p += EncodeCheckDigits(hFmt, state->tabWidth, HDR_TABWIDTH_BITS, p);
|
||||||
|
p += EncodeCheckDigits(hFmt, state->productNumber, HDR_PRODNUM_BITS, p);
|
||||||
|
p += EncodeCheckDigits(hFmt, state->fileNumber, HDR_FILENUM_BITS, p);
|
||||||
|
|
||||||
|
sprintf(p, " Page %ld of %s\n", state->pageNumber + 1,
|
||||||
|
state->fileNameTail);
|
||||||
|
|
||||||
|
if (strlen(lineData) > LINE_LENGTH + 1)
|
||||||
|
{
|
||||||
|
PrintFileError(state, "ERROR: Header line too long");
|
||||||
|
fprintf(stderr, "> %s", lineData);
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Compute checksums and prefix them to line */
|
||||||
|
ChecksumLine(fmt, lineData, strlen(lineData), line, NULL);
|
||||||
|
|
||||||
|
fprintf(state->out, "%c%c%s\n%s\f", HDR_PREFIX_CHAR,
|
||||||
|
fmt->headerTypeChar, line + 2, state->pageBuffer);
|
||||||
|
|
||||||
|
state->pageNumber++;
|
||||||
|
state->lineNumber = 0;
|
||||||
|
state->pageCRC = 0;
|
||||||
|
state->pagePos = state->pageBuffer; /* Clear page buffer */
|
||||||
|
}
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Search for Emacs "tab-width: " maker in file.
|
||||||
|
* Emacs is stricter about the format, but this will do.
|
||||||
|
*/
|
||||||
|
int FindTabWidth(MungeState *state)
|
||||||
|
{
|
||||||
|
char const * const tabWidthMarker = " tab-width: ";
|
||||||
|
char buffer[512];
|
||||||
|
char * p;
|
||||||
|
int length;
|
||||||
|
int tabWidth = 0;
|
||||||
|
|
||||||
|
fseek(state->file, -(sizeof(buffer) - 1), SEEK_END);
|
||||||
|
length = fread(buffer, 1, sizeof(buffer) - 1, state->file);
|
||||||
|
buffer[length] = '\0';
|
||||||
|
p = strstr(buffer, tabWidthMarker);
|
||||||
|
if (p != NULL)
|
||||||
|
{
|
||||||
|
p += strlen(tabWidthMarker);
|
||||||
|
while (*p != '\0' && *p != '\n' && isspace(*p))
|
||||||
|
p++;
|
||||||
|
tabWidth = strtol(p, &p, 10);
|
||||||
|
while (*p != '\0' && *p != '\n' && isspace(*p))
|
||||||
|
p++;
|
||||||
|
if (*p != '\n' || tabWidth < 2)
|
||||||
|
tabWidth = 0;
|
||||||
|
else if (tabWidth > 16)
|
||||||
|
fprintf(stderr, "WARNING: Weird tab-width (%d), %s\n",
|
||||||
|
tabWidth, state->fileName);
|
||||||
|
}
|
||||||
|
return tabWidth;
|
||||||
|
}
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Open the given source file and send the munged output to the
|
||||||
|
* FILE *, with the given options.
|
||||||
|
*/
|
||||||
|
int MungeFile(char const *fileName, FILE *out, EncodeFormat const *fmt,
|
||||||
|
int binaryMode, int defaultTabWidth,
|
||||||
|
long productNumber, long fileNumber)
|
||||||
|
{
|
||||||
|
MungeState * state;
|
||||||
|
int length, used;
|
||||||
|
char line[PREFIX_LENGTH + LINE_LENGTH + 10];
|
||||||
|
char * lineData = line + PREFIX_LENGTH;
|
||||||
|
char buffer[128];
|
||||||
|
int result = 0;
|
||||||
|
|
||||||
|
state = (MungeState *)calloc(1, sizeof(*state));
|
||||||
|
state->fmt = fmt;
|
||||||
|
state->hFmt = &hexFormat;
|
||||||
|
state->origLineNumber = 1;
|
||||||
|
state->fileName = fileName;
|
||||||
|
state->pageCRC = 0;
|
||||||
|
state->productNumber = productNumber;
|
||||||
|
state->fileNumber = fileNumber;
|
||||||
|
state->pageNumber = 0;
|
||||||
|
state->lineNumber = 0;
|
||||||
|
state->fileOffset = 0;
|
||||||
|
state->binaryMode = binaryMode;
|
||||||
|
state->pageBuffer = malloc(PAGE_BUFFER_SIZE);
|
||||||
|
state->pageBuffer[0] = '\0';
|
||||||
|
state->pagePos = state->pageBuffer;
|
||||||
|
state->hdrFlags = 0;
|
||||||
|
state->out = out;
|
||||||
|
|
||||||
|
state->fileNameTail = strrchr(state->fileName, '/');
|
||||||
|
if (state->fileNameTail == NULL)
|
||||||
|
state->fileNameTail = state->fileName;
|
||||||
|
else
|
||||||
|
state->fileNameTail++;
|
||||||
|
|
||||||
|
state->file = fopen(state->fileName, binaryMode ? "rb" : "r");
|
||||||
|
if (state->file == NULL)
|
||||||
|
{
|
||||||
|
result = errno;
|
||||||
|
fprintf(stderr, "ERROR opening %s: %s\n",
|
||||||
|
state->fileName, strerror(result));
|
||||||
|
goto error;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (state->binaryMode)
|
||||||
|
{
|
||||||
|
state->tabWidth = 0;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
state->tabWidth = FindTabWidth(state);
|
||||||
|
if (state->tabWidth == 0)
|
||||||
|
state->tabWidth = defaultTabWidth;
|
||||||
|
rewind(state->file);
|
||||||
|
}
|
||||||
|
|
||||||
|
while (!feof(state->file))
|
||||||
|
{
|
||||||
|
if (state->binaryMode)
|
||||||
|
{
|
||||||
|
length = fread(buffer, 1, BYTES_PER_LINE, state->file);
|
||||||
|
if (length < 1)
|
||||||
|
{
|
||||||
|
if (feof(state->file))
|
||||||
|
break;
|
||||||
|
goto fileError;
|
||||||
|
}
|
||||||
|
if ((result = MaybePageBreak(state)))
|
||||||
|
goto error;
|
||||||
|
if ((result = MungeBinaryLine(state, buffer, length, lineData)))
|
||||||
|
goto error;
|
||||||
|
state->fileOffset += length;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
if (fgets(buffer, sizeof(buffer), state->file) == NULL)
|
||||||
|
{
|
||||||
|
if (feof(state->file))
|
||||||
|
break;
|
||||||
|
goto fileError;
|
||||||
|
}
|
||||||
|
length = strlen(buffer);
|
||||||
|
if ((result = MaybePageBreak(state)))
|
||||||
|
goto error;
|
||||||
|
if ((result = MungeLine(state, buffer, length, lineData, &used)))
|
||||||
|
goto error;
|
||||||
|
|
||||||
|
if (used < length)
|
||||||
|
if (fseek(state->file, used - length, SEEK_CUR))
|
||||||
|
goto fileError;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Compute checksums and prefix them to the line */
|
||||||
|
ChecksumLine(fmt, lineData, strlen(lineData), line, &state->pageCRC);
|
||||||
|
|
||||||
|
strcpy(state->pagePos, line);
|
||||||
|
length = strlen(state->pagePos);
|
||||||
|
/* Suppress trailing whitespace on blank lines */
|
||||||
|
if (length == PREFIX_LENGTH+1 && state->pagePos[length-1] == '\n') {
|
||||||
|
state->pagePos[--length-1] = '\n';
|
||||||
|
state->pagePos[length] = '\0';
|
||||||
|
}
|
||||||
|
state->pagePos += length;
|
||||||
|
|
||||||
|
state->lineNumber++;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (state->lineNumber > 0)
|
||||||
|
{
|
||||||
|
/* Force a final page break */
|
||||||
|
state->lineNumber = LINES_PER_PAGE;
|
||||||
|
state->hdrFlags |= HDR_FLAG_LASTPAGE;
|
||||||
|
if ((result = MaybePageBreak(state)))
|
||||||
|
goto error;
|
||||||
|
}
|
||||||
|
|
||||||
|
result = 0;
|
||||||
|
goto done;
|
||||||
|
|
||||||
|
fileError:
|
||||||
|
result = ferror(state->file);
|
||||||
|
|
||||||
|
error:
|
||||||
|
done:
|
||||||
|
if (state != NULL)
|
||||||
|
{
|
||||||
|
if (state->file != NULL)
|
||||||
|
fclose(state->file);
|
||||||
|
free(state);
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
int main(int argc, char *argv[])
|
||||||
|
{
|
||||||
|
int result = 0;
|
||||||
|
int i, j;
|
||||||
|
int defaultTabWidth = 4;
|
||||||
|
int binaryMode = 0;
|
||||||
|
long productNumber = 1;
|
||||||
|
long fileNumber = 1;
|
||||||
|
char * endOfNumber;
|
||||||
|
EncodeFormat const * fmt = NULL;
|
||||||
|
|
||||||
|
InitUtil();
|
||||||
|
|
||||||
|
for (i = 1; i < argc && argv[i][0] == '-'; i++)
|
||||||
|
{
|
||||||
|
if (0 == strcmp(argv[i], "--"))
|
||||||
|
{
|
||||||
|
i++;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
for (j = 1; argv[i][j] != '\0'; j++)
|
||||||
|
{
|
||||||
|
if (isdigit(argv[i][j]))
|
||||||
|
{
|
||||||
|
defaultTabWidth = argv[i][j] - '0';
|
||||||
|
if (defaultTabWidth < 2 || defaultTabWidth > 9)
|
||||||
|
fprintf(stderr, "WARNING: Weird default tab-width (%d)\n",
|
||||||
|
defaultTabWidth);
|
||||||
|
}
|
||||||
|
else if (argv[i][j] == 'b')
|
||||||
|
{
|
||||||
|
binaryMode = 1;
|
||||||
|
}
|
||||||
|
else if (argv[i][j] == 'F')
|
||||||
|
{
|
||||||
|
fmt = FindFormat(argv[i][j+1]);
|
||||||
|
if (!fmt || argv[i][j+2] != '\0')
|
||||||
|
{
|
||||||
|
fprintf(stderr, "ERROR: Invalid format char\n");
|
||||||
|
exit(1);
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
else if (argv[i][j] == 'p')
|
||||||
|
{
|
||||||
|
productNumber = strtol(&argv[i][j+1], &endOfNumber, 10);
|
||||||
|
if (*endOfNumber != '\0')
|
||||||
|
{
|
||||||
|
fprintf(stderr, "ERROR: Invalid product number\n");
|
||||||
|
exit(1);
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
else if (argv[i][j] == 'f')
|
||||||
|
{
|
||||||
|
fileNumber = strtol(&argv[i][j+1], &endOfNumber, 10);
|
||||||
|
if (*endOfNumber != '\0')
|
||||||
|
{
|
||||||
|
fprintf(stderr, "ERROR: Invalid file number\n");
|
||||||
|
exit(1);
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
fprintf(stderr, "ERROR: Unrecognized option -%c\n", argv[i][j]);
|
||||||
|
exit(1);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (!fmt)
|
||||||
|
fmt = binaryMode ? &radix64Format : &hexFormat;
|
||||||
|
|
||||||
|
for (; i < argc; i++)
|
||||||
|
{
|
||||||
|
if ((result = MungeFile(argv[i], stdout, fmt, binaryMode,
|
||||||
|
defaultTabWidth, productNumber,
|
||||||
|
fileNumber)) != 0)
|
||||||
|
{
|
||||||
|
/* If result > 0, message should have already been printed */
|
||||||
|
if (result < 0)
|
||||||
|
fprintf(stderr, "ERROR: %s\n", strerror(result));
|
||||||
|
exit(1);
|
||||||
|
}
|
||||||
|
fileNumber++;
|
||||||
|
}
|
||||||
|
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Local Variables:
|
||||||
|
* tab-width: 4
|
||||||
|
* End:
|
||||||
|
* vi: ts=4 sw=4
|
||||||
|
* vim: si
|
||||||
|
*/
|
||||||
324
tools/psgen
Normal file
324
tools/psgen
Normal file
@ -0,0 +1,324 @@
|
|||||||
|
#!/usr/bin/perl
|
||||||
|
#
|
||||||
|
# psgen -- Postscript generator for code portion of source books
|
||||||
|
#
|
||||||
|
# Reads in a list of files/dirs from <filelist>, runs munge on each of
|
||||||
|
# them, and generates a single postscript file to stdout. The page numbers
|
||||||
|
# for each file/dir are put into the file <pagenums>.
|
||||||
|
#
|
||||||
|
# usage: psgen [ options... ] <filelist> <pagenums> <volume #> > foo.ps
|
||||||
|
# -l<firstLogicalPage>
|
||||||
|
# -p<firstPhysicalPage>
|
||||||
|
# -f<font>
|
||||||
|
# -D<defs> (passed to yapp)
|
||||||
|
# -P<productNumber>
|
||||||
|
# -o<mungedOutFile>
|
||||||
|
# -e (auto edit errors)
|
||||||
|
#
|
||||||
|
# $Id: psgen,v 1.18 1997/11/13 21:44:16 colin Exp $
|
||||||
|
#
|
||||||
|
|
||||||
|
$bookRoot = $ENV{"BOOKROOT"} || ".";
|
||||||
|
$toolsDir = "$bookRoot/tools";
|
||||||
|
$psDir = "$bookRoot/ps";
|
||||||
|
$editor = $ENV{"EDITOR"} || "vi";
|
||||||
|
|
||||||
|
# Configuration settings - external file names
|
||||||
|
$mungeProg = "$toolsDir/munge";
|
||||||
|
$yappProg = "$toolsDir/yapp";
|
||||||
|
$preambleFile = "$psDir/prolog.ps";
|
||||||
|
$tempFile = "/tmp/psgen-$$";
|
||||||
|
|
||||||
|
# Parse arguments
|
||||||
|
$firstLogPage = $firstPhysPage = 0;
|
||||||
|
$productNumber = 1;
|
||||||
|
$font = "OCRB";
|
||||||
|
$autoEdit = 0;
|
||||||
|
while ($#ARGV >= 0 && $ARGV[0] =~ /^-/)
|
||||||
|
{
|
||||||
|
$_ = shift @ARGV;
|
||||||
|
if (/^--$/)
|
||||||
|
{
|
||||||
|
last;
|
||||||
|
}
|
||||||
|
elsif (/^-l(\d+)$/)
|
||||||
|
{
|
||||||
|
$firstLogPage = $1;
|
||||||
|
}
|
||||||
|
elsif (/^-p(\d+)$/)
|
||||||
|
{
|
||||||
|
$firstPhysPage = $1;
|
||||||
|
}
|
||||||
|
elsif (/^-f(.+)$/)
|
||||||
|
{
|
||||||
|
$font = $1;
|
||||||
|
}
|
||||||
|
elsif (/^-D(.+)$/)
|
||||||
|
{
|
||||||
|
$yappDefs .= " " . $_;
|
||||||
|
}
|
||||||
|
elsif (/^-P(\d+)$/)
|
||||||
|
{
|
||||||
|
$productNumber = $1;
|
||||||
|
}
|
||||||
|
elsif (/^-o(.+)$/)
|
||||||
|
{
|
||||||
|
$mungedOutFile = $1;
|
||||||
|
}
|
||||||
|
elsif (/^-e$/)
|
||||||
|
{
|
||||||
|
$autoEdit = 1;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
&Error("Unrecognized option: '$_'");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
$fileListFile = shift @ARGV || die "Missing file list argument (arg 1)";
|
||||||
|
$pageNumFile = shift @ARGV || die "Missing page number file argument (arg 2)";
|
||||||
|
$volume = shift @ARGV || die "Missing volume number argument (arg 3)";
|
||||||
|
|
||||||
|
# Determine initial page numbers
|
||||||
|
{
|
||||||
|
my $nextLogPage = 1;
|
||||||
|
my $nextPhysPage = 3;
|
||||||
|
my $volNum = 0; # Which volume's page numbers we're reading
|
||||||
|
|
||||||
|
if ($volume > 1)
|
||||||
|
{
|
||||||
|
open(OLDPAGENUMS, "<$pageNumFile") || die;
|
||||||
|
while (<OLDPAGENUMS>)
|
||||||
|
{
|
||||||
|
if (/^Volume\s+(\d+)$/)
|
||||||
|
{
|
||||||
|
$volNum = $1;
|
||||||
|
}
|
||||||
|
elsif (/^Next:\s+(\d+)\s*$/ && $volNum == $volume - 1)
|
||||||
|
{
|
||||||
|
$nextLogPage = $1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
close(OLDPAGENUMS);
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
unlink($pageNumFile);
|
||||||
|
}
|
||||||
|
$firstLogPage = $nextLogPage if ($firstLogPage == 0);
|
||||||
|
$firstPhysPage = $nextPhysPage if ($firstPhysPage == 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
# Names of PostScript operators invoked. These are the interface
|
||||||
|
# between this file and the $preambleFile.
|
||||||
|
$oddPageStartPS = "OddPageStart";
|
||||||
|
$evenPageStartPS = "EvenPageStart";
|
||||||
|
$oddPageEndPS = "OddPageEnd";
|
||||||
|
$evenPageEndPS = "EvenPageEnd";
|
||||||
|
$dirPagePS = "DirPage";
|
||||||
|
# This is short because it's emitted every line
|
||||||
|
$linePS = "L";
|
||||||
|
|
||||||
|
# Handle an error from munge.
|
||||||
|
# A result of 0 means to retry, 1 means to exit
|
||||||
|
sub MungeError
|
||||||
|
{
|
||||||
|
my $result = 1;
|
||||||
|
|
||||||
|
open(FILEH, "<$tempFile") || die;
|
||||||
|
while (<FILEH>)
|
||||||
|
{
|
||||||
|
print STDERR;
|
||||||
|
if (/ in (.*) line (\d+)$/)
|
||||||
|
{
|
||||||
|
my ($fileName, $lineNumber) = ($1, $2);
|
||||||
|
|
||||||
|
if ($autoEdit)
|
||||||
|
{
|
||||||
|
my @statResult = stat($fileName);
|
||||||
|
my $oldMTime = $statResult[9];
|
||||||
|
|
||||||
|
system("'$editor' '+$lineNumber' '$fileName' 1>&2");
|
||||||
|
@statResult = stat($fileName);
|
||||||
|
$result = ($statResult[9] == $oldMTime);
|
||||||
|
last;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
close(FILEH);
|
||||||
|
unlink($tempFile) || die "Couldn't unlink $tempFile";
|
||||||
|
return $result;
|
||||||
|
}
|
||||||
|
|
||||||
|
sub CopyFileToPS
|
||||||
|
{
|
||||||
|
local $fileName = $_[0];
|
||||||
|
local $args = "'-I$psDir' '-Dfont=$font'";
|
||||||
|
local $_;
|
||||||
|
|
||||||
|
$args .= $yappDefs;
|
||||||
|
open(FILEH, "$yappProg $args '$fileName' |") || die;
|
||||||
|
while (<FILEH>)
|
||||||
|
{
|
||||||
|
print PSOUT $_;
|
||||||
|
}
|
||||||
|
close(FILEH) || exit(1);
|
||||||
|
1;
|
||||||
|
}
|
||||||
|
|
||||||
|
# Wrap a string in parens as required by PostScript, with proper quoting.
|
||||||
|
sub StringPS
|
||||||
|
{
|
||||||
|
local $str = $_[0];
|
||||||
|
|
||||||
|
$str =~ s/([\\()])/\\$1/g;
|
||||||
|
"(" . $str . ")";
|
||||||
|
}
|
||||||
|
|
||||||
|
# Emit a start of page. The Postscript DSC %%Page: header
|
||||||
|
# (followed by logical page number, then physical) and
|
||||||
|
# the top-of-page function (which is passed the page number as a string)
|
||||||
|
sub PageStartPS
|
||||||
|
{
|
||||||
|
local $pageNum = $_[0];
|
||||||
|
|
||||||
|
"%%Page: " . ($pageNum + $firstLogPage) . " " .
|
||||||
|
($pageNum + $firstPhysPage) . "\n" .
|
||||||
|
&StringPS($pageNum + $firstLogPage) .
|
||||||
|
((($pageNum + $firstLogPage) % 2) ? $oddPageStartPS
|
||||||
|
: $evenPageStartPS) . "\n";
|
||||||
|
}
|
||||||
|
|
||||||
|
sub PageEndPS
|
||||||
|
{
|
||||||
|
local $pageNum = $_[0];
|
||||||
|
|
||||||
|
((($pageNum + $firstLogPage) % 2) ? $oddPageEndPS : $evenPageEndPS) . "\n";
|
||||||
|
}
|
||||||
|
|
||||||
|
# Save the page number to a table-of-contents file
|
||||||
|
sub SavePageNum
|
||||||
|
{
|
||||||
|
local ($fileName, $pageNum) = @_;
|
||||||
|
|
||||||
|
print PAGENUMS ($pageNum + $firstLogPage), ": $fileName\n";
|
||||||
|
}
|
||||||
|
|
||||||
|
# The main code.
|
||||||
|
|
||||||
|
open(PSOUT, ">-") || die;
|
||||||
|
open(FILELIST, "<$fileListFile") || die;
|
||||||
|
open(PAGENUMS, ">>$pageNumFile") || die;
|
||||||
|
if ($mungedOutFile ne "")
|
||||||
|
{
|
||||||
|
open(MUNGEDOUT, ">$mungedOutFile") || die;
|
||||||
|
}
|
||||||
|
|
||||||
|
print PAGENUMS "Volume $volume\n";
|
||||||
|
|
||||||
|
&CopyFileToPS($preambleFile);
|
||||||
|
|
||||||
|
$fileNumber = 0;
|
||||||
|
$pageNum = 0; # This is 0-based, since it is added to $first{Log,Phys}Page
|
||||||
|
$enable = 0;
|
||||||
|
|
||||||
|
while (<FILELIST>)
|
||||||
|
{
|
||||||
|
/^([VDTB])(\S*)\s+(.*)/ || die "Illegal file list line $.";
|
||||||
|
|
||||||
|
local ($fileType, $options, $arg) = ($1, $2, $3);
|
||||||
|
|
||||||
|
if ($fileType eq "V")
|
||||||
|
{
|
||||||
|
@args = split(/\s+/, $arg);
|
||||||
|
if ($enable = ($args[0] == $volume))
|
||||||
|
{
|
||||||
|
$defaultTabWidth = int($args[1]);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
elsif ($fileType eq "D")
|
||||||
|
{
|
||||||
|
next unless $enable; # Do nothing if we're in the wrong volume
|
||||||
|
$dirName = $arg;
|
||||||
|
&SavePageNum($dirName, $pageNum);
|
||||||
|
print PSOUT &PageStartPS($pageNum);
|
||||||
|
print PSOUT &StringPS($dirName), $dirPagePS, "\n";
|
||||||
|
print PSOUT &PageEndPS($pageNum);
|
||||||
|
$pageNum++;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
my $done = 0;
|
||||||
|
|
||||||
|
$fileNumber++;
|
||||||
|
$fileName = $arg;
|
||||||
|
next unless $enable; # Do nothing if we're in the wrong volume
|
||||||
|
&SavePageNum($fileName, $pageNum);
|
||||||
|
$quotedFileName = $fileName;
|
||||||
|
$quotedFileName =~ s/'/\\'/g;
|
||||||
|
$tabWidth = ($options =~ /(\d)/) ? $1 : $defaultTabWidth;
|
||||||
|
$args = ($fileType eq "B") ? "-b" : "";
|
||||||
|
$args .= " -$tabWidth -p$productNumber -f$fileNumber";
|
||||||
|
while (!$done)
|
||||||
|
{
|
||||||
|
if (open(FILE, "$mungeProg $args '$quotedFileName' 2>$tempFile |"))
|
||||||
|
{
|
||||||
|
$line = <FILE>;
|
||||||
|
print MUNGEDOUT $line;
|
||||||
|
|
||||||
|
while ($line ne "")
|
||||||
|
{
|
||||||
|
print PSOUT &PageStartPS($pageNum);
|
||||||
|
|
||||||
|
while ($line ne "" and $line !~ /^\f/)
|
||||||
|
{
|
||||||
|
chop $line;
|
||||||
|
print PSOUT &StringPS($line), $linePS, "\n";
|
||||||
|
$line = <FILE>;
|
||||||
|
print MUNGEDOUT $line;
|
||||||
|
}
|
||||||
|
$line =~ s/^\f//;
|
||||||
|
|
||||||
|
print PSOUT &PageEndPS($pageNum);
|
||||||
|
$pageNum++;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (close(FILE))
|
||||||
|
{
|
||||||
|
$done = 2;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
$done = &MungeError();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
$done = &MungeError();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if ($done == 1)
|
||||||
|
{
|
||||||
|
die;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Print PostScript DSC trailer with the correct number of pages
|
||||||
|
print PSOUT "%%Trailer\n%%Pages: ", $pageNum, "\n%%EOF\n";
|
||||||
|
|
||||||
|
print PAGENUMS "Pages: ", $pageNum, "\n";
|
||||||
|
print PAGENUMS "Next: ", ((($pageNum+1) & ~1) + $firstLogPage), "\n";
|
||||||
|
|
||||||
|
close(PAGENUMS) || die;
|
||||||
|
close(FILELIST) || die;
|
||||||
|
close(PSOUT) || die;
|
||||||
|
|
||||||
|
if ($mungedOutFile ne "")
|
||||||
|
{
|
||||||
|
close(MUNGEDOUT) || die;
|
||||||
|
}
|
||||||
|
|
||||||
|
#
|
||||||
|
# vi: ai ts=4
|
||||||
|
# vim: si
|
||||||
|
#
|
||||||
1851
tools/repair.c
Normal file
1851
tools/repair.c
Normal file
File diff suppressed because it is too large
Load Diff
185
tools/sortpages
Normal file
185
tools/sortpages
Normal file
@ -0,0 +1,185 @@
|
|||||||
|
#!/usr/bin/perl
|
||||||
|
#
|
||||||
|
# $Id: sortpages,v 1.8 1997/12/11 19:20:58 mhw Exp $
|
||||||
|
#
|
||||||
|
|
||||||
|
@fileNameFromNumber = ();
|
||||||
|
@pagesFound = ();
|
||||||
|
$theProductNumber = 0;
|
||||||
|
|
||||||
|
for $fileIndex (0..$#ARGV)
|
||||||
|
{
|
||||||
|
$fileName = $ARGV[$fileIndex];
|
||||||
|
open(FILE, "<$fileName") || die;
|
||||||
|
while (!eof(FILE))
|
||||||
|
{
|
||||||
|
$filePos = tell(FILE);
|
||||||
|
$_ = <FILE>;
|
||||||
|
if (/^\f?-\S/)
|
||||||
|
{
|
||||||
|
my ($versionHex, $flagsHex, $pageCRCHex, $tabWidthHex,
|
||||||
|
$productNumberHex, $fileNumberHex, $pageNumber, $name)
|
||||||
|
= (/^\f?-\S\S{4}\ # CRC followed by a space
|
||||||
|
([0-9a-f]) # Format version
|
||||||
|
([0-9a-f]{2}) # Flags
|
||||||
|
([0-9a-f]{8}) # Running CRC32
|
||||||
|
([0-9a-f]) # Tab width (0 means radix64)
|
||||||
|
([0-9a-f]{3}) # Product number
|
||||||
|
([0-9a-f]{4}) # File number
|
||||||
|
\ Page\ (\d+)\ of\ (.*)/x);
|
||||||
|
my $version = hex($versionHex);
|
||||||
|
my $flags = hex($flagsHex);
|
||||||
|
my $productNumber = hex($productNumberHex);
|
||||||
|
my $fileNumber = hex($fileNumberHex);
|
||||||
|
|
||||||
|
unless ($version == 0 && $productNumber > 0
|
||||||
|
&& $fileNumber > 0 && $pageNumber > 0
|
||||||
|
&& $name ne "")
|
||||||
|
{
|
||||||
|
print STDERR "ERROR: Invalid header info ",
|
||||||
|
"at $fileName line $.\n";
|
||||||
|
exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!defined($fileNameFromNumber[$fileNumber]))
|
||||||
|
{
|
||||||
|
$fileNameFromNumber[$fileNumber] = $name;
|
||||||
|
}
|
||||||
|
elsif ($fileNameFromNumber[$fileNumber] ne $name)
|
||||||
|
{
|
||||||
|
print STDERR "ERROR: Mismatched filename ",
|
||||||
|
"at $fileName line $.\n";
|
||||||
|
exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!$theProductNumber)
|
||||||
|
{
|
||||||
|
$theProductNumber = $productNumber;
|
||||||
|
}
|
||||||
|
elsif ($theProductNumber != $productNumber)
|
||||||
|
{
|
||||||
|
print STDERR "ERROR: Different product number ",
|
||||||
|
"at $fileName line $.\n";
|
||||||
|
exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
push @pagesFound, (sprintf "%5d:%4d:%d:%d:%d",
|
||||||
|
$fileNumber, $pageNumber, $flags, $fileIndex, $filePos);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
close(FILE) || die;
|
||||||
|
}
|
||||||
|
|
||||||
|
@pagesFound = sort @pagesFound;
|
||||||
|
|
||||||
|
$result = 0;
|
||||||
|
$lastFileNumber = 0;
|
||||||
|
$lastPageNumber = 0;
|
||||||
|
$nextFileNumber = 1;
|
||||||
|
$nextPageNumber = 1;
|
||||||
|
$fileIndexOpen = -1;
|
||||||
|
foreach (@pagesFound)
|
||||||
|
{
|
||||||
|
my ($fileNumber, $pageNumber, $flags, $fileIndex, $filePos) = split /:/;
|
||||||
|
|
||||||
|
$fileNumber = int($fileNumber);
|
||||||
|
$pageNumber = int($pageNumber);
|
||||||
|
|
||||||
|
if ($fileNumber == $lastFileNumber && $pageNumber == $lastPageNumber)
|
||||||
|
{
|
||||||
|
print STDERR "DUPLICATE: File $fileNumber, page $pageNumber, skipped\n";
|
||||||
|
next;
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($nextFileNumber < $fileNumber && $nextPageNumber != 1)
|
||||||
|
{
|
||||||
|
print STDERR "MISSING: File $nextFileNumber, ",
|
||||||
|
"pages $nextPageNumber - END\n";
|
||||||
|
$nextPageNumber = 1;
|
||||||
|
$nextFileNumber++;
|
||||||
|
$result = 1;
|
||||||
|
}
|
||||||
|
if ($nextFileNumber < $fileNumber)
|
||||||
|
{
|
||||||
|
print STDERR "MISSING: Files $nextFileNumber - ",
|
||||||
|
$fileNumber-1, "\n";
|
||||||
|
$nextFileNumber = $fileNumber;
|
||||||
|
$nextPageNumber = 1;
|
||||||
|
$result = 1;
|
||||||
|
}
|
||||||
|
if ($nextFileNumber != $fileNumber)
|
||||||
|
{
|
||||||
|
print STDERR "ERROR: Internal error, unexpected fileNumber\n";
|
||||||
|
exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($nextPageNumber < $pageNumber)
|
||||||
|
{
|
||||||
|
print STDERR "MISSING: File $fileNumber, pages $nextPageNumber - ",
|
||||||
|
$pageNumber-1, "\n";
|
||||||
|
$nextPageNumber = $pageNumber;
|
||||||
|
$result = 1;
|
||||||
|
}
|
||||||
|
if ($nextPageNumber != $pageNumber)
|
||||||
|
{
|
||||||
|
print STDERR "ERROR: Internal error, unexpected pageNumber\n";
|
||||||
|
exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($fileIndexOpen != $fileIndex)
|
||||||
|
{
|
||||||
|
if ($fileIndexOpen >= 0)
|
||||||
|
{
|
||||||
|
close(FILE) || die;
|
||||||
|
$fileIndexOpen = -1;
|
||||||
|
}
|
||||||
|
$fileName = $ARGV[$fileIndex];
|
||||||
|
open(FILE, "<$fileName") || die;
|
||||||
|
$fileIndexOpen = $fileIndex;
|
||||||
|
}
|
||||||
|
seek(FILE, $filePos, 0) || die($!);
|
||||||
|
|
||||||
|
$_ = <FILE>;
|
||||||
|
print;
|
||||||
|
while (<FILE>)
|
||||||
|
{
|
||||||
|
last if /^\f?-\S/;
|
||||||
|
print;
|
||||||
|
}
|
||||||
|
$lastFileNumber = $fileNumber;
|
||||||
|
$lastPageNumber = $pageNumber;
|
||||||
|
|
||||||
|
if ($flags & 1) # Bit 0 of flags indicates last page of file
|
||||||
|
{
|
||||||
|
$nextFileNumber++;
|
||||||
|
$nextPageNumber = 1;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
$nextPageNumber++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($nextPageNumber != 1)
|
||||||
|
{
|
||||||
|
print STDERR "MISSING: File $nextFileNumber, ",
|
||||||
|
"pages $nextPageNumber - END\n";
|
||||||
|
$nextPageNumber = 1;
|
||||||
|
$nextFileNumber++;
|
||||||
|
$result = 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
print STDERR "Highest file number encountered: ", $nextFileNumber - 1, "\n";
|
||||||
|
|
||||||
|
if ($fileIndexOpen >= 0)
|
||||||
|
{
|
||||||
|
close(FILE) || die;
|
||||||
|
$fileIndexOpen = -1;
|
||||||
|
}
|
||||||
|
|
||||||
|
exit($result);
|
||||||
|
|
||||||
|
#
|
||||||
|
# vi: ai ts=4
|
||||||
|
# vim: si
|
||||||
|
#
|
||||||
222
tools/subst.c
Normal file
222
tools/subst.c
Normal file
@ -0,0 +1,222 @@
|
|||||||
|
/*
|
||||||
|
* subst.c -- Repair substitution tables
|
||||||
|
*
|
||||||
|
* Copyright (C) 1997 Pretty Good Privacy, Inc.
|
||||||
|
*
|
||||||
|
* Written by Colin Plumb
|
||||||
|
*
|
||||||
|
* $Id: subst.c,v 1.14 1997/11/03 22:12:00 colin Exp $
|
||||||
|
*
|
||||||
|
* IT IS EXPECTED that users of this program will play with these tables
|
||||||
|
* and the cost values in the subst.h header. (Some day, they'll all
|
||||||
|
* get moved to an external config file.)
|
||||||
|
*
|
||||||
|
* NOTE: Other cost are hiding in the Filter functions in repair.c.
|
||||||
|
* Remember to keep them all on the same scale.
|
||||||
|
*/
|
||||||
|
|
||||||
|
/*
|
||||||
|
* The repair program copies its input to its output, making various
|
||||||
|
* substitutions, until it manages to produce a version that satisfies
|
||||||
|
* the parser. This includes having a correct CRC for each line.
|
||||||
|
* Each substitution has a cost, and the combinations are tried in order
|
||||||
|
* of increasing cost. NOTE that even translating "A"->"A" counts as
|
||||||
|
* a substitution, although it may have zero cost.
|
||||||
|
*
|
||||||
|
* The intention is to correct transcription errors, where the
|
||||||
|
* errors have a distinctly non-uniform distribution. Slight
|
||||||
|
* differences in cost produce a preference in trying some errors
|
||||||
|
* first. If an error costs half as much as another, combinations
|
||||||
|
* of two of that error will be compared to one of the more expensive.
|
||||||
|
* Too many cheap substitutions will result is repair spending
|
||||||
|
* a very log time searching before considering the more expensive
|
||||||
|
* substitutions.
|
||||||
|
*
|
||||||
|
* The following parameters and the raw substitution tables are expected
|
||||||
|
* to be edited by the user based on experience. Eventually, this
|
||||||
|
* will be moved into an external config file, but for now it's a matter
|
||||||
|
* of recompiling.
|
||||||
|
*/
|
||||||
|
|
||||||
|
#include "subst.h"
|
||||||
|
#include "util.h"
|
||||||
|
|
||||||
|
/* what the OCR software reports for "unrecognizable */
|
||||||
|
#define UNRECOG_STRING "~\274"
|
||||||
|
|
||||||
|
/*
|
||||||
|
* The input substitutions to make (one-to-one). These are listed in
|
||||||
|
* the order of correction. i.e. uncorrected input first, then corrected
|
||||||
|
* output. Substitutions are one-way; to get two-way, list it twice.
|
||||||
|
*/
|
||||||
|
|
||||||
|
struct RawSubst const substSingles[] = {
|
||||||
|
/* Identity substitutions - note that period (.) is excluded */
|
||||||
|
{ "!\"#$%&'()*+,-./0123456789:;<=>?" SPACE_STRING,
|
||||||
|
"!\"#$%&'()*+,-./0123456789:;<=>?" SPACE_STRING, 0, 0, NULL },
|
||||||
|
{ "@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_\t" TAB_STRING,
|
||||||
|
"@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_\t" TAB_STRING, 0, 0, NULL },
|
||||||
|
{ "`abcdefghijklmnopqrstuvwxyz{|}~\f" FORMFEED_STRING,
|
||||||
|
"`abcdefghijklmnopqrstuvwxyz{|}~\f" FORMFEED_STRING, 0, 0, NULL },
|
||||||
|
#if (TAB_PAD_CHAR & 128) /* Not already included? */
|
||||||
|
{ TAB_PAD_STRING, TAB_PAD_STRING, 0, NULL },
|
||||||
|
#endif
|
||||||
|
{ "\r\n" CONTIN_STRING, "\n\n" CONTIN_STRING, 0, 0, NULL },
|
||||||
|
|
||||||
|
/* Occasionally these just get inserted as glitches */
|
||||||
|
{ ".,'`", NULL, 5, 10, FilterNearBlanks },
|
||||||
|
/* This is now pretty infrequent */
|
||||||
|
{ "-_", "_-", 0, 10, FilterAfterRepeat },
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Capitalization errors are common in some cases
|
||||||
|
* c/C, s/S, u/U are fucked up all the time.
|
||||||
|
* Also o/O, v/V and w/W. x, y and z also give some problems.
|
||||||
|
*/
|
||||||
|
{ "cilmopsuvwxyz", "CILMOPSUVWXYZ", 7, 13, FilterNearLower },
|
||||||
|
{ "CILMOPSUVWXYZ", "cilmopsuvwxyz", 7, 13, FilterNearUpper },
|
||||||
|
/* Other errors */
|
||||||
|
{ "g9aaiji;xX00Si", "9gg2ji;i%%oO3f", 10, 0, NULL },
|
||||||
|
/* This seems to happen a lot */
|
||||||
|
{ "c", "r", 9, 0, NULL },
|
||||||
|
|
||||||
|
{ "j", ";", 9, 0, NULL },
|
||||||
|
{ "' ", "``", 10, 0, NULL },
|
||||||
|
|
||||||
|
/* Uncommon errors */
|
||||||
|
|
||||||
|
/* Wierd stuff that's happened in the checksum part */
|
||||||
|
/* A highish weight is okay here */
|
||||||
|
{ "sSEdJl", "554437", 15, 0, NULL },
|
||||||
|
{ "LESsPZ", "bb8a22", 15, 0, NULL },
|
||||||
|
|
||||||
|
/* Wierd stuff that has happened */
|
||||||
|
{ "BasAeaeRoooo", "3334a@QQpqbd", 5, 15, FilterIsBinary },
|
||||||
|
{ "oooo", "pqbd", 0, 15, FilterIsBinary },
|
||||||
|
{ "ttTCCflO", "iff{[lfG", 12, 0, NULL },
|
||||||
|
#if 0
|
||||||
|
/* If the line-breaks get screwed up, use these */
|
||||||
|
{ " ", "\n", 10, COST_INFINITY, FilterChecksumFollows },
|
||||||
|
{ "\n", " ", COST_INFINITY, 10, FilterChecksumFollows },
|
||||||
|
{ "\n", NULL, COST_INFINITY , 11, FilterChecksumFollows },
|
||||||
|
#endif
|
||||||
|
|
||||||
|
{ NULL, NULL, 0, 0, NULL }
|
||||||
|
};
|
||||||
|
|
||||||
|
/* The many-to-many substitutions */
|
||||||
|
struct RawSubst const substMultiples[] = {
|
||||||
|
{ "''", "\"", 2, 0, NULL },
|
||||||
|
{ "``", "\"", 2, 0, NULL },
|
||||||
|
{ ",'", "\"", 2, 0, NULL },
|
||||||
|
{ "',", "\"", 2, 0, NULL },
|
||||||
|
{ ",,", "\"", 2, 0, NULL },
|
||||||
|
/* Extra inserted spaces are common */
|
||||||
|
{ " ", " ", COST_INFINITY, 0, FilterFollowsSpace },
|
||||||
|
{ " ", "", 0, 15, FilterFollowsSpace },
|
||||||
|
{ "\t", " ", COST_INFINITY, 0, FilterFollowsSpace },
|
||||||
|
{ "\t", "", 0, 10, FilterFollowsSpace },
|
||||||
|
/* Convert between SPACE_CHAR dots and periods */
|
||||||
|
{ ".", SPACE_STRING, 1, COST_INFINITY, FilterFollowsSpace },
|
||||||
|
{ ".", " "SPACE_STRING, COST_INFINITY, 10, FilterFollowsSpace },
|
||||||
|
{ SPACE_STRING, ".", 15, 5, FilterFollowsSpace },
|
||||||
|
{ SPACE_STRING, " "SPACE_STRING, COST_INFINITY, 5, FilterFollowsSpace },
|
||||||
|
|
||||||
|
/* Replace "unknown" by zero - it often is */
|
||||||
|
{ UNRECOG_STRING, "0", 1, 0, NULL },
|
||||||
|
{ UNRECOG_STRING, "_", 2, 0, NULL },
|
||||||
|
{ UNRECOG_STRING, ")", 3, 0, NULL },
|
||||||
|
{ UNRECOG_STRING, "^", 4, 0, NULL },
|
||||||
|
/* Except that these glitches are common */
|
||||||
|
{ UNRECOG_STRING"'", "\\\"", 0, 0, NULL },
|
||||||
|
{ UNRECOG_STRING"'", "\"", 1, 0, NULL },
|
||||||
|
{ "'"UNRECOG_STRING, "\"", 0, 0, NULL },
|
||||||
|
{ UNRECOG_STRING UNRECOG_STRING , "\"", 0, 0, NULL },
|
||||||
|
/* Something else that has been seen */
|
||||||
|
{ "V'", "\\\"", 5, 0, NULL },
|
||||||
|
|
||||||
|
/* A common transposition */
|
||||||
|
{ "\"'", "'\"", 5, 0, NULL },
|
||||||
|
{ "'\"", "\"'", 5, 0, NULL },
|
||||||
|
/* These also happen fairly often */
|
||||||
|
{ " \"", "''", 5, 0, NULL },
|
||||||
|
{ "\" ", "''", 5, 0, NULL },
|
||||||
|
|
||||||
|
/* Common glitches */
|
||||||
|
{ "\t.\n", "\n", 5, 0, NULL },
|
||||||
|
{ "\t,\n", "\n", 5, 0, NULL },
|
||||||
|
{ "\t-\n", "\n", 5, 0, NULL },
|
||||||
|
{ "\t_\n", "\n", 5, 0, NULL },
|
||||||
|
{ "\t'\n", "\n", 5, 0, NULL },
|
||||||
|
{ "\t`\n", "\n", 5, 0, NULL },
|
||||||
|
{ "\t~\n", "\n", 5, 0, NULL },
|
||||||
|
{ "\t:\n", "\n", 5, 0, NULL },
|
||||||
|
{ "\t"SPACE_STRING"\n", "\n", 5, 0, NULL },
|
||||||
|
|
||||||
|
/* Less common */
|
||||||
|
{ " .\n", "\n", 10, 0, NULL },
|
||||||
|
{ " ,\n", "\n", 10, 0, NULL },
|
||||||
|
{ " -\n", "\n", 10, 0, NULL },
|
||||||
|
{ " _\n", "\n", 10, 0, NULL },
|
||||||
|
{ " '\n", "\n", 10, 0, NULL },
|
||||||
|
{ " `\n", "\n", 10, 0, NULL },
|
||||||
|
{ " ~\n", "\n", 10, 0, NULL },
|
||||||
|
{ " :\n", "\n", 10, 0, NULL },
|
||||||
|
{ " "SPACE_STRING"\n", "\n", 10, 0, NULL },
|
||||||
|
|
||||||
|
/* Even less common */
|
||||||
|
{ ".\n", "\n", 15, 0, NULL },
|
||||||
|
{ ",\n", "\n", 15, 0, NULL },
|
||||||
|
{ "-\n", "\n", 15, 0, NULL },
|
||||||
|
{ "_\n", "\n", 15, 0, NULL },
|
||||||
|
{ "'\n", "\n", 15, 0, NULL },
|
||||||
|
{ "`\n", "\n", 15, 0, NULL },
|
||||||
|
{ "~\n", "\n", 15, 0, NULL },
|
||||||
|
{ ":\n", "\n", 15, 0, NULL },
|
||||||
|
{ SPACE_STRING"\n", "\n", 15, 0, NULL },
|
||||||
|
|
||||||
|
/* Wierd stuff that has happened */
|
||||||
|
{ "lJ", "U", 10, 0, NULL },
|
||||||
|
{ "ll", "U", 10, 0, NULL },
|
||||||
|
{ "l1", "U", 10, 0, NULL },
|
||||||
|
{ "il", "U", 10, 0, NULL }, /* Fairly common, actually */
|
||||||
|
{ "li", "U", 10, 0, NULL },
|
||||||
|
{ "l)", "U", 10, 0, NULL },
|
||||||
|
{ "Ll", "U", 10, 0, NULL },
|
||||||
|
{ "LI", "U", 10, 0, NULL },
|
||||||
|
{ "L1", "U", 10, 0, NULL },
|
||||||
|
|
||||||
|
{ "lo", "b", 10, 0, NULL },
|
||||||
|
{ "cl", "d", 10, 0, NULL },
|
||||||
|
{ "cliff", "diff", 2, 0, NULL },
|
||||||
|
{ "*\n", "*/\n", 10, 0, NULL },
|
||||||
|
|
||||||
|
/* That big black block has odd things happen to it */
|
||||||
|
{ "d", CONTIN_STRING, 10, 0, NULL },
|
||||||
|
{ "d\n", CONTIN_STRING"\n", 3, 0, NULL },
|
||||||
|
{ "S", CONTIN_STRING, 10, 0, NULL },
|
||||||
|
{ "S\n", CONTIN_STRING"\n", 3, 0, NULL },
|
||||||
|
|
||||||
|
/* Tab-stop wonders */
|
||||||
|
{ TAB_STRING, TAB_STRING"", 0, 0, TabFilter },
|
||||||
|
{ TAB_STRING, TAB_STRING" ", 0, 0, TabFilter },
|
||||||
|
{ TAB_STRING, TAB_STRING" ", 0, 0, TabFilter },
|
||||||
|
{ TAB_STRING, TAB_STRING" ", 0, 0, TabFilter },
|
||||||
|
{ TAB_STRING, TAB_STRING" ", 0, 0, TabFilter },
|
||||||
|
{ TAB_STRING, TAB_STRING" ", 0, 0, TabFilter },
|
||||||
|
{ TAB_STRING, TAB_STRING" ", 0, 0, TabFilter },
|
||||||
|
{ TAB_STRING, TAB_STRING" ", 0, 0, TabFilter },
|
||||||
|
/* Some scan errors */
|
||||||
|
{ "D ", TAB_STRING"", 1, 5, TabFilter },
|
||||||
|
{ "D ", TAB_STRING" ", 1, 5, TabFilter },
|
||||||
|
{ "D ", TAB_STRING" ", 1, 5, TabFilter },
|
||||||
|
{ "D ", TAB_STRING" ", 1, 5, TabFilter },
|
||||||
|
{ "D ", TAB_STRING" ", 1, 5, TabFilter },
|
||||||
|
{ "D ", TAB_STRING" ", 1, 5, TabFilter },
|
||||||
|
{ "D ", TAB_STRING" ", 1, 5, TabFilter },
|
||||||
|
{ "D ", TAB_STRING" ", 1, 5, TabFilter },
|
||||||
|
#if TAB_PAD_CHAR != ' '
|
||||||
|
#error Fix those tab patterns!
|
||||||
|
#endif
|
||||||
|
{ NULL, NULL, 0, 0, NULL }
|
||||||
|
};
|
||||||
66
tools/subst.h
Normal file
66
tools/subst.h
Normal file
@ -0,0 +1,66 @@
|
|||||||
|
/*
|
||||||
|
* subst.h -- Header for repair substitutions
|
||||||
|
*
|
||||||
|
* Copyright (C) 1997 Pretty Good Privacy, Inc.
|
||||||
|
*
|
||||||
|
* Written by Colin Plumb
|
||||||
|
*
|
||||||
|
* $Id: subst.h,v 1.9 1997/11/03 22:12:00 colin Exp $
|
||||||
|
*/
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Give up if the list of pending changes to attempt grows to this many
|
||||||
|
* elements. Each element is 32 bytes, so 128K is 8 MB of memory.
|
||||||
|
* (Other than this, repair's memory usage is fairly modest.)
|
||||||
|
*/
|
||||||
|
#define MAX_HEAP (1<<17)
|
||||||
|
|
||||||
|
/*
|
||||||
|
* There is a hack in the code to find a single substitution that will fix a
|
||||||
|
* line, even if it's not in the tables. It gets added to the tables "on
|
||||||
|
* probation", with an infinite cost, and if it leads to a successful
|
||||||
|
* correction of the entire page, is "learned" for future use and its
|
||||||
|
* cost reduced to something finite.
|
||||||
|
* (This is not remembered across runs of the program, though.
|
||||||
|
* Edit the tables in the source to fix it.)
|
||||||
|
*/
|
||||||
|
#define DYNAMIC_COST_LEARNED 15
|
||||||
|
|
||||||
|
/*
|
||||||
|
* This negative-cost bonus for passing the end of a line with the right
|
||||||
|
* CRC makes the search engine reluctant to backtrack past a correct CRC,
|
||||||
|
* greatly improving efficiency. It's rather a hack, though. Think of
|
||||||
|
* this in terms of "how many errors should be considered in the current
|
||||||
|
* line before considering the possibility of errors in the previous line?"
|
||||||
|
*
|
||||||
|
* This bonus is halved for lines that are the result of a correction
|
||||||
|
* that was computed from the checksum, since a correct checksum is
|
||||||
|
* much less significant in such a case.
|
||||||
|
*/
|
||||||
|
#define COST_LINE -30
|
||||||
|
|
||||||
|
/* The cost of a full-line nastyline substitution. */
|
||||||
|
#define NASTY_COST 5
|
||||||
|
|
||||||
|
/* Type describing filter functions used in substitutions */
|
||||||
|
struct ParseNode;
|
||||||
|
struct Substitution;
|
||||||
|
#include "heap.h"
|
||||||
|
typedef HeapCost FilterFunc(struct ParseNode *parent, char const *limit,
|
||||||
|
struct Substitution const *subst);
|
||||||
|
FilterFunc TabFilter, FilterFollowsSpace, FilterNearBlanks;
|
||||||
|
FilterFunc FilterNearUpper, FilterNearLower, FilterNearXDigit;
|
||||||
|
FilterFunc FilterAfterRepeat, FilterCharConst, FilterChecksumFollows;
|
||||||
|
FilterFunc FilterLikelyUnderscore, FilterIsDynamic, FilterIsBinary;
|
||||||
|
|
||||||
|
/* The external substitution format */
|
||||||
|
typedef struct RawSubst {
|
||||||
|
char const *input;
|
||||||
|
char const *output;
|
||||||
|
HeapCost cost, cost2;
|
||||||
|
FilterFunc *filter;
|
||||||
|
} RawSubst;
|
||||||
|
|
||||||
|
/* The substitutions to make */
|
||||||
|
extern struct RawSubst const substSingles[];
|
||||||
|
extern struct RawSubst const substMultiples[];
|
||||||
666
tools/unmunge.c
Normal file
666
tools/unmunge.c
Normal file
@ -0,0 +1,666 @@
|
|||||||
|
/*
|
||||||
|
* unmunge.c -- Program to convert a munged file to original form
|
||||||
|
*
|
||||||
|
* Copyright (C) 1997 Pretty Good Privacy, Inc.
|
||||||
|
*
|
||||||
|
* Designed by Colin Plumb, Mark H. Weaver, and Philip R. Zimmermann
|
||||||
|
* Written by Mark H. Weaver
|
||||||
|
*
|
||||||
|
* $Id: unmunge.c,v 1.13 1997/11/13 23:27:08 mhw Exp $
|
||||||
|
*/
|
||||||
|
|
||||||
|
#include <sys/stat.h>
|
||||||
|
#include <sys/types.h>
|
||||||
|
#include <fcntl.h>
|
||||||
|
#include <unistd.h>
|
||||||
|
|
||||||
|
/*#include <direct.h> teun: MS VC wants direct.h for mkdir */
|
||||||
|
|
||||||
|
#include <stdio.h>
|
||||||
|
#include <errno.h>
|
||||||
|
#include <string.h>
|
||||||
|
#include <ctype.h>
|
||||||
|
#include <stdlib.h>
|
||||||
|
#include <assert.h>
|
||||||
|
|
||||||
|
#include "util.h"
|
||||||
|
|
||||||
|
typedef struct UnMungeState
|
||||||
|
{
|
||||||
|
char const * mungedFileName;
|
||||||
|
char dirName[128];
|
||||||
|
char fileName[128];
|
||||||
|
char * fileNameTail;
|
||||||
|
int binaryMode, tabWidth;
|
||||||
|
long productNumber, fileNumber, pageNumber, lineNumber;
|
||||||
|
long manifestLineNumber;
|
||||||
|
word16 hdrFlags;
|
||||||
|
CRC pageCRC, seenPageCRC;
|
||||||
|
FILE * manifest;
|
||||||
|
FILE * file;
|
||||||
|
FILE * out;
|
||||||
|
} UnMungeState;
|
||||||
|
|
||||||
|
|
||||||
|
/* Returns number of characters decoded, or -1 on error */
|
||||||
|
static int
|
||||||
|
Decode4(char const src[4], byte dest[3])
|
||||||
|
{
|
||||||
|
int i, length;
|
||||||
|
byte srcVal[4];
|
||||||
|
|
||||||
|
for (i = 0; i < 4 && src[i] != RADIX64_END_CHAR; i++)
|
||||||
|
if ((srcVal[i] = Radix64DigitValue(src[i])) == (byte) -1)
|
||||||
|
return 1;
|
||||||
|
|
||||||
|
length = i - 1;
|
||||||
|
if (length < 1)
|
||||||
|
return -1;
|
||||||
|
|
||||||
|
for (; i < 4; i++)
|
||||||
|
srcVal[0] = 0;
|
||||||
|
|
||||||
|
dest[0] = (srcVal[0] << 2) | (srcVal[1] >> 4);
|
||||||
|
dest[1] = (srcVal[1] << 4) | (srcVal[2] >> 2);
|
||||||
|
dest[2] = (srcVal[2] << 6) | (srcVal[3]);
|
||||||
|
|
||||||
|
return length;
|
||||||
|
}
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Return number of characters decoded, or -1 on error
|
||||||
|
*/
|
||||||
|
static int
|
||||||
|
DecodeLine(char const *src, char *dest, int srclength)
|
||||||
|
{
|
||||||
|
int destlength = 0;
|
||||||
|
int result;
|
||||||
|
|
||||||
|
if (srclength % 4 || !srclength)
|
||||||
|
return -1; /* Must be a multiple of 4 */
|
||||||
|
|
||||||
|
while (srclength -= 4) {
|
||||||
|
if (Decode4(src, dest + destlength) != 3)
|
||||||
|
return -1;
|
||||||
|
src += 4;
|
||||||
|
destlength += 3;
|
||||||
|
}
|
||||||
|
result = Decode4(src, dest + destlength);
|
||||||
|
if (result < 1)
|
||||||
|
return -1;
|
||||||
|
return destlength + result;
|
||||||
|
}
|
||||||
|
|
||||||
|
int PrintFileError(UnMungeState *state, char const *message)
|
||||||
|
{
|
||||||
|
fprintf(stderr, "%s, %s line %ld\n", message,
|
||||||
|
state->mungedFileName, state->lineNumber);
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
int ReadManifest(UnMungeState *state, long fileNumberWanted,
|
||||||
|
char const *fileTailPrefix, long prefixLen)
|
||||||
|
{
|
||||||
|
long fileNumber = 0;
|
||||||
|
long firstMissingFileNum = 0, lastMissingFileNum = 0;
|
||||||
|
char buffer[512];
|
||||||
|
char * p;
|
||||||
|
|
||||||
|
if (state->manifest == NULL)
|
||||||
|
{
|
||||||
|
if (fileNumberWanted != 0)
|
||||||
|
{
|
||||||
|
assert(fileTailPrefix != NULL);
|
||||||
|
strncpy(state->fileName, fileTailPrefix, sizeof(state->fileName));
|
||||||
|
state->fileName[sizeof(state->fileName) - 1] = '\0';
|
||||||
|
state->fileNameTail = state->fileName;
|
||||||
|
}
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
while (fgets(buffer, sizeof(buffer), state->manifest))
|
||||||
|
{
|
||||||
|
if ((p = strchr(buffer, '\n')) != NULL)
|
||||||
|
*p = '\0';
|
||||||
|
state->manifestLineNumber++;
|
||||||
|
if (buffer[0] == 'D')
|
||||||
|
{
|
||||||
|
if (buffer[1] != ' ')
|
||||||
|
goto invalidManifest;
|
||||||
|
strncpy(state->dirName, buffer + 2, sizeof(state->dirName));
|
||||||
|
if (state->dirName[sizeof(state->dirName) - 1] != '\0')
|
||||||
|
goto invalidManifest;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
fileNumber = strtol(buffer, &p, 10);
|
||||||
|
if (p == buffer || *p != ' ')
|
||||||
|
goto invalidManifest;
|
||||||
|
p++;
|
||||||
|
|
||||||
|
if (fileNumberWanted == 0 || fileNumber < fileNumberWanted)
|
||||||
|
{
|
||||||
|
if (firstMissingFileNum == 0)
|
||||||
|
firstMissingFileNum = fileNumber;
|
||||||
|
lastMissingFileNum = fileNumber;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
else if (fileNumber > fileNumberWanted)
|
||||||
|
break;
|
||||||
|
else
|
||||||
|
{
|
||||||
|
size_t len;
|
||||||
|
|
||||||
|
len = strlen(state->dirName);
|
||||||
|
assert(sizeof(state->fileName) >= sizeof(state->dirName));
|
||||||
|
memcpy(state->fileName, state->dirName, len);
|
||||||
|
strncpy(state->fileName + len, p,
|
||||||
|
sizeof(state->fileName) - len);
|
||||||
|
if (strncmp(p, fileTailPrefix, prefixLen) != 0)
|
||||||
|
{
|
||||||
|
fprintf(stderr, "Mismatched filename, headers say '%s',\n"
|
||||||
|
" manifest says '%s'\n",
|
||||||
|
fileTailPrefix, p);
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
p = state->dirName;
|
||||||
|
while ((p = strchr(p, '/')) != NULL)
|
||||||
|
{
|
||||||
|
*p = '\0';
|
||||||
|
mkdir(state->dirName, 0777);
|
||||||
|
*p++ = '/';
|
||||||
|
}
|
||||||
|
state->fileNameTail = state->fileName + len;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (firstMissingFileNum != 0)
|
||||||
|
{
|
||||||
|
fprintf(stderr, "Missing files %ld-%ld\n",
|
||||||
|
firstMissingFileNum, lastMissingFileNum);
|
||||||
|
}
|
||||||
|
if (fileNumberWanted != 0 && fileNumber != fileNumberWanted)
|
||||||
|
{
|
||||||
|
fprintf(stderr, "Can't find file %ld in manifest file\n",
|
||||||
|
fileNumberWanted);
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
return 0;
|
||||||
|
|
||||||
|
invalidManifest:
|
||||||
|
fprintf(stderr, "Error parsing manifest file, line %ld\n",
|
||||||
|
state->manifestLineNumber);
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
int UnMungeFile(char const *mungedFileName, char const *manifestFileName,
|
||||||
|
int forceOverwrite, int forcePartialFiles)
|
||||||
|
{
|
||||||
|
UnMungeState * state;
|
||||||
|
EncodeFormat const * fmt = NULL;
|
||||||
|
char buffer[512];
|
||||||
|
char outbuf[BYTES_PER_LINE+1];
|
||||||
|
char * line;
|
||||||
|
char * lineData;
|
||||||
|
char * p;
|
||||||
|
int length;
|
||||||
|
int result = 0;
|
||||||
|
int skipPage = 0;
|
||||||
|
CRC lineCRC;
|
||||||
|
word32 num;
|
||||||
|
|
||||||
|
state = (UnMungeState *)calloc(1, sizeof(*state));
|
||||||
|
state->mungedFileName = mungedFileName;
|
||||||
|
|
||||||
|
if (manifestFileName != NULL)
|
||||||
|
{
|
||||||
|
if ((state->manifest = fopen(manifestFileName, "r")) == NULL)
|
||||||
|
goto errnoError;
|
||||||
|
}
|
||||||
|
|
||||||
|
if ((state->file = fopen(state->mungedFileName, "r")) == NULL)
|
||||||
|
goto errnoError;
|
||||||
|
|
||||||
|
while (!feof(state->file))
|
||||||
|
{
|
||||||
|
if (fgets(buffer, sizeof(buffer), state->file) == NULL)
|
||||||
|
{
|
||||||
|
if (feof(state->file))
|
||||||
|
break;
|
||||||
|
goto fileError;
|
||||||
|
}
|
||||||
|
|
||||||
|
state->lineNumber++;
|
||||||
|
|
||||||
|
line = buffer;
|
||||||
|
/* Strip leading whitespace */
|
||||||
|
while (isspace(*line))
|
||||||
|
line++;
|
||||||
|
if (*line == '\0')
|
||||||
|
continue;
|
||||||
|
|
||||||
|
/* Strip trailing whitespace */
|
||||||
|
p = line + strlen(line);
|
||||||
|
while (p > line && (byte)p[-1] < 128 && isspace(p[-1]))
|
||||||
|
p--;
|
||||||
|
|
||||||
|
lineData = line + PREFIX_LENGTH;
|
||||||
|
|
||||||
|
/* Pad up to at least PREFIX_LENGTH */
|
||||||
|
while (p < lineData)
|
||||||
|
*p++ = ' ';
|
||||||
|
*p++ = '\n';
|
||||||
|
*p = '\0';
|
||||||
|
length = p - lineData;
|
||||||
|
|
||||||
|
if (line[0] == HDR_PREFIX_CHAR)
|
||||||
|
{
|
||||||
|
fmt = FindFormat(line[1]);
|
||||||
|
if (!fmt)
|
||||||
|
{
|
||||||
|
result = PrintFileError(state, "ERROR: Invalid header type");
|
||||||
|
goto error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
lineCRC = CalculateCRC(fmt->lineCRC, 0, (byte const *)lineData, length);
|
||||||
|
|
||||||
|
p = line + EncodedLength(fmt, fmt->runningCRCBits);
|
||||||
|
if (DecodeCheckDigits(fmt, p, NULL, fmt->lineCRC->bits, &num)
|
||||||
|
|| lineCRC != num)
|
||||||
|
{
|
||||||
|
result = PrintFileError(state, "ERROR: Line CRC failed");
|
||||||
|
goto error;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (line[0] == HDR_PREFIX_CHAR)
|
||||||
|
{
|
||||||
|
int formatVersion;
|
||||||
|
int flags;
|
||||||
|
CRC seenPageCRC;
|
||||||
|
int tabWidth;
|
||||||
|
long productNumber;
|
||||||
|
long fileNumber;
|
||||||
|
long pageNumber;
|
||||||
|
char * fileNameTail;
|
||||||
|
int skipNextPage = 0;
|
||||||
|
char * p;
|
||||||
|
EncodeFormat const * hFmt = &hexFormat;
|
||||||
|
|
||||||
|
/* Parse header line */
|
||||||
|
p = lineData;
|
||||||
|
|
||||||
|
if (DecodeCheckDigits(hFmt, p, &p, HDR_VERSION_BITS, &num))
|
||||||
|
{
|
||||||
|
invalidHeader:
|
||||||
|
result = PrintFileError(state, "ERROR: Invalid header");
|
||||||
|
goto error;
|
||||||
|
}
|
||||||
|
formatVersion = num;
|
||||||
|
|
||||||
|
if (DecodeCheckDigits(hFmt, p, &p, HDR_FLAG_BITS, &num))
|
||||||
|
goto invalidHeader;
|
||||||
|
flags = num;
|
||||||
|
|
||||||
|
if (DecodeCheckDigits(hFmt, p, &p, fmt->pageCRC->bits, &num))
|
||||||
|
goto invalidHeader;
|
||||||
|
seenPageCRC = num;
|
||||||
|
|
||||||
|
if (DecodeCheckDigits(hFmt, p, &p, HDR_TABWIDTH_BITS, &num))
|
||||||
|
goto invalidHeader;
|
||||||
|
tabWidth = num;
|
||||||
|
|
||||||
|
if (DecodeCheckDigits(hFmt, p, &p, HDR_PRODNUM_BITS, &num))
|
||||||
|
goto invalidHeader;
|
||||||
|
productNumber = num;
|
||||||
|
|
||||||
|
if (DecodeCheckDigits(hFmt, p, &p, HDR_FILENUM_BITS, &num))
|
||||||
|
goto invalidHeader;
|
||||||
|
fileNumber = num;
|
||||||
|
|
||||||
|
if (sscanf(p, " Page %ld of ", &pageNumber) < 1)
|
||||||
|
goto invalidHeader;
|
||||||
|
|
||||||
|
if (formatVersion > 0)
|
||||||
|
{
|
||||||
|
result = PrintFileError(state,
|
||||||
|
"ERROR: Format too new for "
|
||||||
|
"this version of unmunge");
|
||||||
|
goto error;
|
||||||
|
}
|
||||||
|
|
||||||
|
p = strstr(p, " of ");
|
||||||
|
if (p == NULL)
|
||||||
|
goto invalidHeader;
|
||||||
|
|
||||||
|
fileNameTail = p + 4;
|
||||||
|
p = fileNameTail + strlen(fileNameTail);
|
||||||
|
if (p < fileNameTail + 3 || p[-1] != '\n')
|
||||||
|
goto invalidHeader;
|
||||||
|
else
|
||||||
|
p[-1] = '\0';
|
||||||
|
|
||||||
|
if (state->out != NULL && state->pageCRC != state->seenPageCRC)
|
||||||
|
{
|
||||||
|
result = PrintFileError(state,
|
||||||
|
"ERROR: Page CRC mismatch on page before");
|
||||||
|
goto error;
|
||||||
|
}
|
||||||
|
|
||||||
|
if ((state->hdrFlags & HDR_FLAG_LASTPAGE) && state->out != NULL)
|
||||||
|
{
|
||||||
|
fclose(state->out);
|
||||||
|
state->out = NULL;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (state->out != NULL)
|
||||||
|
{
|
||||||
|
if (pageNumber != state->pageNumber + 1 ||
|
||||||
|
fileNumber != state->fileNumber ||
|
||||||
|
productNumber != state->productNumber ||
|
||||||
|
tabWidth != state->tabWidth ||
|
||||||
|
strcmp(fileNameTail, state->fileNameTail) != 0)
|
||||||
|
{
|
||||||
|
if (fileNumber == state->fileNumber &&
|
||||||
|
pageNumber > state->pageNumber + 1)
|
||||||
|
{
|
||||||
|
(void)PrintFileError(state,
|
||||||
|
"ERROR: Missing pages of this file");
|
||||||
|
if (forcePartialFiles && !state->binaryMode)
|
||||||
|
{
|
||||||
|
fputs("\n\n@@@@@@ Missing pages here! @@@@@@\n\n",
|
||||||
|
state->out);
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
skipNextPage = 1;
|
||||||
|
fclose(state->out);
|
||||||
|
state->out = NULL;
|
||||||
|
remove(state->fileName);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
(void)PrintFileError(state,
|
||||||
|
"ERROR: Missing pages of previous file");
|
||||||
|
if (forcePartialFiles && !state->binaryMode)
|
||||||
|
{
|
||||||
|
fputs("\n\n@@@@@@ Missing pages here! @@@@@@\n\n",
|
||||||
|
state->out);
|
||||||
|
/* Make it non-fatal, though... */
|
||||||
|
fclose(state->out);
|
||||||
|
state->out = NULL;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
fclose(state->out);
|
||||||
|
state->out = NULL;
|
||||||
|
remove(state->fileName);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (state->out == NULL)
|
||||||
|
{
|
||||||
|
if (pageNumber != 1 && !skipPage)
|
||||||
|
(void)PrintFileError(state,
|
||||||
|
"ERROR: File doesn't begin with page 1");
|
||||||
|
|
||||||
|
state->binaryMode = (tabWidth == 0);
|
||||||
|
|
||||||
|
if (pageNumber != 1 && (state->binaryMode
|
||||||
|
|| !forcePartialFiles))
|
||||||
|
{
|
||||||
|
skipNextPage = 1;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
/* TODO: Use global filelist to get pathname */
|
||||||
|
result = ReadManifest(state, fileNumber, fileNameTail,
|
||||||
|
strlen(fileNameTail));
|
||||||
|
if (result != 0)
|
||||||
|
goto error;
|
||||||
|
|
||||||
|
if (!forceOverwrite)
|
||||||
|
{
|
||||||
|
FILE * file;
|
||||||
|
|
||||||
|
/* Make sure file doesn't already exist */
|
||||||
|
file = fopen(state->fileName, "r");
|
||||||
|
if (file != NULL)
|
||||||
|
{
|
||||||
|
fclose(file);
|
||||||
|
fprintf(stderr, "ERROR: %s already exists\n",
|
||||||
|
state->fileName);
|
||||||
|
result = 1;
|
||||||
|
goto error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
state->out = fopen(state->fileName,
|
||||||
|
state->binaryMode ? "wb" : "w");
|
||||||
|
if (state->out == NULL)
|
||||||
|
goto errnoError;
|
||||||
|
|
||||||
|
if (pageNumber != 1)
|
||||||
|
fputs("\n\n@@@@@@ Missing pages here! @@@@@@\n\n",
|
||||||
|
state->out);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
state->pageCRC = 0;
|
||||||
|
state->seenPageCRC = seenPageCRC;
|
||||||
|
state->hdrFlags = (word16)flags;
|
||||||
|
state->pageNumber = pageNumber;
|
||||||
|
state->fileNumber = fileNumber;
|
||||||
|
state->productNumber = productNumber;
|
||||||
|
state->tabWidth = tabWidth;
|
||||||
|
skipPage = skipNextPage;
|
||||||
|
}
|
||||||
|
else if (!skipPage)
|
||||||
|
{
|
||||||
|
if (state->out == NULL)
|
||||||
|
{
|
||||||
|
result = PrintFileError(state, "ERROR: Missing header line");
|
||||||
|
goto error;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Normal data line */
|
||||||
|
state->pageCRC = CalculateCRC(fmt->pageCRC, state->pageCRC,
|
||||||
|
(byte const *)lineData,
|
||||||
|
length);
|
||||||
|
line[2] = '\0';
|
||||||
|
if (DecodeCheckDigits(fmt, line, NULL, fmt->runningCRCBits, &num)
|
||||||
|
|| RunningCRCFromPageCRC(fmt, state->pageCRC) != num)
|
||||||
|
{
|
||||||
|
result = PrintFileError(state, "ERROR: Running CRC failed");
|
||||||
|
goto error;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (state->binaryMode)
|
||||||
|
{
|
||||||
|
length = DecodeLine(lineData, outbuf, length-1);
|
||||||
|
if (length < 0 || length > BYTES_PER_LINE) {
|
||||||
|
result = PrintFileError(state,
|
||||||
|
"ERROR: Corrupt radix-64 data");
|
||||||
|
goto error;
|
||||||
|
}
|
||||||
|
fwrite(outbuf, 1, length, state->out);
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
p = lineData;
|
||||||
|
while (*p != '\0')
|
||||||
|
{
|
||||||
|
if (*p == TAB_CHAR)
|
||||||
|
{
|
||||||
|
p++;
|
||||||
|
putc('\t', state->out);
|
||||||
|
while ((p - lineData) % state->tabWidth)
|
||||||
|
{
|
||||||
|
if (*p == '\n')
|
||||||
|
break;
|
||||||
|
else if (*p == ' ')
|
||||||
|
p++;
|
||||||
|
else
|
||||||
|
{
|
||||||
|
result = PrintFileError(state,
|
||||||
|
"ERROR: Not enough spaces "
|
||||||
|
"after a tab character");
|
||||||
|
goto error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else if (*p == FORMFEED_CHAR)
|
||||||
|
{
|
||||||
|
p++;
|
||||||
|
if (*p != '\n')
|
||||||
|
{
|
||||||
|
result = PrintFileError(state,
|
||||||
|
"ERROR: Formfeed character "
|
||||||
|
"not at end of line");
|
||||||
|
goto error;
|
||||||
|
}
|
||||||
|
p++; /* Skip newline */
|
||||||
|
putc('\f', state->out);
|
||||||
|
}
|
||||||
|
else if (*p == CONTIN_CHAR)
|
||||||
|
{
|
||||||
|
p++;
|
||||||
|
if (*p != '\n')
|
||||||
|
{
|
||||||
|
result = PrintFileError(state,
|
||||||
|
"ERROR: Continuation character "
|
||||||
|
"not at end of line");
|
||||||
|
goto error;
|
||||||
|
}
|
||||||
|
p++; /* Skip newline */
|
||||||
|
}
|
||||||
|
else if (*p == SPACE_CHAR)
|
||||||
|
{
|
||||||
|
putc(' ', state->out);
|
||||||
|
p++;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
putc(*p, state->out);
|
||||||
|
p++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (state->out != NULL)
|
||||||
|
{
|
||||||
|
if (!(state->hdrFlags & HDR_FLAG_LASTPAGE))
|
||||||
|
{
|
||||||
|
result = PrintFileError(state, "ERROR: Missing pages");
|
||||||
|
goto error;
|
||||||
|
}
|
||||||
|
if (state->pageCRC != state->seenPageCRC)
|
||||||
|
{
|
||||||
|
result = PrintFileError(state,
|
||||||
|
"ERROR: Page CRC failed on previous page");
|
||||||
|
goto error;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Check for missing files at the end */
|
||||||
|
result = ReadManifest(state, 0, NULL, 0);
|
||||||
|
goto done;
|
||||||
|
|
||||||
|
errnoError:
|
||||||
|
result = errno;
|
||||||
|
goto printError;
|
||||||
|
|
||||||
|
fileError:
|
||||||
|
result = ferror(state->file);
|
||||||
|
|
||||||
|
printError:
|
||||||
|
fprintf(stderr, "ERROR: %s\n", strerror(result));
|
||||||
|
|
||||||
|
error:
|
||||||
|
done:
|
||||||
|
if (state != NULL)
|
||||||
|
{
|
||||||
|
if (state->out != NULL)
|
||||||
|
fclose(state->out);
|
||||||
|
if (state->file != NULL)
|
||||||
|
fclose(state->file);
|
||||||
|
if (state->manifest != NULL)
|
||||||
|
fclose(state->manifest);
|
||||||
|
free(state);
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
}
|
||||||
|
|
||||||
|
void UsageAndExit(int result)
|
||||||
|
{
|
||||||
|
fprintf(stderr,
|
||||||
|
"Usage: unmunge [-fp] <file> [<manifest>]\n"
|
||||||
|
" -f Force overwrites of existing files\n"
|
||||||
|
" -p Force unmunge of partial files\n");
|
||||||
|
exit(result);
|
||||||
|
}
|
||||||
|
|
||||||
|
int main(int argc, char *argv[])
|
||||||
|
{
|
||||||
|
int result = 0;
|
||||||
|
int forceOverwrite = 0;
|
||||||
|
int forcePartialFiles = 0;
|
||||||
|
char * fileName = NULL;
|
||||||
|
char * manifestFileName = NULL;
|
||||||
|
int i, j;
|
||||||
|
|
||||||
|
InitUtil();
|
||||||
|
|
||||||
|
for (i = 1; i < argc && argv[i][0] == '-'; i++)
|
||||||
|
{
|
||||||
|
if (0 == strcmp(argv[i], "--"))
|
||||||
|
{
|
||||||
|
i++;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
for (j = 1; argv[i][j] != '\0'; j++)
|
||||||
|
{
|
||||||
|
if (argv[i][j] == 'h')
|
||||||
|
UsageAndExit(0);
|
||||||
|
else if (argv[i][j] == 'f')
|
||||||
|
forceOverwrite = 1;
|
||||||
|
else if (argv[i][j] == 'p')
|
||||||
|
forcePartialFiles = 1;
|
||||||
|
else
|
||||||
|
{
|
||||||
|
fprintf(stderr, "ERROR: Unrecognized option -%c\n", argv[i][j]);
|
||||||
|
UsageAndExit(1);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (i < argc)
|
||||||
|
fileName = argv[i++];
|
||||||
|
if (i < argc)
|
||||||
|
manifestFileName = argv[i++];
|
||||||
|
if (fileName == NULL || i < argc)
|
||||||
|
UsageAndExit(1);
|
||||||
|
|
||||||
|
if ((result = UnMungeFile(fileName, manifestFileName,
|
||||||
|
forceOverwrite, forcePartialFiles)) != 0)
|
||||||
|
{
|
||||||
|
/* If result > 0, message should have already been printed */
|
||||||
|
if (result < 0)
|
||||||
|
fprintf(stderr, "ERROR: %s\n", strerror(result));
|
||||||
|
exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Local Variables:
|
||||||
|
* tab-width: 4
|
||||||
|
* End:
|
||||||
|
* vi: ts=4 sw=4
|
||||||
|
* vim: si
|
||||||
|
*/
|
||||||
|
|
||||||
198
tools/util.c
Normal file
198
tools/util.c
Normal file
@ -0,0 +1,198 @@
|
|||||||
|
/*
|
||||||
|
* util.c -- Miscellaneous shared code/data
|
||||||
|
*
|
||||||
|
* Copyright (C) 1997 Pretty Good Privacy, Inc.
|
||||||
|
*
|
||||||
|
* Written by Mark H. Weaver
|
||||||
|
*
|
||||||
|
* $Id: util.c,v 1.11 1997/11/07 00:44:10 mhw Exp $
|
||||||
|
*/
|
||||||
|
|
||||||
|
#include <stdlib.h>
|
||||||
|
#include "util.h"
|
||||||
|
|
||||||
|
char const hexDigits[] = "0123456789abcdef";
|
||||||
|
char const radix64Digits[] =
|
||||||
|
#if 0 /* Standard */
|
||||||
|
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
|
||||||
|
#else /* Modified form that avoids hard-to-OCR characters */
|
||||||
|
"ABCDEFGHIJKLMNPQRSTVWXYZabcdehijklmnpqtuwy145689\\^!#$%&*+=/:<>?@";
|
||||||
|
#endif
|
||||||
|
|
||||||
|
signed char hexDigitsInv[256];
|
||||||
|
signed char radix64DigitsInv[256];
|
||||||
|
|
||||||
|
/* teun: moved intitialisation of all three CRCPoly's to initUtil() */
|
||||||
|
|
||||||
|
/* CRC-CCITT: x^16 + x^12 + x^5 + 1 */
|
||||||
|
CRCPoly crcCCITTPoly;
|
||||||
|
/*
|
||||||
|
* PRZ's magic 24-bit polynomial - (x+1) * (irreducible of degree 23)
|
||||||
|
* x^24 +x^23 +x^18 +x^17 +x^14 +x^11 +x^10 +x^7 +x^6 +x^5 +x^4 +x^3 +x +1
|
||||||
|
* (Developed by Neal Glover). Note: this is bit-reversed from the form
|
||||||
|
* used in PGP, 0x1864cfb.
|
||||||
|
*/
|
||||||
|
CRCPoly crc24Poly;
|
||||||
|
/* CRC-32: x^32+x^26+x^23+x^22+x^16+x^12+x^11+x^10+x^8+x^7+x^5+x^4+x^2+x+1 */
|
||||||
|
CRCPoly crc32Poly;
|
||||||
|
|
||||||
|
EncodeFormat const hexFormat =
|
||||||
|
{
|
||||||
|
NULL, /* nextFormat */
|
||||||
|
'-', /* headerTypeChar */
|
||||||
|
hexDigits, /* digits */
|
||||||
|
hexDigitsInv, /* digitsInv */
|
||||||
|
4, /* bitsPerDigit */
|
||||||
|
16, /* radix */
|
||||||
|
&crcCCITTPoly, /* lineCRC */
|
||||||
|
&crc32Poly, /* pageCRC */
|
||||||
|
8, /* runningCRCBits */
|
||||||
|
24, /* runningCRCShift */
|
||||||
|
0xFF /* runningCRCMask */
|
||||||
|
};
|
||||||
|
|
||||||
|
EncodeFormat const radix64Format =
|
||||||
|
{
|
||||||
|
&hexFormat, /* nextFormat */
|
||||||
|
'A', /* headerTypeChar */
|
||||||
|
radix64Digits, /* digits */
|
||||||
|
radix64DigitsInv, /* digitsInv */
|
||||||
|
6, /* bitsPerDigit */
|
||||||
|
64, /* radix */
|
||||||
|
&crc24Poly, /* lineCRC */
|
||||||
|
&crc32Poly, /* pageCRC */
|
||||||
|
12, /* runningCRCBits */
|
||||||
|
20, /* runningCRCShift */
|
||||||
|
0xFFF /* runningCRCMask */
|
||||||
|
};
|
||||||
|
|
||||||
|
EncodeFormat const * firstFormat = &radix64Format;
|
||||||
|
|
||||||
|
|
||||||
|
static void InitCRCPoly(CRCPoly *poly)
|
||||||
|
{
|
||||||
|
int i, oneBit;
|
||||||
|
CRC crc = 1;
|
||||||
|
|
||||||
|
poly->table[0] = 0;
|
||||||
|
for (oneBit = 0x80; oneBit > 0; oneBit >>= 1) {
|
||||||
|
crc = (crc >> 1) ^ ((crc & 1) ? poly->poly : 0);
|
||||||
|
for (i = 0; i < 0x100; i += 2 * oneBit)
|
||||||
|
poly->table[i + oneBit] = poly->table[i] ^ crc;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
CRC CalculateCRC(CRCPoly const *poly, CRC crc,
|
||||||
|
byte const *buffer, size_t length)
|
||||||
|
{
|
||||||
|
while (length--)
|
||||||
|
crc = (crc >> 8) ^ poly->table[(crc & 0xFF) ^ (*buffer++)];
|
||||||
|
return crc;
|
||||||
|
}
|
||||||
|
|
||||||
|
CRC ReverseCRC(CRCPoly const *poly, CRC crc, byte b)
|
||||||
|
{
|
||||||
|
int i, highBit = poly->highBit;
|
||||||
|
|
||||||
|
for (i = 0; i < 8; i++) {
|
||||||
|
if (crc & highBit) /* highBit is 2^(poly->bits-1) */
|
||||||
|
crc = ((crc ^ poly->poly) << 1) ^ 1;
|
||||||
|
else
|
||||||
|
crc <<= 1;
|
||||||
|
}
|
||||||
|
return crc ^ b;
|
||||||
|
}
|
||||||
|
|
||||||
|
static void InitDigitsInv(char const *digits, signed char *digitsInv)
|
||||||
|
{
|
||||||
|
int i;
|
||||||
|
|
||||||
|
for (i = 0; i < 256; i++)
|
||||||
|
digitsInv[i] = -1;
|
||||||
|
for (i = 0; digits[i]; i++)
|
||||||
|
digitsInv[(byte)digits[i]] = i;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Returns the number of chars encoded */
|
||||||
|
int EncodeCheckDigits(EncodeFormat const *fmt, word32 num,
|
||||||
|
int numBits, char *dest)
|
||||||
|
{
|
||||||
|
int destLen = EncodedLength(fmt, numBits);
|
||||||
|
word32 digitMask = fmt->radix - 1;
|
||||||
|
int i;
|
||||||
|
|
||||||
|
for (i = destLen - 1; i >= 0; i--)
|
||||||
|
{
|
||||||
|
dest[i] = EncodeDigit(fmt, num & digitMask);
|
||||||
|
num >>= fmt->bitsPerDigit;
|
||||||
|
}
|
||||||
|
return destLen;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Returns 1 if there's an error */
|
||||||
|
int DecodeCheckDigits(EncodeFormat const *fmt, char const *src, char **endPtr,
|
||||||
|
int numBits, word32 *valuePtr)
|
||||||
|
{
|
||||||
|
word32 value = 0;
|
||||||
|
int digitValue;
|
||||||
|
int i = EncodedLength(fmt, numBits);
|
||||||
|
|
||||||
|
while (i--)
|
||||||
|
{
|
||||||
|
digitValue = DecodeDigit(fmt, *src++);
|
||||||
|
if (digitValue < 0)
|
||||||
|
{
|
||||||
|
/* Invalid digit found */
|
||||||
|
*valuePtr = 0;
|
||||||
|
if (endPtr)
|
||||||
|
*endPtr = NULL;
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
value = (value << fmt->bitsPerDigit) | digitValue;
|
||||||
|
}
|
||||||
|
*valuePtr = value;
|
||||||
|
if (endPtr)
|
||||||
|
*endPtr = (char *)src;
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
EncodeFormat const *FindFormat(char headerTypeChar)
|
||||||
|
{
|
||||||
|
EncodeFormat const * fmt = firstFormat;
|
||||||
|
|
||||||
|
while (fmt && fmt->headerTypeChar != headerTypeChar)
|
||||||
|
fmt = fmt->nextFormat;
|
||||||
|
return fmt;
|
||||||
|
}
|
||||||
|
|
||||||
|
void InitUtil()
|
||||||
|
{
|
||||||
|
/* teun: removed "{ }" for MS VC compile */
|
||||||
|
|
||||||
|
crcCCITTPoly.bits = 16;
|
||||||
|
crcCCITTPoly.poly = 0x8408;
|
||||||
|
crcCCITTPoly.highBit = 0x8000;
|
||||||
|
|
||||||
|
crc24Poly.bits = 24;
|
||||||
|
crc24Poly.poly = 0xdf3261;
|
||||||
|
crc24Poly.highBit = 0x800000;
|
||||||
|
|
||||||
|
crc32Poly.bits = 32;
|
||||||
|
crc32Poly.poly = 0xedb88320;
|
||||||
|
crc32Poly.highBit = 0x80000000;
|
||||||
|
|
||||||
|
InitCRCPoly(&crcCCITTPoly);
|
||||||
|
InitCRCPoly(&crc24Poly);
|
||||||
|
InitCRCPoly(&crc32Poly);
|
||||||
|
InitDigitsInv(hexDigits, hexDigitsInv);
|
||||||
|
InitDigitsInv(radix64Digits, radix64DigitsInv);
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Local Variables:
|
||||||
|
* tab-width: 4
|
||||||
|
* End:
|
||||||
|
* vi: ts=4 sw=4
|
||||||
|
* vim: si
|
||||||
|
*/
|
||||||
149
tools/util.h
Normal file
149
tools/util.h
Normal file
@ -0,0 +1,149 @@
|
|||||||
|
/*
|
||||||
|
* util.h -- Miscellaneous defines
|
||||||
|
*
|
||||||
|
* Copyright (C) 1997 Pretty Good Privacy, Inc.
|
||||||
|
*
|
||||||
|
* Written by Mark H. Weaver
|
||||||
|
*
|
||||||
|
* $Id: util.h,v 1.23 1997/11/12 23:28:56 mhw Exp $
|
||||||
|
*/
|
||||||
|
|
||||||
|
#ifndef UTIL_H
|
||||||
|
#define UTIL_H 1
|
||||||
|
|
||||||
|
typedef unsigned long word32;
|
||||||
|
typedef unsigned short word16;
|
||||||
|
typedef unsigned char byte;
|
||||||
|
|
||||||
|
#define FMT32 "%08lx"
|
||||||
|
#define FMT16 "%04x"
|
||||||
|
#define FMT8 "%02x"
|
||||||
|
|
||||||
|
#define TAB_CHAR '\244' /* Currency symbol, like o in top of x */
|
||||||
|
#define TAB_STRING "\244"
|
||||||
|
#define TAB_PAD_CHAR ' ' /* The fact that this is space has leaked. */
|
||||||
|
#define TAB_PAD_STRING " " /* It may not be freely changed. */
|
||||||
|
#define FORMFEED_CHAR '\245' /* Yen symbol, like = on top of Y */
|
||||||
|
#define FORMFEED_STRING "\245"
|
||||||
|
#define SPACE_CHAR '\267' /* Middle dot, or bullet */
|
||||||
|
#define SPACE_STRING "\267"
|
||||||
|
#define CONTIN_CHAR '\266' /* Pilcrow (paragraph symbol) */
|
||||||
|
#define CONTIN_STRING "\266"
|
||||||
|
|
||||||
|
#define BYTES_PER_LINE 60 /* When using radix 64 */
|
||||||
|
|
||||||
|
#define LINES_PER_PAGE 72 /* Exclusive of 2 header lines */
|
||||||
|
#define LINE_LENGTH 80
|
||||||
|
#define PREFIX_LENGTH 7 /* Length of prefix, including the space */
|
||||||
|
|
||||||
|
#define HDR_PREFIX_CHAR '-'
|
||||||
|
#define RADIX64_END_CHAR '-'
|
||||||
|
|
||||||
|
typedef struct EncodeFormat EncodeFormat;
|
||||||
|
typedef word32 CRC;
|
||||||
|
typedef word16 CRCFragment;
|
||||||
|
|
||||||
|
typedef struct
|
||||||
|
{
|
||||||
|
CRC table[256];
|
||||||
|
int bits;
|
||||||
|
CRC poly;
|
||||||
|
CRC highBit;
|
||||||
|
} CRCPoly;
|
||||||
|
|
||||||
|
struct EncodeFormat
|
||||||
|
{
|
||||||
|
EncodeFormat const *nextFormat;
|
||||||
|
char headerTypeChar;
|
||||||
|
char const * digits;
|
||||||
|
signed char const * digitsInv;
|
||||||
|
int bitsPerDigit;
|
||||||
|
int radix;
|
||||||
|
CRCPoly const * lineCRC;
|
||||||
|
CRCPoly const * pageCRC;
|
||||||
|
int runningCRCBits;
|
||||||
|
int runningCRCShift;
|
||||||
|
int runningCRCMask;
|
||||||
|
};
|
||||||
|
|
||||||
|
|
||||||
|
#define HDR_ENC_LENGTH 19 /* Length of encoded prefix on header */
|
||||||
|
|
||||||
|
#define HDR_VERSION_BITS 4
|
||||||
|
#define HDR_FLAG_BITS 8
|
||||||
|
/* Page CRC bits omitted, since it's not constant */
|
||||||
|
#define HDR_TABWIDTH_BITS 4
|
||||||
|
#define HDR_PRODNUM_BITS 12
|
||||||
|
#define HDR_FILENUM_BITS 16
|
||||||
|
|
||||||
|
|
||||||
|
/* Enough to hold one whole page of munged data */
|
||||||
|
/* There is no point making this excessively too large */
|
||||||
|
#define PAGE_BUFFER_SIZE 8192
|
||||||
|
|
||||||
|
#if PAGE_BUFFER_SIZE < (LINES_PER_PAGE + 2) * (LINE_LENGTH + PREFIX_LENGTH + 2)
|
||||||
|
#error PAGE_BUFFER_SIZE is too small
|
||||||
|
#endif
|
||||||
|
|
||||||
|
|
||||||
|
/* Header flags */
|
||||||
|
#define HDR_FLAG_LASTPAGE 0x01 /* Indicates last page of file */
|
||||||
|
|
||||||
|
|
||||||
|
#define elemsof(array) (sizeof(array)/sizeof(*(array)))
|
||||||
|
|
||||||
|
|
||||||
|
extern char const hexDigits[];
|
||||||
|
extern char const radix64Digits[];
|
||||||
|
|
||||||
|
extern signed char hexDigitsInv[256];
|
||||||
|
extern signed char radix64DigitsInv[256];
|
||||||
|
|
||||||
|
extern CRCPoly crcCCITTPoly, crc24Poly, crc32Poly;
|
||||||
|
|
||||||
|
extern EncodeFormat const hexFormat, radix64Format;
|
||||||
|
extern EncodeFormat const * firstFormat;
|
||||||
|
|
||||||
|
|
||||||
|
#define HexDigitValue(ch) hexDigitsInv[(byte)(ch)]
|
||||||
|
#define Radix64DigitValue(ch) radix64DigitsInv[(byte)(ch)]
|
||||||
|
|
||||||
|
/* Returns the number of chars needed to encode the given number of bits */
|
||||||
|
#define EncodedLength(fmt, numBits) \
|
||||||
|
(((numBits) + (fmt)->bitsPerDigit - 1) / (fmt)->bitsPerDigit)
|
||||||
|
#define EncodeDigit(fmt, value) ((fmt)->digits[value])
|
||||||
|
#define DecodeDigit(fmt, digit) ((fmt)->digitsInv[(byte)digit])
|
||||||
|
|
||||||
|
#define AdvanceCRC(poly, crc, b) \
|
||||||
|
((crc) >> 8) ^ (poly)->table[((crc) ^ (b)) & 0xFF]
|
||||||
|
|
||||||
|
#define RunningCRCFromPageCRC(fmt, pageCRC) \
|
||||||
|
(((pageCRC) >> (fmt)->runningCRCShift) & (fmt)->runningCRCMask)
|
||||||
|
|
||||||
|
|
||||||
|
CRC CalculateCRC(CRCPoly const *poly, CRC crc,
|
||||||
|
byte const *buffer, size_t length);
|
||||||
|
CRC ReverseCRC(CRCPoly const *poly, CRC crc, byte b);
|
||||||
|
|
||||||
|
/* Returns the number of chars encoded */
|
||||||
|
int EncodeCheckDigits(EncodeFormat const *fmt, word32 num,
|
||||||
|
int numBits, char *dest);
|
||||||
|
|
||||||
|
/* Returns 1 if there's an error */
|
||||||
|
int DecodeCheckDigits(EncodeFormat const *fmt, char const *src, char **endPtr,
|
||||||
|
int numBits, word32 *valuePtr);
|
||||||
|
|
||||||
|
EncodeFormat const *FindFormat(char headerTypeChar);
|
||||||
|
|
||||||
|
void InitUtil();
|
||||||
|
|
||||||
|
|
||||||
|
#endif /* !UTIL_H */
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Local Variables:
|
||||||
|
* tab-width: 4
|
||||||
|
* End:
|
||||||
|
* vi: ts=4 sw=4
|
||||||
|
* vim: si
|
||||||
|
*/
|
||||||
286
tools/yapp
Normal file
286
tools/yapp
Normal file
@ -0,0 +1,286 @@
|
|||||||
|
#!/usr/bin/perl
|
||||||
|
#
|
||||||
|
# Yet another preprocessor
|
||||||
|
#
|
||||||
|
# $Id: yapp,v 1.5 1997/10/24 07:51:05 mhw Exp $
|
||||||
|
#
|
||||||
|
|
||||||
|
%vars = ('' => '$');
|
||||||
|
@incPath = (".");
|
||||||
|
|
||||||
|
sub Error
|
||||||
|
{
|
||||||
|
print STDERR $_[0], "\n";
|
||||||
|
exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
sub VarSubst
|
||||||
|
{
|
||||||
|
my ($varName, $undefOkay) = @_;
|
||||||
|
|
||||||
|
if (defined($vars{$varName}))
|
||||||
|
{
|
||||||
|
return $vars{$varName};
|
||||||
|
}
|
||||||
|
elsif (!$undefOkay)
|
||||||
|
{
|
||||||
|
&Error("Undefined variable '$varName' in $fileName line $.");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
sub NullFilter
|
||||||
|
{
|
||||||
|
0;
|
||||||
|
}
|
||||||
|
|
||||||
|
sub IfFilter
|
||||||
|
{
|
||||||
|
local $_ = $_[0];
|
||||||
|
|
||||||
|
if (/^##else(\s+.*)?/)
|
||||||
|
{
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
elsif (/^##endif(\s+.*)?/)
|
||||||
|
{
|
||||||
|
return 2;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
sub DoFile
|
||||||
|
{
|
||||||
|
local $fileName = $_[0];
|
||||||
|
my $path;
|
||||||
|
local *FILE;
|
||||||
|
|
||||||
|
if ($fileName =~ m|^/|)
|
||||||
|
{
|
||||||
|
$path = $fileName;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
for $dir (@incPath)
|
||||||
|
{
|
||||||
|
if (-e "$dir/$fileName")
|
||||||
|
{
|
||||||
|
$path = "$dir/$fileName";
|
||||||
|
last;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if ($path eq "")
|
||||||
|
{
|
||||||
|
&Error("Can't find '$fileName', from $fileName line $.");
|
||||||
|
}
|
||||||
|
|
||||||
|
open(FILE, "<$path") || &Error("Can't open $path: $!");
|
||||||
|
&DoOpenFile(*FILE, *NullFilter, 0);
|
||||||
|
close(FILE) || die;
|
||||||
|
0;
|
||||||
|
}
|
||||||
|
|
||||||
|
sub DoPrepass
|
||||||
|
{
|
||||||
|
local ($_, $skipFlag) = @_;
|
||||||
|
|
||||||
|
return "" if /^###/;
|
||||||
|
s/\s*###.*//; # Strip comments
|
||||||
|
s/\${(\w+)}/&VarSubst($1, $skipFlag)/eg; # Do variable substitutions
|
||||||
|
$_;
|
||||||
|
}
|
||||||
|
|
||||||
|
sub DoOpenFile
|
||||||
|
{
|
||||||
|
local *FILE = $_[0];
|
||||||
|
local *filter = $_[1];
|
||||||
|
my $skipFlag = $_[2];
|
||||||
|
my $result;
|
||||||
|
local $_;
|
||||||
|
|
||||||
|
while (<FILE>)
|
||||||
|
{
|
||||||
|
$_ = &DoPrepass($_, $skipFlag);
|
||||||
|
if ($result = &filter($_))
|
||||||
|
{
|
||||||
|
return $result;
|
||||||
|
}
|
||||||
|
elsif (/^##(\w*)(\s+(.*))?/)
|
||||||
|
{
|
||||||
|
my ($cmd, $params) = ($1, $3);
|
||||||
|
|
||||||
|
if ($cmd =~ /^if/)
|
||||||
|
{
|
||||||
|
my $condition;
|
||||||
|
my $ifStartLine = $.;
|
||||||
|
|
||||||
|
if ($cmd eq "if")
|
||||||
|
{
|
||||||
|
if ($params =~ /^(\d+)\s*$/)
|
||||||
|
{
|
||||||
|
$condition = int($1);
|
||||||
|
}
|
||||||
|
elsif ($params =~ /^(\d+)\s*([=!]=|[<>]=?)\s*(\d+)\s*$/)
|
||||||
|
{
|
||||||
|
my ($left, $op, $right) = ($1, $2, $3);
|
||||||
|
|
||||||
|
$condition = eval($left . $op . $right);
|
||||||
|
}
|
||||||
|
elsif ($params =~ /^(\S+)\s*(eq|ne)\s*(\S+)\s*$/)
|
||||||
|
{
|
||||||
|
my ($left, $op, $right) = ($1, $2, $3);
|
||||||
|
|
||||||
|
$left =~ s/([\\'])/\\$1/g;
|
||||||
|
$right =~ s/([\\'])/\\$1/g;
|
||||||
|
$condition = eval("'$left' $op '$right'");
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
&Error("Invalid ##if params: '$params' " .
|
||||||
|
"in $fileName line $.");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
elsif ($cmd =~ /^ifn?def$/)
|
||||||
|
{
|
||||||
|
if ($params =~ /^(\w+)\s*$/)
|
||||||
|
{
|
||||||
|
$condition = defined($vars{$1});
|
||||||
|
$condition = !$condition if ($cmd eq "ifndef");
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
&Error("Invalid ##$cmd param: '$params' " .
|
||||||
|
"in $fileName line $.");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# Do main body of if
|
||||||
|
$result = &DoOpenFile(*FILE, *IfFilter,
|
||||||
|
$skipFlag || !$condition);
|
||||||
|
|
||||||
|
if ($result == 1) # an '##else' was found
|
||||||
|
{
|
||||||
|
# Handle else
|
||||||
|
$result = &DoOpenFile(*FILE, *IfFilter,
|
||||||
|
$skipFlag || $condition);
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($result == 1) # a second '##else' was found
|
||||||
|
{
|
||||||
|
&Error("Two ##else's in a row in $fileName line $.");
|
||||||
|
}
|
||||||
|
elsif ($result == 0) # EOF was encountered
|
||||||
|
{
|
||||||
|
&Error("Unterminated ##if " .
|
||||||
|
"in $fileName line $ifStartLine");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
elsif ($cmd eq "include")
|
||||||
|
{
|
||||||
|
if ($skipFlag)
|
||||||
|
{
|
||||||
|
}
|
||||||
|
elsif ($params =~ /^"(.*)"\s*$/)
|
||||||
|
{
|
||||||
|
my $incFile = $1;
|
||||||
|
|
||||||
|
&DoFile($incFile);
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
&Error("Invalid ##include params: '$params'");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
elsif ($cmd eq "set")
|
||||||
|
{
|
||||||
|
if ($params =~ /^(\w+)=<<(")(.*)"\s*$/ or
|
||||||
|
$params =~ /^(\w+)=<<(')(.*)'\s*$/)
|
||||||
|
{
|
||||||
|
my $varName = $1;
|
||||||
|
my $quoteChar = $2;
|
||||||
|
my $endTag = $3 . "\n";
|
||||||
|
my $value;
|
||||||
|
|
||||||
|
while (<FILE>)
|
||||||
|
{
|
||||||
|
if ($_ eq $endTag)
|
||||||
|
{
|
||||||
|
chop $value;
|
||||||
|
last;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
if ($quoteChar eq '"')
|
||||||
|
{
|
||||||
|
$_ = &DoPrepass($_, $skipFlag);
|
||||||
|
}
|
||||||
|
$value .= $_;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (!$skipFlag)
|
||||||
|
{
|
||||||
|
$vars{$varName} = $value;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
elsif ($params =~ /^(\w+)="(.*)"\s*$/ or
|
||||||
|
$params =~ /^(\w+)=(\S*)\s*$/)
|
||||||
|
{
|
||||||
|
if (!$skipFlag)
|
||||||
|
{
|
||||||
|
$vars{$1} = $2;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
&Error("Invalid ##set command: '$params'");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
&Error("Unrecognized command: '$_'");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
elsif (!$skipFlag)
|
||||||
|
{
|
||||||
|
print;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
$optEnable = 1;
|
||||||
|
|
||||||
|
foreach (@ARGV)
|
||||||
|
{
|
||||||
|
if ($optEnable and /^-/)
|
||||||
|
{
|
||||||
|
if (/^--$/)
|
||||||
|
{
|
||||||
|
$optEnable = 0;
|
||||||
|
}
|
||||||
|
elsif (/^-D(\w+)=(.*)$/)
|
||||||
|
{
|
||||||
|
$vars{$1} = $2;
|
||||||
|
}
|
||||||
|
elsif (/^-I(.*)$/)
|
||||||
|
{
|
||||||
|
unshift @incPath, $1;
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
&Error("Unrecognized option: '$_'");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
&DoFile($_);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#
|
||||||
|
# vi: ai ts=4
|
||||||
|
# vim: si
|
||||||
|
#
|
||||||
48
tools/yapp.doc
Normal file
48
tools/yapp.doc
Normal file
@ -0,0 +1,48 @@
|
|||||||
|
YAPP is a simple macro preprocessor designed to do minor tweaking to
|
||||||
|
another program's inputs.
|
||||||
|
|
||||||
|
In its input, anything of the form ${foo} is expanded with the variable
|
||||||
|
named foo. It is an error if ${foo} is not defined.
|
||||||
|
If you need to escape a dollar sign for some reason, the variable
|
||||||
|
with the empty string name , ${}, has the value "$".
|
||||||
|
|
||||||
|
The result of macro expansion is *not* re-expanded. Expansion is done only
|
||||||
|
when definitions are made.
|
||||||
|
|
||||||
|
After variable expansion, lines are checked to see if they are control lines.
|
||||||
|
Control lines begin with ## (after optional leading whitespace) All such lines are deleted and
|
||||||
|
do not appear in the output. ### is a comment. Other options
|
||||||
|
are:
|
||||||
|
|
||||||
|
##set variable=value
|
||||||
|
|
||||||
|
value may have one of the following forms:
|
||||||
|
token: Trailing whitespace is stripped. The token may not contain
|
||||||
|
any whitespace. Use quotes if it's complicated.
|
||||||
|
"string": The string may have embedded quotes, and whitespace after
|
||||||
|
the closing quote.
|
||||||
|
<<"DELIM": This is a here-document, and the value is all of the following
|
||||||
|
lines up until, but not including, the newline that precedes a line
|
||||||
|
that consists soley of DELIM, for any DELIM string.
|
||||||
|
The Delim must be in quotes. You have two options:
|
||||||
|
"DELIM": Expand macros in the body of the here-document.
|
||||||
|
'DELIM': Do not expand macros in the here-document.
|
||||||
|
|
||||||
|
##include "filename": Insert the named file in place of the current line.
|
||||||
|
|
||||||
|
##if num == num
|
||||||
|
##if num != num
|
||||||
|
##if num < num
|
||||||
|
##if num > num
|
||||||
|
##if num <= num
|
||||||
|
##if num >= num
|
||||||
|
##if token eq token
|
||||||
|
##if token ne token
|
||||||
|
##ifdef symbol
|
||||||
|
##ifndef symbol
|
||||||
|
##else
|
||||||
|
##endif
|
||||||
|
You can figure this one out. Macros in between are expanded as usual
|
||||||
|
(so the ##else or ##endif may be in a macro expansion), but the result
|
||||||
|
is ignored. String comparison is allowed only between simple words.
|
||||||
|
#ifdef symbol is true if ${symbol} is defined.
|
||||||
Loading…
Reference in New Issue
Block a user