Chapter 7
Friedman
Books must follow sciences, and not sciences books.
--Francis Bacon
William F. Friedman, who is best known for leading a team that broke the Japanese Purple Code at the beginning
of World War II, was born in 1891 of well educated Rumanian parents. Mostly by accident, and through
scholarships in various fields, he became an agricultural geneticist and a graduate of Cornell University.
Meanwhile George Fabyan, who had made a fortune in the textile business, had been operating scientific
laboratories on his Riverbank estate near Geneva, Illinois. Friedman was hired by Fabyan to work on genetic
experiments. Known as "The Colonel," he had very little education but he was interested in agriculture, chemistry,
acoustics and cryptology; he had established departments for each discipline in the buildings on his 500 acre
property. Also working at Riverbank was Mrs. Elizabeth Wells Gallup. She had convinced Fabyan, and many
others, that there was a cipher concealed in the typography of first editions of Shakespeare, Bacon, and other
writers of the Elizabethan and Jacobean era.
She relied on a description of a system that Francis Bacon had pub lished called the Bi-literarie Cipher and which
he had illustrated by using two slightly different styles of engraved (not typeset) printing. Mrs. Gallup said that
she had found two such diverse fonts of type, as set in the printing of such books and then, financed by George
Fabyan, she worked diligently to decipher the messages. David Kahn, in his consummate work The Codebreakers
[@], describes the circumstances, and here enter love, romance and marriage versus the imagined evils of
Elizabethan courtly life:
"Neither she [Elizebeth Smith, an English major and a former librarian] nor Friedman had given any particular
previous thought to cryptology, but they began to get personally interested in the work. It is yet another of the
ironies of cryptologic history that the interest of the two foremost cryptologists was aroused by a false doctrine--a
doctrine, moreover, against which they later were to wage a lifetime battle. For at table at the Riverbank cottages
they heard gaudy tales of lusty Elizabethan life, of the not-so-Virgin Queen, of courtier's intrigues and the secret
histories of the great names of English history--all actually invalid decipherments of Shakespeare's plays tending to
prove that Bacon had written them, and related by the gentle, upright but self-deluded woman who had
deciphered them, Mrs. Elizabeth Wells Gallup. These stories stirred Friedman's dormant interest; he began to do
some of the cryptology, and inevitably its puissant magic seeped like the fume of poppies into his mind and spirit
and intoxicated him..."
"He soon found himself head of the Department of Ciphers as well as the Department of Genetics at Riverbank.
The attraction he felt for cryptology was reinforced by the attraction he felt for a cryptologist: the quick-witted and
sprightly Miss Smith. In May of 1917 they were married and started the most famous husband-and-wife team in
the history of cryptology."
The Friedmans went on into other, more respectable, fields of this science, and with remarkable success. Toward
the end of the First World War the British were about to adopt a cipher device invented by one Vincent Pletts of
their War Office cryptanalytic bureau. They dispatched to the Riverbank Laboratories five short messages to be
tested for security. Kahn describes the result:
"The [Pletts] machine was a modification of the Wheatstone apparatus, proposed as a field cipher. So highly did
the British regard it that one argument advanced against its adoption was that if the Germans captured one and
adopted it, the Allies would no longer be able to solve enemy messages! Friedman, however, at once recovered the
keyword CIPHER to one of the mixed alphabets. But he could not seem to get anywhere with the other keyword
and, stymied, he resorted to a bit of psychological cryptanalysis. He turned to the new Mrs. Friedman and asked
her to make her mind a blank.
"'Now,' he went on, 'I want you to tell me the first word that comes into your mind when I say a word.' He paused.
"'Cipher,' he said.
"'Machine,' she replied.
"It turned out to be the very key desired. Three hours after Friedman received the cryptograms, their plaintexts
were being cabled to London. (The first one read, in a phrase dear to proud inventors, This cipher is absolutely
undecipherable .) Needless to say, it ended consideration of the Pletts device for Allied use."
As will be seen later in this book, enciphering the word cipher , or some variation thereof, is unusual and perilous
but not in antiquity unprecedented. To do so is more like a favor.
At Riverbank, Friedman went on to write successive monographs on the subject of cryptography and showed,
again and again, new ways to crack the most difficult ciphers. Kahn says of this work:
"Riverbank Publication No. 22, written in 1920 when Friedman was 28, must be regarded as the most important
single publication in cryptology. It took the science into a new world. Entitled The Index of Coincidence and Its
Applications in Cryptography , it described the solution of two complicated cipher systems. . .In it, Friedman
devised two new techniques. One was brilliant. It permitted him to reconstruct a primary cipher alphabet without
having to guess at a single plaintext letter. But the other was profound. For the first time in cryptology, Friedman
treated a frequency distribution as an entity, as a curve whose several points were causally related, not just as a
collection of individual letters that happen to stand in a certain order for noncausal (historical) reasons, and to this
curve he applied statistical concepts. The results can only be described as Promethean, for Friedman's stroke of
genius inspired the numerous, varied, and vital statistical tools that are indispensable to the cryptology of today.
"Before Friedman, cryptology eked out an existence as a study unto itself, as an isolated phenomenon, neither
borrowing from nor contributing to other bodies of knowledge. Frequency counts, linguistic characteristics, Kasiski
examinations--all were peculiar and particular to cryptology. It dwelt a recluse in the world of science. Friedman
led cryptology out of this lonely wilderness and into the broad rich domain of statistics. . .This is why Friedman
has said, in looking back over his career, that The Index of Coincidence was his greatest single creation. It alone
would have won him his reputation. But in fact it was only the beginning."
After the war Friedman resigned from the Riverbank Laboratories and began teaching and writing about
cryptography, first with the Army Signal Corps and later in civil service. His text, Elements of Cryptanalysis ,
became a standard. At his suggestion, the Army adopted the M-94 Jefferson- Bazeries cylinder cipher device for
field use. He tested various other instruments which were claimed to produce unbreakable ciphers, and he broke
them all. Kahn describes one of them:
"Most difficult of these was the machine with five wired codewheels --rotors--invented by Edward H. Hebern,
whose principle is today [1967] the most widely used in high level cryptography. Each of the rotors generates a
progressive cipher, and in 1925 Friedman devised the kappa test and extended his Index of Coincidence analyses
to determine the order and starting positions of the rotors. The five progressive ciphers intertwine in a cipher of
hideous nightmare complexity, but in a later solution Friedman sorted them out and reconstructed the wiring of
the rotors. This work was of the utmost importance, for it laid the foundations for the PURPLE machine solution
and for today's many solutions of modern rotor machines. The technique was far in advance of its time. So far as is
known, not another cryptanalyst on the globe could duplicate it--and none did, apparently, for more than two
decades. With this solution of Friedman's, world leadership in cryptology passed to America."
In 1929 the U. S. State Department abandoned cryptographic research. The Secretary of State, Henry L. Stimson,
had declared that "Gentlemen do not read other people's mail." Quickly the Army established the Signal
Intelligence Service and named Friedman as its first director. His later conquest of the Japanese PURPLE code
(actually a cipher) is described by David Kahn:
"...The solution of the PURPLE machine was, in fact, the greatest feat of cryptanalysis the world had yet known.
"The cipher machine that Americans knew as PURPLE bore the resounding official Japanese title of 97-shiki O-bun
In-ji-ki. This meant Alphabetical Typewriter '97, the '97 an abbreviation for the year 2597 of the Japanese calendar,
which corresponds to 1937. The Japanese usually referred to it simply as "the machine" or as "J", the name given it
by the Imperial Japanese Navy, which had adapted it from the German Enigma cipher machine and then had lent it
to the Foreign Ministry, which in turn had further modified it. Its operating parts were housed in a drawer-sized
box between two big black electrically operated Underwood typewriters, which were connected to it by 26 wires
plugged into a row of sockets called a plugboard. To encipher a message, the cipher clerk would consult the thick
YU GO book of machine keys, plug in the wire connections according to the key for the day, turn the four disks in
the box so the numbers on their edges were those directed by the YU GO, and type out the plaintext. His machine
would record that plaintext while the other, getting the electric impulses after the coding box had twisted them
through devious paths, would print out the ciphertext. Deciphering was the same, though the machine irritatingly
printed the plaintext in the five-letter groups of the ciphertext input. . .The guts and heart of the machine were the
plugboards and the coding wheels. They diverted the current flowing along the connections from the input
typewriter to the output one so that when the a key was depressed on the input keyboard, an a would not be
typed on the output machine. The diversion began with the plugboard connections. If the coding box were not
present, a plugboard wire would take the electric impulse from the a key of the plaintext typewriter and bring it
directly to, say, the R typebar of the ciphertext machine. Other wires would similarly connect the plaintext keys to
noncorresponding ciphertext typebars. This would automatically produce a cipher, though a very elementary one.
Each time plaintext a was depressed ciphertext R would appear. So simple a system affords no security. The
plugboard connections can be changed from message to message, or even within a message, but this does not
noticeably augment the system's strength.
"Here is where the four coding wheels came in. Interposed between the plugboard of the plaintext typewriter and
that of the ciphertext machine, they were shifted constantly with respect to one another by their supporting
assembly. The enciphering current had to traverse their winding wire paths to get from one typewriter to the other,
and the constant shifting continually set up different paths. Thus impulses from a given plaintext letter were
switched through the box along ever varying detours to emerge at ever differing cipher-text letters. Plaintext a
might be represented in a long message by all 26 letters. Conversely, any given ciphertext letter might stand for
any one of 26 plaintext letters. Switches on the coding wheels could be flicked one way or the other; this
constituted part of the key and was done by the code clerk before enciphering. Usually the plugboard connections
were changed each day.
"These factors united to produce a cipher of exceptional difficulty. The more a cipher deviates from the simple
form in which one ciphertext letter invariably replaces the same plaintext letter, the harder it is to break. A cipher
might replace a given plaintext letter by five different ciphertext letters in rotation, for example. But the
Alphabetical Typewriter produced a substitution series hundreds of thousands of letters long. Its coding wheels,
stepping a space--or two, or three, or four--after every letter or so, did not return to their original positions to
recreate the same series of paths, and hence the same sequence of substitutes, until hundreds of thousands of
letters had been enciphered.
"...As William Friedman recalled, "When the PURPLE system was first introduced it presented an extremely
difficult problem on which the Chief Signal Officer asked us to direct our best efforts. After work by my associates
when we were making very slow progress, the Chief Signal Officer asked me personally to take a hand. I had been
engaged largely in administrative duties up to that time, so at his request I dropped everything else that I could
and began to work with the group.
"...Lighting his way with some of the methods that he himself had developed, he led the cryptanalysts through the
murky PURPLE shadowland. He assigned teams to test various hypotheses. Some prospected fruitlessly, their
only result a demonstration that success lay in another direction. Others found bits and pieces that seemed to make
sense.
. .Errors, caused perhaps by garbled interceptions or simple mistakes in the cryptanalysis, jarred these delicate
analyses and delayed the work. But slowly it progressed. A cryptanalyst, brooding sphinxlike over the cross-ruled
paper on his desk, would glimpse the skeleton of a pattern in a few scattered letters; he tried fitting a fragment
from another recovery into it; he tested the new values that resulted and found that they produced acceptable
plaintext; he incorporated his essay into the over-all solution and pressed on. Experts in Japanese filled in missing
letters; mathematicians tied in one cycle with another and both to the tables. Every weapon of cryptanalytic
science--which in the stratospheric realm of this solution drew heavily upon mathematics, using group theory,
congruences, Poisson distributions--was thrown into the fray.
"Eventually the solution reached the point where the cryptanalysts had a pretty good pencil-and-paper analog of
the PURPLE machine. S.I.S. then constructed a mechanism that would do automatically what the cryptanalysts
could do manually with their tables and cycles. They assembled it out of ordinary hardware and easily available
pieces of communication equipment, such as the selector switches used for telephones. It was hardly a beautiful
piece of machinery, and when not running just right it spewed sparks and made loud whirring noises. Though the
Americans never saw the 97-shiki O-bun In-ji-ki, their contraption bore a surprising physical resemblance to it, and
of course exactly duplicated it cryptographically..."
Beginning in August, 1940 the S.I.S. began regularly delivering the deciphered Japanese messages, which were
code-named MAGIC.
Friedman humbly gave the credit for the solution to his team. Some have tried to take away that honor, claiming
that the improved German ENIGMA machine, secretly obtained by the British in 1938 from two Polish
cryptographers, and later confided to the United States, was the basis for their cryptographic victory. But both that
device and the Japanese version were based on earlier machines which had been sold for commercial use as long
before as 1923. David Kahn continues:
"Friedman was, despite his partial disclaimer, the captain of that team. The solution had taken a terrific toll. The
restless turning of the mind tormented by a puzzle, the preoccupation at meals, the insomnia, the sudden
wakening at midnight, the pressure to succeed because failure could have national consequences, the despair of
the long weeks when the problem seemed insoluble, the repeated dashings of uplifted hopes, the mental shocks,
the tension and the frustration and the urgency and the secrecy all converged and hammered furiously upon his
skull. He collapsed in December, 1940. After three and a half months in Walter Reed General Hospital recovering
from the nervous breakdown, he returned to S.I.S. on shortened hours, working at first in the more relaxed area of
cryptosecurity. By the time of Pearl Harbor he was again able to do some cryptanalysis, this time of German
systems..."
The usefulness of MAGIC, the term for the decryption of messages by the PURPLE machine, continued
throughout World War Two. The Japanese, though unsuspecting, changed the coding of their wheels from time to
time but they never abandoned their general system. Our admirals knew the current position of every group of the
Japanese battle fleet; the messages were often deciphered before the enemy commanders received them. The
advantage thus acquired was enormous.
Friedman received almost every civilian decoration that the U.S. government could bestow, including the
Commendation for Exceptional Civilian Service, The Medal of Merit, presented to him by President Truman
himself, and then, from Allen W. Dulles, director of Central Intelligence, the National Security Medal. MAGIC
alone did not win the war with Japan, but it certainly shortened it and saved thousands of lives.
He retired in 1955. Surprisingly enough, he became a historian of what he considered to be false cipher methods.
Kahn says:
"As the pressure of his duties declined, Friedman and his wife returned to the cryptologic field that had gotten
them started--the Baconian ciphers. They summed up the experience of a lifetime in a long and exhaustive report
that won them the Folger Shakespeare Library literary prize in 1955. After his retirement, they collaborated in
preparing this work for 1957 publication by the Cambridge University Press as The Shakespearean Ciphers
Examined [@]. While, as The New York Times Book Review accurately said, they buried "these
pseudocryptograms beneath a mass of evidence as crushing as an avalanche," they also introduced their readers to
a rogues' gallery of pseudocryptologists not to be met elsewhere in literature. The Friedmans here display a rather
surprising-- surprising to one who has perused only his technical writing--wit and talent for personality sketches."
In the Preface to his book, Friedman acknowledged his thanks to the officers of the London Francis Bacon Society
during his visits to their headquarters, Canonbury Tower, in 1953 and again in 1954. He refused, Cmdr. Martin
Pares (the Chairman) later said, to become a member. Yet one cannot read into the Friedman's book any outright
denial of the possibility that Francis Bacon had a hand in composing the Shakespeare manuscripts; much less is
there an evidentiary refutation of that theory.
Neither does he seriously attack the historical evidence favoring the Bacon hypothesis which exists outside of the
alleged Baconian "cipher solutions." One must credit that he and his wife Elizebeth had read deeply in that
spectrum of legitimate literary research wherein the authorship is seriously questioned.
In their first chapter the Friedmans sum up a few of the historical arguments raised against the authorship of the
Works by William Shakespeare, and most briefly mention the names and grounds for nominating several other
Elizabethans as being likely candidates. They hardly could cover the field in so few pages and, after all, they were
writing a book on cryptography. They conclude, "In fact the historical argument can never produce certainty either
way: there is always a counter-argument, always an appeal to the lack of evidence, a counter-interpretation of
what evidence there is, much inference, some coincidence and, despite Mr. Percy Allen [a spiritualist], no voice
from the grave...It is with relief that we turn to the more certain ground of cryptology. . .where, if properly
handled, it is a matter on which great things are decided. . ."
What the Friedmans do object to are claims for cipher in the Shake speare Works where none can be proven in a
respectable scientific manner. More than that, they object to the imaginative "history" derived from the decryption
of such ciphers. Nevertheless, I have quoted in this book extracts from thirty years of Baconiana, the journal of the
Francis Bacon Society, and I doubt that they would have seriously controverted much that is included there. But
that is not to say that they might agree with every allegation ever printed in Baconiana. Many Baconians, within
their field (and one that is beyond the verge of organized academia) have been and still are genuine private
scholars. The modern sort has drawn away from the enthusiastic errors of turn-of-the-century Baconians; the
emphasis on now-disproven cryptographic methods has mostly turned elsewhere, toward the historical and
literary arguments which, compared to those of the Stratfordians, beckon rather more persuasively. The silent
doubters, the moles within the learned disciplines are growing in number, to the consternation of a shrinking
group of others that Sam Clemens referred to as troglodytes. Sam's word, not mine.
It is worth observing that, before the Second World War, and espe cially before the Friedmans, the science of
cryptography was almost unknown to the universities and to the public. Except for the rare and scattered and
concealed professional practitioners, there were hardly any authorities for those interested to consult. Where it was
taught, it was taught secretly. Books explaining cryptography were mostly out of print and never had much
circulation. The casual reader became aware of the topic through Herbert O. Yardley's book, The American Black
Chamber which was published in 1931. The U.S. State Department had closed its own cipher room in 1929. There
was really no way for a reader to make a sophisticated judgment of the cipher "systems" which were invented.
Very possibly some of the authors of these methods, in their ardor, had no better way to innocently judge their
own creations. It is too bad, but many of them actually harmed their cause.
The Friedmans, using wry but cheerful humor, took aim at the Baconian crypto-cryptologists and sank their frail,
poorly armed, mostly 19th century vessels. The litany of the names of the drowned and the dates of their too-early
ventures into combat with the forces of science and mathematics, not to mention the Friedmans, is a grievous
sorrow; they sailed forth almost unarmed. To wit: Ignatius Donnelly, 1887; Dr. Orville Owen, 1893; William Stone
Booth, 1909; Sir Edward Durning-Lawrence, 1910; Walter Arensberg, 1921; Frank and Parker Woodward, 1923;
Elizabeth Wells Gallup, 1899; Mrs. Henry Pott, 1891. Their bones, already bleached, were exhumed, sorted,
categorized, mounted, and illuminated by the Friedmans in their entertaining treatise. (A much longer version of
their original manuscript exists in the Folger Shakespeare Library, but the curator cannot permit quotations
because of copyright laws.)
Thus the Friedmans spoke mostly about those who had written on the subject prior to or about the time that they
left the service of Col. George Fabyan at the Riverbank Laboratories in 1921. We too must turn with a shudder,
though with ill-concealed grief, away from the plight of those pitiable unfortunates who became their bleeding
victims. Since the Friedmans published The Shakespearean Ciphers Examined , the rule in academia for someone
tempted to enter the lists has become "Don't publish lest you perish."
And the Friedmans were right. They had been hoodwinked by the wealthy and devious Col. Fabyan and wanted to
correct the records. That Col. Fabyan (whose military rank was self-imposed) was, nevertheless, a fine judge of
intellect. Without him, without board and room and $50 a month and the fascination of cryptography, the
Friedmans might have gone to work as geneticists for Henry Wallace on his corn farm. A small bronze memorial to
George Fabyan might be appropriate to nail on the wall of the Marine Barracks at Midway Island.
The Friedmans touch us easily with their pointer of logic; they explain what must be done, if there is anything to be
done. Gently they elucidate the rules by which to judge a valid cipher. These will be paraphrased as follows:
-
The first rule is that a cipher system must have rules; to use such devices one must methodically follow the
rules. A cipher unit (the "cipher-text") is formed of one letter of the alphabet, and it must correspond to
another letter in the deciphered text (the "plaintext"). -
There may be, and there often is, more than one rule; if there is a key there may be more than one key. But
the rules and the keys must be unambiguous and they must be precisely applied. -
Finding the right solution to a cryptogram is not a matter of opinion; there must be scientific confirmation.
Every decipherer must reach the same conclusion. -
The plaintext solution must make sense; the message must say something. However, a name alone may be
sufficient to establish the authorship of the work in which the cipher is found. -
The cryptologist must become convinced that the solution could not have happened by accident. If the odds
against a chance occurrence of the solution are one in a thousand million, his confidence in the decipherment
is justified. -
There must be a key to the cryptogram, such as a modification or rearrangement of the alphabet which is
used to reach the solution, and the cryptogram must be of a reasonable length. For a mono-alphabetic
substitution cipher, about 25 letters are necessary before the cryptanalyst can be sure that his is the only
possible solution. -
Whether or not the message may be considered to be long enough for theoretical justification, corroboration
of the cipher method, by the analysis of other messages which yield to the same system, provides conclusive
proof. -
An allowance may be made for the mistakes of the encipherer. Every solution must be considered on its own
merits, but ordinarily not more than about ten percent of such errors are tolerable. -
To change the key, or alphabet, on a given signal is a common and accepted practice in cryptography.
-
Acrostics, in which the first, last or other specific letters of succeeding words constitute the cipher letters,
have often been found in literature and are an acceptable form of encipherment. -
Discovery of an enciphered name (however spelled) leaves no doubt that the author of the open text must
also have been responsible for the message; evidence of this kind must be taken as decisive. -
The rules for selecting the letters of an acrostic must be invariable and the letters must be chosen in a
particular order. One cannot at random select letters, whether in order or not; nor may one select letters in a
particular order and then rearrange them until an anagram is found that represents a message or name. -
Acrostics were popular in Elizabethan literature in which spelling was, compared to modern orthography,
very diverse. Even proper names were spelled variably. The Friedmans go so far as to say this: "We should
not be surprised if it is claimed that acrostics appear in Shakespeare's works, for they abounded in the
literature of the time; nor should we be surprised if these devices concern the authorship of the works, for
they have often been used to this end. We should even be tolerant of variable and erratic spelling, for this was
to some extent a common Elizabethan practice." -
The Friedmans particularly do not insist that the existence of a cipher be clearly signaled in the open text. In
fact they consider it unreasonable to expect to find signs calling attention to concealed information. They
quote Francis Bacon himself, who wrote "The vertues of cyphars whereby they are to be preferred are three;
that they be not laborious to write and reade; that they be impossible to decypher; and in some cases, that
they be without suspicion." [steganographic] The Friedmans' view was that "One does not put something in a
secret hiding-place and then put up a sign saying, `Notice: Secret Hiding-place'."
So, they do not "demand any external guide to the presence of the secret texts." As to the additional probative
value of an actual external beacon and guide, they are silent. Neither do the Friedmans comment upon the reverse
situation, where a reference might be made in the deciphered information to the open text, or vice-versa.
The book, if I can read between the lines of their scorching logic, was written with a fond sigh. It was a masterful
lecture, addressed to a few of the (otherwise often credible) Baconian scholars, warning that they had stepped off
the paths of their expertise. It was the last professional thing the Friedmans did, so far as I know, and they placed
their more extended manuscript in the vaults of the Folger Shakespeare Library. They had apprenticed in an
unsuccessful search for the validation of a faulty Shakespearean cryptogram and they denied any verification, yet
they may have detected an odor of cipher there. As good scientists must, they refrained from any conviction. I
wonder if this lengthy paper was not both a nostalgic review of their early careers, and an exercise in instruction
for others who might dare to swim in such dark pools.