**Abridged**

A. M. Hasofer

Emeritus Professor of Statistics

University of New South Wales

Sydney, Australia

I first became interested in the Torah codes when I read in 1987 an expository paper by Michelson in B'Or Ha'Torah [4] where he presented statistical analyses which he claimed supported the existence of hidden codes in the Torah. As an Orthodox Jew and an experienced statistician who had been previously exposed to pseudo-scientific claims based on numerology, I was gravely concerned. I wrote a paper entitled "Codes in the Torah: A Rejoinder" in which I argued that:

- Michelson's conclusions were not justified within the framework of accepted statistical inference procedures,
- The whole approach could be inconsistent with the teachings of our Sages.

The paper was eventually published in 1993 [2], and was followed by a comprehensive discussion [5], which did nothing to change my mind on the subject.

Since 1993 much has occurred in codes research.
But the most publicized development was undoubtedly the publication in 1994
in *Statistical Science* of a paper by Witztum, Rips and Rosenberg
entitled "Equidistant Letter Sequences in the Book of Genesis"
[6]. In the rest of this paper, Equidistant Letter Sequences will be
denoted by ELS's. Recently, a number of mathematicians have closely examined
he "scientific" claims of the codes research and have concluded
that they suffer from major flaws. In particular, Dr. B. McKay of the
Australian National University and Dr. D. Bar-Natan of the Hebrew University
in Jerusalem have carried out large-scale computer experiments that have
fully supported the concerns I voiced in my 1992 paper [2] about the
statistical aspects of the research. A full account of their research
together with links to other information sources on the subject can be found
at the following Internet site:

cs.anu.edu.au/~bdm/dilugim/torah.html

This paper is an update of my original paper [2].
My basic position remains unchanged. As I wrote in the Reply to Discussion
[5]: "It has always been the Jewish way to accept as authentic only
those teachings that emanated from *Tsaddikey ha'Dor *(the righteous
leaders of the generation) because we know that their teachings were
inspired by *Ruach ha'Kodesh *(the spirit of holiness). Why should
we suddenly abandon this holy tradition and accept as authentic the efforts
of laymen in partnership with a dumb computer?"

The article "Jesus Codes: Uses and abuses"
written in 1997 by Daniel Mechanic in consultation with Doron Witztum
and Harold Ganz states "It is the specifics of the methodology that
make it even possible to verify that the "Famous Rabbis" Codes
were deliberately encoded in the Torah." In other words, Witztum
and his collaborators developed a "black box" detailed in the
*Statistical Science *article [6], through which to pass collections
of ELSs, and which they claim separates "genuine codes" from
"meaningless coincidences", whatever they may refer to.
Furthermore, they claim that their black box is based on standard statistical
methodology.

In a presentation to the Israel Academy of Science
in 1996 they made an even stronger claim. They wrote: "The purpose
of the present research is to see if it is possible to prove the existence
of a "hidden text" in a formal, mathematical way, *without
relying in any way on knowledge received by tradition.*"

In the remainder of this paper, I will argue the three following points:

- The
*Statistical Science*paper's methodology for separating "genuine codes" from "meaningless coincidences" is not valid according to accepted scientific standards. - Patterns that apparently pass the Witztum
*et al*"black box" test can be found in other literary texts. - The only difference between the Witztum
*et al*Famous Rabbis test on Genesis and a similar successful test on a part of the Hebrew translation of Tolstoy's War and Peace is whether the data were chosen*a priori*or not. In other words, what Witztum*et al*are asking the public is to accept on faith their honesty. This also applies to the other tests made by Witztum*et al*.

It is clear from the quote in the preceding Section
that Witztum holds that the only way to demonstrate to the skeptics that
a word pattern is a genuine code is by using the specifics of the
methodology employed for the "Famous Rabbis" experiment
published in the *Statistical Science* paper "Equidistant
letter sequences
in the book of Genesis." In what follows I shall refer to that paper
as WRR.

Much has been written about the fact that some
renowned mathematicians had been impressed by the early code work and that
the paper was published in a respectable scientific journal. But most of
the early support has by now evaporated. The standing of the paper has
been even further eroded by its publication *in extenso* by Drosnin in
his infamous book "The Bible Code" [1] and the wild claims he
and others have made about it.

I have put my name to a petition stating that the
above paper has not established a *prima facie *case for its claims.
In addition, I have submitted to *Statistical Science *for
publication a rebuttal paper entitled "A Statistical Critique of the
Witztum *et al *paper". Preprints are available from me. My paper
is of course technical in nature. But I would like to summarize in this
paper the main points of my criticism.

The data of the WRR paper consist of two lists of famous rabbis, who all lived long after the Torah was given, together with their birth dates and dates of death. A distance measure between the names and the dates is set up. The paper claims that the names and corresponding dates are surprisingly close.

The central task of the WRR paper is to carry out a test of hypothesis. Now it is well known to any one who has done a first University course in Statistics that such a test requires the statement (before the experiment) of two hypotheses: the null hypothesis, and the alternative hypothesis. The two hypotheses are compared, and the more likely one is accepted in the light of the experimental outcome. Usually, the test is deemed successful if the null hypothesis is rejected in favour of the alternative.

Now in the WRR paper, there is a null hypothesis. Essentially it states that the patterns revealed by the experiment are purely accidental. (I have severe reservations about the way the null hypothesis is framed, but they are rather technical. Anyone interested should read my rebuttal paper.) However, there is not a word about the alternative hypothesis. Of course, we all know what Witztum and his collaborators are trying to prove. They want to show that Hashem encoded in the Torah hidden information relating to events that occurred a long time after the Torah was given. They also want to show that other literary texts do not contain such hidden information. For otherwise why would they try the experiment on the Hebrew translation of Tolstoy's War and Peace and on the Book of Isaiah? So the appropriate alternative hypothesis would be: "Hashem wrote the Torah and encoded in it information relating to the future. Such information is not to be found in texts written by humans." The difficulty with this alternative hypothesis is that it does not allow us, as is required by the experimental procedure, to evaluate probabilities related to the outcome under the assumption that the alternative hypothesis is true. To do this, we would need to be able to read the mind of Hashem and as we know: "My thoughts are not your thoughts." (Isaiah 55:8).

Not stating an alternative hypothesis is practically
fatal to the whole test. As Kendall and Stuart put it in their standard
textbook *The Advanced Theory of Statistics* [3]: "We cannot
say whether a given body of observations favours a given hypothesis unless
we know to what alternative(s) this hypothesis is compared. It is perfectly
possible for a sample of observations to be a rather 'unlikely' one if
the original hypothesis were true; but it may be much more 'unlikely' on
another hypothesis. If the situation is such that we are forced to choose
one hypothesis or the other, we shall obviously choose the first,
notwithstanding the 'unlikeliness' of the observations."

There is consensus among statisticians that when an alternative hypothesis is not stated and the data appear unlikely according to the model assumed by the null hypothesis, all we can conclude is that the model chosen is unsuitable, no more. The reason for this is that it has been known for a long time that patterns which appear meaningful and highly unlikely to have occurred by chance can be found in any large enough amount of data.

The natural alternative hypothesis, which most people would expect, is that Hashem encoded the hidden text in the Torah. But when we look at the data, we find very bizarre things. The heuristics of the paper contend that there is a significant proximity (according to the distance measure defined by WRR) between the appellations of the personalities and their birth and death dates. Dr. McKay has determined that there are 930 different legitimate forms of dates in the Jewish Calendar which have an ELS in Genesis. Now if we look for example at Ha Rambam, we find that his birth date (14 Nissan) appears in two forms which have ranks 332 and 696 among all dates when ordered according to the WRR distance. Let us remember that this means that there are 331 dates out of 930 which are nearer to his name than the correct one. His date of death (20 Tevet) also appears in two forms, which have ranks 686 and 890. Looking at Ha Maharsha, for whom we have only a date of death (5 Kislev), we have three forms, and they are ranked 459, 688 and 788. In fact, the only appellation for which the correct date has rank one in the first list of rabbis (the most famous ones) is Rabbenu Tam (died 4 Tammuz). Contrary to what is implied in the WRR paper, most appellations of Rabbis are closer to wrong dates than to right ones. What kind of code is that? Professor Barry Simon, of Caltech, USA, who is an Orthodox Jew, asks: "If Hashem placed this evidence there, would He do it in such an incredibly indirect and imperfect way?"

Another puzzling fact is that the code research consistently refers to "Torah Codes". Michelson [4] wrote in 1987 that the full electronic error-free text of the whole Torah was already available to WRR then. On what grounds did WRR decide, before they conducted the crucial experiment (on the second list of Rabbis), to conduct it on Genesis only? It is now known that when the experiment is carried out on the other four books of the Torah it fails. That fact alone puts in question the relevance of the fact that the test with the second list of personalities fails when tried on the Hebrew translation of War and Peace, the Book of Isaiah and various permutations of Genesis.

The way the distance measure is defined raises grave
concerns. WRR first define a "proximity measure", denoted by
"omega", which, according to them, "very roughly measures
the maximum closeness of some of the more noteworthy appearances of two
words as ELS's". It appears more or less reasonable. But then in
the *Statistical Science* paper, they introduce a "corrected
distance", denoted by "c", without any motivation. They claim
that c is small when the two words are "unusually close" and
is 1 or almost 1, when the two words are "unusually far". The fact
that they chose omega to measure proximity and c to measure distance
(the inverse of proximity) is very strange.

In their presentation to the Israel Academy of Science
in March 1996, WRR compare the "corrected distance" to ranking in a
"race" between the distance of the original words and the
distances of "perturbed ELS's" representing the words.
The trouble is that each pair of ELS's "races" with perturbations
of itself. Thus different word pairs each "race" with a different
group, and therefore the results cannot be expected *a priori*
to be comparable.

To test the validity of the claims for c in the actual data, I obtained from Dr. McKay the values of omega and c for all correctly matched appellations and dates from the two lists of personalities that had both measures. There were 320 of them. Generally speaking, the two measures correlated very badly. Precise details are given in my rebuttal paper. Here is an illuminating example:

First note that the statistics for omega were:

Minimum 77.33

Mean: 3976

Maximum: 60,365

And for c:

Minimum 0.008

Mean: 0.333

Maximum: 1.000

Let us look at a pair of words for the Vilna Gaon: HaGaon and Tet Vav Nissan. The omega value, which, remember, measures proximity, is 1364, which is way below the mean. This means that the two words are quite far. But the corresponding c, which measures distance, is 0.076, which is extremely small, showing that according to the c measure, the two words are very near!

Looking now at a pair of words for the Rema: Rabbi Moshe and b'Yud Chet Iyyar, we find an omega proximity value of 6,223, way above the mean, indicating that the two words are "unusually" near, while the c value, measuring distance, is 0.400, way above the mean, indicating that the two words are "unusually" far!

No wonder that Professor Barry Simon, a seasoned mathematician, writes: "I find the method of assigning a distance ranking (adopted by WRR) unnatural - I think it highly unlikely that some other mathematicians trying to find a notion of closeness would use the one in the paper. This very unnaturalness makes me uncomfortable and suggests that the authors were led to their metric by experimenting with a few pieces of data - perhaps for a few famous rabbis. Given that other aspects of their analysis give undue weight to a few select anomalously "close" pairs, a little unintended bias in the method can go a long way."

I have asked Dr. McKay to rerun the Famous Rabbis experiment using the geometric mean of the original omega proximity measure (the one that does make some sense) as a summary statistic. This is what any experienced statistician would try at first. The significance of the result was reduced by a factor of the order of one thousand. Such a result would have probably led the referees to reject the paper. This experiment illustrates well the fact that the exact choice of distance measure is crucial to the success of the experiment.

The conclusion of my rebuttal paper is that until the flaws I have outlined are remedied the claims made in the paper must be considered as statistically unfounded. This is not just my private opinion. A "Mathematicians' Statement on the Bible Codes", which makes the same assertion, has to date more than 40 signatories. All hold PhD's in Mathematics or Statistics or are faculty members in a Department of Mathematics or Statistics at a college or University. They have themselves examined the evidence and found it entirely unconvincing. Some are professional statisticians. More than four are members of the National Academy of Science of some country. Many are Orthodox Jews who believe in the Divine Origin of the Torah.

The fact that patterns that appear to be highly improbable on the assumption of a random model can be found in all literary works of sufficient length has been known for a long time. I documented a few cases in my article in B'Or Hatorah [2]. Computer search has opened a new dimension in this area. Drosnin and various missionaries have had no trouble finding patterns to suit their purposes and in dramatically misusing them.

In order to demonstrate that the *Statistical
Science *methodology does not by itself enable onlookers to be satisfied
that what passes the test is in any sense "genuine" McKay and
his collaborators conducted the following experiment. After consulting
various books, encyclopaedias and experts, they made a small number of
perfectly reasonable changes to the WRR's list of personalities and
appellations, keeping within the same guidelines that WRR themselves claimed
they had used. In fact, Professor Menachem Cohen, of the Department of Bible
Studies at Bar Ilan University, has stated in a letter dated 27 October
1997: "I see no essential difference between the two lists for the
purpose of using them for ELS experiments in any text.". Of course
McKay and his collaborators do not hide the fact that they "cooked"
some of the data to obtain their results, that is, they manipulated
the data *a posteriori*. When they applied the *Statistical Science
*methodology, using the modified second Famous Rabbis list, in Genesis
and in a segment of the same length from the beginning of the Hebrew
translation of Tolstoy's War and Peace, the test failed in Genesis but
succeeded in Tolstoy's text to the same degree as the original list had
succeeded in Genesis. Duplication of other experiments carried out by Witztum
yielded similar results. By showing that the experiment can be easily
manipulated, McKay and his collaborators completely deprived the work
of WRR of any serious import.

There is a basic difference between the WRR
experiment and the McKay *et al *experiment on the Famous Rabbis in
Tolstoy.* *The data of McKay *et al *were admittedly
"cooked", i.e. they manipulated the appellations *a
posteriori* to obtain a significant result. On the other hand, in his
response on the Internet to McKay *et al*, Witztum writes that for the
original experiment the list of names was prepared in advance, following an
objective procedure. The names and appellations of the rabbis were determined
by Professor Havlin.

When I read the details of the data selection procedure, together with the experiments carried out by McKay, I was overwhelmed. As has been pointed out in the previous section, it is known from McKay's experiments that the data of the WRR paper are extremely fragile, in the sense that small departures from the list used can totally destroy the significance achieved.

However, what Witztum *et al*, together with
Professor Havlin, set out to achieve is, in my mind, far more miraculous
than the result of the experiment. They went through a sequence of choices
where one wrong choice could wreck the whole experiment. Let me detail
some of the choices:

- They chose to carry out the experiment on the Koren text of the Torah in preference to other available texts.
- They chose to carry out the test on the book of Genesis only.
- They chose the list of personalities from Dr. Margaliot's Encyclopaedia when other equally acceptable texts could have been chosen.
- They decided arbitrarily to choose for the first list those rabbis whose entry was at least three columns in length and for the second list those Rabbis whose entry was between one and a half and three columns in length.
- They deleted all rabbis who did not have either a birth date or a death date in the Encyclopaedia.
- They chose arbitrarily formats for the dates.
- They arbitrarily restricted the appellations in length to the range 5-8 letters.
- They developed rules for choosing the appellations on an arbitrary, subjective basis. Professor Havlin freely admits that any researcher (including himself) in the field of appellations must utilize his personal discretion with regards to a number of issues. Other scholars have since voiced serious reservations about Professor Havlin's rulings. In fact Professor Cohen, in the letter quoted above, has written : "The list prepared by Prof. Havlin, following the considerations detailed in his brief, has, in my humble opinion, no scientific basis, and is entirely the result of arbitrary and inconsistent choice."
- They replaced the original omega measure by a "c" measure and used an extremely complicated formula for obtaining a summary statistic which they themselves admitted had no theoretical basis in the context of their null hypothesis. This substitution increased the significance of the result by a factor of the order of one thousand.
- They restricted perturbations of the ELS's to the last three letters and to a special set of perturbations.

Each of these choices is now known *to be vital
to the success of the experiment*, although this *could not have been
known in advance*, yet Witztum, Havlin and their collaborators unerringly
made all the right choices, *a priori*. The experiment succeeded.
As reported by the editor of *Statistical Science*, the referees were
baffled by the success of the experiment they had themselves commissioned.
They had not believed it would succeed.

What can we say about the results of the WRR's
experiment? At the end of the day, the credibility of the *Statistical
Science *paper rests entirely on the statement of WRR that they did the
experiment honestly. Of course, the skeptics will not accept WRR's word, if
only because accepting it would involve their acceptance not only
that the Torah is of Divine origin, but also that Witztum and his
collaborators were Divinely inspired at every step of the experiment
construction. And not only skeptics, but also many Orthodox Jews, are
unwilling to accept this and prefer to doubt the honesty of WRR.

It is now firmly established that the *Statisticas
Science *methodology is scientifically flawed and that it is unable to
discriminate between patterns in the Torah and patterns in other literary
texts without having to rely on the honesty of the experimenters.
Michelson has written in the discussion of my B'Or HaTorah paper [5] that
when leading Torah authorities in Israel were asked about publishing the
findings of codes research they insisted that it should be done
"professionally". The WRR black box in no way fulfils this
requirement.
And let us not forget that bitter experience has taught us that
misinterpretations of our Holy Torah have often resulted in the past in
disaster and catastrophe for our people.

All Witztum and his collaborators needed to do to
validate their discoveries was to have them authenticated by *Gedolei
Yisrael.* By this I do not mean just words of general encouragement.
What is needed is an explicit endorsement and authentication, in writing,
of their specific interpretation of the Famous Rabbis experiment, the
Nations experiment, the Subcamps of Auschwitz experiment, and any other
experiment already carried out or to be carried out in the future. In other
words, an explicit "Haskama" (Approbation) for each of them. They
could then dismiss Drosnin and the missionaries simply on the grounds
that they did not get the appropriate authentication.

A final word of advice: In my long experience with Baalei Tshuvah I have always found that the most successful long-term approach was to encourage them to fulfil practical Mitsvot rather than presenting them with "miraculous" signs.

[1] Drosnin, M. (1997) *The Bible Code.* Weidenfeld
and Nicholson.

[2] Hasofer, A. M. (1993) Codes in the Torah: A Rejoinder.
*B'Or Ha'Torah*, **8E**, 121-131

[3] Kendall, M. G. and Stuart, A. (1973) *The Advanced
Theory of Statistics.* Vol II. Charles Griffin & Co. Third Edition.

[4] Michelson, D. (1987) Codes in the Torah: Reading
with Equal Intervals. *B'Or Ha'Torah*, **6E**,7-39.

[5] Michelson, D., Eidelberg, P. & Stolper, Rabbi
P. versus Hasofer, Prof. A. M. (1995) Debate on the significance and
methodology of the codes found in the Torah by computer search. *B'Or
Ha'Torah*, **9E**, 114-129.

[6] Witztum, D., Rips, E. and Rosenberg, Y. (1994)
Equidistant Letter Sequences in the Book of Genesis. *Statistical
Science*, **9**, 3, 429-438.