The Choice of Date Forms - a reply to Doron Witztum

Brendan McKay, Australian National University

Summary

Witztum, Rips and Rosenberg (WRR) claimed to prove the existence of a hidden code in the Bible by an experiment that involved the names and appellations of famous rabbis, together with their dates of birth or death. The two main aspects of their experiment deserving of critical scrutiny are the data and the experimental method. Our reply in Statistical Science examined both of these in detail, and came to the conclusion that the data collection process is especially suspect, particularly with regard to the selection of appellations and their spellings.

A lesser question about WRR's experiment concerns the choice of dates, and especially the choice of how to write them. Dates in the Hebrew calendar can be written using only letters, with one or two letters representing the day of the month and the rest being the name of the month. There are also some attachments representing "in" or "of" that can be used, in at least 8 combinations. Naturally, the popularity of the various forms varies greatly between the forms, and has also varied over time.

We will use the same notation as in our paper. D is the day of the month (one or two Hebrew letters), M is the name of the month, b is the prefix "in" (the letter beth), l is the prefix "of" (the letter lamed), and shel is the word "of" (the letters shin and lamed). The reasonable combinations of these are:

D M, bD M, D bM, bD bM, D lM, bD lM, D shel M, bD shel M

The first three of these forms were what WRR used. At the time this choice was made, they did not attempt to justify or explain it. Years later, when the choice was questioned, they said that the choice was made by a linguist Yacov Orbach who had since died. They had no idea why he made that choice, they said.

Since at least one date form must be used, there are 255 possible choices of date forms out of the 8 available. For the success measure used by WRR in their published paper, the choice made by WRR gives uniquely the best result out of the 255 possibilities. (Since the date forms are not equal, and for other reasons, this observation does not represent a 1/255 probability.) We should not infer from this that the choice of date forms was made by WRR to optimise the result of their published experiment, since in fact the choice of date forms had been already made for an earlier experiment. Rather, it most likely represents a side-effect of a bias towards appellations which favour those date forms.

Witztum has now published an article which asserts that everything we wrote about the dates was (deliberate) nonsense. Most of Witztum's article is not worth replying to, as our paper already covers the subject well enough, but we are prompted to make a few final comments.

The choice of success measure

In the final analysis, the only thing that really matters is the success measure used for WRR's published experiment. As indicated above, WRR's choice of dates was very fortuitous for that measure. However, in analysing the history of these choices, it is relevant to study the effect of the choices on the success measures WRR used during the earlier period when the data for their experiment was being compiled.

Witztum claims that the correct measures to use are "P1 or P2, or, more probably, [the minimum of P1 and P2]". Here P1 and P2 are two numbers that WRR originally thought were probabilities (but which are not). The trouble with Witztum's claim is that none of his early preprints, where the data was first published, give the value of P1 or even the definition of P1. Another measure related to P1 was presented, but it cannot be used to take a minimum with P2. The measure min(P1,P2) was in fact a much later invention of Witztum who found that it obscures the effects of data tuning for some reason. Any use of it for the purpose Witztum uses it is completely invalid.

The frequency of date forms

Witztum is correct in stating that the various dates are very different in their frequency of use, as is mentioned in our paper. His case is that the common forms should be used and the uncommon forms should not be:

[Witztum:] [D lM] is a nonstandard form of referring to a date. For example, both Margalioth's encyclopedia, as well as the Encyclopedia Hebraica use the forms we used, and not this form. It is clear that the forms we used are the most widely used forms. We conducted a survey regarding the use of the various forms, using the computerized responsa database of Bar Ilan University. Here are the results for a pool of modern Halachic authorities:

Then Witztum presents a table of counts where the first 6 of the date forms we listed are grouped into three categories. This gave the following counts:

date forms
count
D M and bD M
4096
D bM and bD bM
1552
D lM and bD lM
15

Witztum's case may look convincing, but any experienced Witztum watcher will immediately notice two strange aspects of Witztum's case and wonder why he provided no explanation for them. Indeed, it is a simple matter to conjecture answers for them:

  1. Question: Why did Witztum group the date forms two at a time instead of listing them separately?
    Conjecture: To hide the falsity of his claim to have used "the most widely used forms".
  2. Question: Why were only "modern Halachic authorities" considered, and not the (generally more revered) traditional authorities?
    Conjecture: Because the traditional authorities had different preferences, and Witztum doesn't want us to know.

To check these conjectured answers, we counted the date forms separately in the whole of the Reponsa database (version 8):

date form
count
D M
7765
bD M
481
D bM
2771
bD bM
1296
D lM
308
bD lM
71

We can see that both conjectured answers are correct. One of the forms omitted by WRR (bD bM) is more common in Responsa than one of the forms they used (bD M). Moreover, the forms with lamed have an overall frequency more than 11 times what Witztum reported.

To see exactly how serious Witztum really is about the issue of frequency of use, we note that his article on the dates introduces yet two more date forms: D dM and bD dM, where d represents the letter dalet. These are Aramaic forms of doubtful validity in a Hebrew experiment. Witztum uses them without any apparent pang of conscience, yet when we count their usage in Responsa we find only 31 cases of D dM and 5 of bD dM. Apparently, Witztum's rule about only using popular forms applies only to us: he attacks us for using rare date forms then uses even rarer forms himself.

Variations that add data

Witztum presents a number of examples where adding additional data to his experiment (such as additional date forms) improves the value of P1 or P2. This can occasionally be justified in a study of choice making, though not always. Leaving that isssue aside, we wish to record here why such experiments are irrelevent to the issue of whether the codes are genuine.

Our study began with WRR's first list of rabbis, with their dates and appellations as WRR gave them. In addition, we added to each rabbi an extra "date" which was, in fact, just 6 letters chosen at random from the text of Genesis. This represents a substantial amount of noise that nobody would expect to show "codes". Nevertheless, when we ran this experiment many times, the added random data actually improved the value of P1 more than 30% of the time. It even improved it by large amounts fairly often: by a factor of 1000 about 1% of the time. Once in our 1000 trials it improved P1 by a factor of more than a million.

Much the same counterintuitive phenomenon occurs for P2, and for WRR's second list of rabbis. This shows just how useless P1 and P2 are as measures of success and how little it matters if adding extra date forms makes them appear better or worse.

One final observation

From the data given in our paper, it is easy to see that the success of WRR's first list of rabbis depended mostly on the date form bD M which is only weakly successful in the second list. Conversely, the success of the second list is mostly dependent on the form D M which is a clear failure in the first list.

This discrepancy shows at the very least that WRR don't really understand why particular date forms succeed in codes experiments and therefore that they aren't really in a position to make pronouncements on the matter. Perhaps more interestingly, this observation seriously undermines WRR's claim that the second list of rabbis confirmed the observations they made with the first list. Contradictions like this are in fact commonly taken as pointing to illusory phenomena, since objective phenomena tend to be more consistent.

Update

On Dec 16 2001, Witztum published what he claimed to be a rejoinder to the above article. However, we cannot see anything worth replying to and so will not bother.

Back to Bible Codes Refuted
Creator: Brendan McKay, bdm@cs.anu.edu.au.