This is a copy of an article that Doron Witztum published as a PDF file. The Hebrew has been transliterated, using the Michigan-Clairmont encoding scheme. No claims of accuracy are made.

A reply to this article is available.

Does Tolstoy Really Love Brendan McKay? Doron Witztum, 1997. On July 31st, 1997, Dr. Brendan McKay of the Australian National University presented an article on the internet, in which he claims that the personal details of his life "amazingly" appear encoded at minimum ELSs along with his name, in Tolstoy's War and Peace. He claims the probability of this occurring by chance to be less than one in five thousand. In this response, we will show that Dr. McKay achieved his results through a combination of systematic error and very partial reporting of the number of trials conducted. The two errors were employed by Dr. McKay with the aim of creating a "counterfeit" effect of success. We will demonstrate that real probability involved in this experiment corresponds exactly with what is expected to take place randomly. A. THE TYPE OF SAMPLE In a previous version of his sample (in a letter sent to the Internet TCODE group on May 5, 1997), Dr. McKay describes his experiment as follows: Forwarded message: From: 0 bdm@cs.anu.edu.au (Brendan McKay) Sender: 0 owner-tcode@express.ior.com Reply-to: 0 tcode@express.ior.com To: 0 tcode@express.ior.com Date: 97-05-05 06:02:02 EDT Just today I calculated my own birth date in the Jewish calendar. It was 26th Tishri 5712. Then I tested it against my name BRNDN, using the usual date forms and the year forms chosen for our new experiment. (My surname is shorter than 5 letters.) The results are amazing. The average distance over the three date forms is 0.176, compared to 0.328 and 0.330 for the two lists in the StatSci paper. The average distance over the five year forms in the 5-8 length range is 0.261, though it would be only 0.138 except for a single large value. The geometric means (which are a crude measure of how much contribution is made to the P2 statistics) are 0.168 and 0.149, compared to 0.210 and 0.215 for the two StatSci lists. I would be quite a champion in the StatSci lists. Brendan PS. Perhaps I should mention that the text in which I measured my name was War and Peace. In other words, he searched for "meetings" of the ELSs of the name BRNDN (Brendan) with those of his birth date (day and month) in three forms, and with the year of his birth written in six forms. This type of sample, in which we measure the proximities between ELSs of a single expression and those of a list of other expressions, we termed a "heading sample". In our paper "Hidden Code in the Book of Genesis", ("CPN XMWY BSPR BR)$YT" preprint 1996; originally presented as a lecture before the Israeli National Academy of Sciences, 19 March, 1996), we analyzed this type of sample, and we explained how to carry out the randomization process for it. In his final version, Dr. McKay selected another heading, DR MQYY (Dr. McKay) and created another heading sample with the same dates; the final product being the union (merger) of these two samples. (He did one other experiment, searching for "Canberra". This will be discussed in section F). B. THE METHOD OF MEASUREMENT As we explained in the above-mentioned paper, the proper method for analyzing a heading sample has three stages: 1. We use the function c, ("the Corrected Distance" in the Statistical Science paper), to measure proximity between the heading and each expression w (from the list). However, the function has the following alteration: the heading is only taken as ELSs, while the expression w is taken as ELSs and as PLSs (perturbed letter sequences). In other words, ELSs of w compete with the PLSs of w over the more successful proximities to the ELSs of the heading. If there are n expressions in the list, the values of P1 and P2 (the Overall Proximity Measures) are calculated based on the n results. 2. Given a list with n expressions, a series of n permutations is randomly drawn: permutation i changes the order of the letters in expression i. In this way, n new expressions are derived and their proximities with the heading are calculated (as explained above in stage 1). The values of P'1 and P'2 are calculated for this series of n permutations. 3. This process is carried out N times, and the rank order of P1 and P2 is calculated. C. THE RESULTS FOR MCKAY'S SAMPLE: When Dr. McKay's double "heading" sample is calculated in this manner, the following results are obtained: At stage 1: P1 = 0.082 and P2 = 0.031 At stage 3: The rank order of P1 for N = 100,000: 3955 The rank order of P2 for N = 100,000: 1199 i.e., the probability is p = 2 x 0.012 = 0.024, that is, only 1/40, instead of 1/5000. (Of course, 1/40 is the probability if the headings BRNDN and DR MQYY were the only ones checked. But this is not the case. In section E, we will show that the final probability is actually much higher than even this.) In what follows, we explain the source of the differences between these findings and those of McKay: D. MCKAY'S SYSTEMATIC ERROR 1. A few months ago, (in a posting called "Equidistant Letter Sequences in Genesis- A Report. Feb. 22, 1997), Dr. McKay noted that some of the names in the list of Rabbis included in the experiment published in Statistical Science, had the following property: their ELSs appeared more often than their PLSs. This advantage may allow for easier "successful" proximities with ELSs of other expressions. Suppose that expression x possesses this advantage. If we calculate the proximities between its ELSs and those of the series of expressions, w1, w2,.. wn, we will receive an overabundance of "successful" results. This will happen whether the other expressions are related to x or not. This is a systematic error because we exploit a certain property of the expression more than once. This possible advantage connected to the number of ELSs of a word, may be only one possible advantage that an expression may have. Regardless of the origin of the advantage, let us call expressions with these types of advantages "charismatic" expressions. Naturally, besides charismatic expressions, some expressions may also be "anti-charismatic". For instance, if the ELSs of an expression appear less often than its PLSs. Such expressions will tend to produce "failures" in measuring proximities with other expressions. In the sample of the Rabbis, there were many names and appellations. Some may be charismatic, and others may be anti-charismatic. One would expect that these effects will balance each other out. The permutations test proposed by Professor Persi Diaconis, as described in the Statistical Science paper, is supposed to eliminate any such residual effects of charisma for symmetry considerations. If we assume that the success of the proximities of name x is due to its charisma, then it should be just as successful with unrelated dates to which it is paired as to related dates. Thus, the permutation test should cancel the "charisma" (advantage) of individual terms. However, Dr. McKay (in his Report) argued that the permutations test may not have succeeded in canceling this effect. We thought Dr. McKay was wrong, and that Professor Diaconis' permutation test was a good one. Nonetheless, it occurred to us that to demonstrate to Dr. McKay that he is incorrect, one could apply the test described in B1 (described in the above mentioned paper). In a response sent to him (A Preliminary Analysis and Comments on the Report of New ELS Tests, June 19, 1997. Posted on the internet), Professor Eliyahu Rips proposed the following test (Stage 1 in section B above): To repeat the experiment described in Statistical Science, with the single variation that the names will be taken only as ELSs, while the dates as ELSs and PLSs. This simple procedure cancels all possible effects due to charismatic names. When we carried out this experiment, it turned out that the results improved. Whereas the ranking of the most successful statistic P4 was originally four in a million, it now improved to one in a million. In other words, the success of the Statistical Science experiment was clearly not caused by this effect. 2. When one is conducting an experiment that involves a heading sample, the need for caution cannot be overstated. If the expression being used as a heading happens to be charismatic, then it will "succeed" with many expressions, whether they are related or not. It appears that Dr. McKay neglected to take into account his own observation: he happens to have used charismatic expressions for the heading in his sample, and seems not to have realized that the "success" of his results is entirely artificial. Moreover, the randomization technique Dr. McKay performed, instead of offsetting the charisma, ignored it altogether. Indeed, as per the discussion in section B above, the use of only ELSs for the headings and the appropriate randomization technique, is sufficient to make the artificial result of 1 in 5,000 into a real one of 1 in 40. 3 . In my opinion, there is an even more basic approach to the prevention of errors of this sort, and it touches on the question of the exact phenomenon being traced. Here is not the place to discuss this subject at length. It will be covered in detail in my forthcoming book. One more remark: in the summer of 1985, we set out to define a "distance" between two expressions, w and w'(a measure of proximity). In general, each expression will be represented by several ELSs. First, we define a "distance" between a particular ELS of w and a particular ELS of w'. We then have two options: a) To sum all the "distances" between ELSs of w and ELSs of w'. We call this option TOT. b) To take only the best "distance" of all these distances. We call this option BEST. At the time, Professor Rips suggested that TOT would provide a more stable measure, so we used it for all our experiments, including the Rabbis experiments. This is the function described in our 1994 Statistical Science paper, from which function c is derived. Last year, I conducted several experiments (also described in detail in my forthcoming book), using the BEST measure. I believe that BEST avoids the problem of charisma, arising from ELSs appearing more often than expected. It must be stressed, however, that BEST should be considered only as an additional or complementary measure of proximity, not as a replacement for TOT. If we measure Dr. McKay's sample using BEST, after randomization (section B, stages 2 and 3) the resulting p value is reduced even further to p = 0.019 x 2 = 0.038, or roughly 1/25. In other words, using BEST, there is no success for Dr. McKay's list. E. SELECTION OF THE HEADINGS: Until now, we did not take into account the number of headings used by Dr. McKay. Basically, there are two possible ways of transcribing the name "McKay." There is a transcription according to the written characters and a phonetic transcription (which Dr. McKay claims he prefers). There are also two ways to spell the name, either with a single "Y" at the end or with a double "YY". It is a mistake to write McKay with a double "YY". Nonetheless, since he used it, we will go along with his choice. 1. Transcription according to the written characters: Dr. McKay uses the Encyclopedia Hebraica for transcription. For example, he directs us to copy the name QNBRH (Canberra) from there. In this encyclopedia, in volume XXII, p. 965, (eighth line from the end of the page), one finds two transcriptions beside each other: MQQLWQ for McCulloch and MQY for Mackay. According to the encyclopedia, Dr. McKay should properly spell his name: MQQY. Moreover, Dr. McKay himself, in a paper describing Chanukah codes in War and Peace, used the transcription MKQY (with a single "Y"). Dr. McKay also provides us with an explanation why he ignored the very encyclopedia he used in his present article and why he is not consistent in his usage of MKQY as before. He states that the only time his name was mentioned in the Israeli press (the Israeli newspaper Maariv), it was spelled MQYY. In truth, we think that the transcription MQYY is a mistake altogether. A newspaper should not be considered an authoritative source for spellings of transliterated words. Nonetheless, since he used it, we will consider its use. However, Dr. McKay's claim that his name was never otherwise mentioned in the Israeli press is simply not true. In an article in the magazine Mishpacha, published on the July 3, 1997, (page 13, Column 1, Line 14), his name is spelled MQQYY. And if Dr. McKay should claim that he did not know of this publication, it would be a strange claim the dedication at the head of his own article: LA.L. M$T"P )MYC (M HAMT is based on that same piece in Mishpacha. In summary, using the transcription according to written characters, in view of previous publications by Dr. McKay, the sources he has variously cited himself, and the fact that he sometimes uses one "Y" and sometimes two, we have with the following eight variations: MKQY, MKQYY, MQQY and MQQYY, and with the D"R (Dr.) as a prefix: DR MKQY, DR MKQYY, DR MQQY and DR MQQYY. Of these, there are 5 expressions of 5-8 letters each, which appear as ELSs in War and Peace: D"R MKQY, D"R MKQYY, MKQYY, D"R MQQY, MQQYY. It should be remembered that according to the criteria established in his email (page 1 above), Dr. McKay only wanted to use his surname, but in fact, he used both his surname and the D"R prefix. 2. The phonetic transcription: According to the above-mentioned Encyclopedia, the name should be spelled MQY with a single "Y". This usage of one "Y" is how he himself spelled it in his paper on Chanukah. Here he freely adopted MQYY with a double "YY". By adding the prefix D"R, we have 2 expressions of 5-8 letters each: D"R MQY and D"R MQYY. In all, we have found 5+2=7 different expressions which may be found as ELSs in War and Peace. 3. The transcription of his first name: Dr. McKay spells the name Brendan: BRNDN, but in the same article of the Maariv newspaper which is the source of his spelling of MQYY, indeed, at the same place in the article, the name is written BRNDWN rather than BRNDN. In summary, Dr. McKay thus appears to have tried at least 16 different headings. According to his letter, he tried BRNDN separately and according to his article he tried combinations. In other words, he could have readily tried BRNDN separately and also BRNDN plus each of the above 7 possibilities, i.e., a total of 8 headings. The same goes for BRNDWN, which translates into a total of 16 different headings. Please note that all of the above calculations are based on Dr. McKay's own data and selections, and we have made no references to other possibilities which may have been taken into consideration. F. THE SELECTION OF THE EXPRESSIONS IN THE LIST: Once Dr. McKay added QNBRH (Canberra, which is not the place of birth) to the list, he revealed that he had an inestimable space of biographical details to work with. Not only is it prohibited to include QNBRH on the list, it also casts a dark shadow on the remainder of the list. Indeed, it is clear that Dr. McKay was interested in both the date and place of his birth. Thus, his list of names and dates is itself a partial list of the biographical details of his birth: it contains only the date (day and month), and the year, but does not include the place. If he takes a partial group of the details, we have to calculate how many sub-groups he could have taken, searching for that cluster which offers the most impressive proximity. Dr. McKay had three options available: 1. Dates stating only the day and month (K"W T$RY, etc.). 2. Dates stating only the year. 3. Options 1 and 2 combined. G. SUMMARY: If we compile all the selections Dr. McKay could have made for the headings and for the list, it follows that even the most conservative calculation leads to at least 16x3=48 different possibilities. In fact, in his email, Dr. McKay reports the use of additional two statistical analyses beyond P1 and P2, so the final probability obtained in section C (p=0.024) should be multiplied by at least 2x48=96. In other words, the statistical significance of Dr. McKay's results does not deviate at all from that expected for a random event. Note: All the above calculations are based on the assumption that the data supplied by Dr. McKay is accurate. We have reason to be skeptical of its accuracy given the serious flaws in the data of his earlier reports. We have shown that Dr. McKay's success was in fact artificial. For the sake of comparison, we will shortly publish an authentic example of a heading sample.