The Lleida calendar: a match that survived three tests and turned out to be an artefact

We tell this story not because it turned out well but because it is instructive. We got a handsome result, it survived three independent tests, and it was still empty. This is what a false positive looks like from the inside.

What we were looking for

The Voynich zodiac pages are all built the same way: a circle divided into sectors, a figure in each sector, and beside each figure a short label. There are about thirty such labels per sign.

If these are days, a church calendar suggests itself: in medieval calendars every day has its saint, and the order of the saints is a rigid sequence characteristic of a particular diocese. A match between the labels and such a list would give us the thing we have in no form whatever - an external foothold, a piece of plaintext.

We assembled about two dozen calendars from southern Europe, from the right region and the right period: Toulouse, Narbonne, Beziers, Elne, Girona, Valencia, Zaragoza, Pamplona, Lleida and others.

What we found

The Lleida calendar of 1524 matched the Voynich rings noticeably better than the rest.

After that we did what one is supposed to do with a result of this kind:

• Correction for multiplicity. We were comparing many calendars, so one of them could have won by chance. The Bonferroni correction takes account of that, and the match stayed significant.
• Replication. The result reproduced itself on a different part of the data.
• Held-out sample. It held up on the part that had taken no share in the selection.

Three tests from the standard set, all passed. At this point many studies publish a discovery.

What killed it

We applied a fourth test, one that is not part of the standard set: a test for anachronism.

The Lleida calendar is dated 1524. The Voynich parchment is 1404-1438. There is roughly a century between them. A historical connection is possible only through an older stratum of the calendar, since diocesan calendars are inherited and an old core is present in them.

So if the match is real, it ought to rest on the old part of the calendar. We decomposed the match by the time at which its elements appeared, and it turned out to rest on elements that are late, later than the manuscript. In other words, the match cannot be a historical connection, because it leans on things that did not exist in the fifteenth century.

The handsome result was an artefact.

We also checked the opposite possibility, that the older Lleida calendar we needed simply has not come down to us. The answer: a complete Lleida sanctoral of 1450-1510 does not physically exist, so there is nothing to test against.

A test that does not depend on the key

After that story we did what we should have done at the start: we built a test that does not depend on how exactly the labels correspond to days.

The idea is this. Any list - a calendar, a catalogue, a series of degrees - falls into groups: feasts and ordinary days, male and female, important and routine. If two lists are related, their partitions ought to be similar, however we match up the individual elements. This is an invariant: it does not depend on the cipher key at all.

The result: the partition of the Voynich rings matches "its own" calendar no better than it matches other people's. We checked two further calendars in addition, Vic 1496 and Tortosa 1524, and both gave zero (p = 0.63 and 0.64).

The main conclusion: these are not days but degrees

The whole series implies a change of frame. The Voynich rings are not a calendar. There are about thirty positions per sign, and they behave like degrees of the zodiac, not like days of a month. These are different traditions: lists by degree form a separate astrological genre, in which every degree is assigned an image or a characteristic.

We ran that class of sources too - about nine varieties of by-degree list, including the images of the Astrolabium planum, paranatellonta, star catalogues and the Liber Hermetis. All gave zero. For what remained after that, see the degree signal and what survived of it.

What this story teaches

Three standard statistical tests do not save you from a false discovery if the hypothesis has not been tested by a property it is obliged to satisfy. A correction for multiplicity answers the question "could this be chance?", whereas a test for anachronism answers the question "could this be true at all?". The second question is the stronger one.

That is exactly why we keep and publish our false positives alongside our successes: without them there is no telling what the successes are worth.

No decipherment of the Voynich manuscript exists. A coincidence is not a reading, even one that has passed Bonferroni.

← All hypotheses