Thoughts after Workshop: Textual Analysis Using Stylometry (AUB, 24-25 April)

On 24 and 25 April 2018, Najla Jarkas organised a workshop on “Textual Analysis Using Stylometry” at the american University of Beirut. The workshop was hosted by David Wrisley and given by Maciej Eder, an Associate Professor at the Institute of Polish Students at the Pedagogical University of Krakow, Poland, and and at the Institute of Polish Language at the Polish Academy of Sciences.

For those who are not familiar with the term stylometry (as was I before the workshop), it is a method of textual analysis that does not focus on the big and exceptional words but on the frequency of every-day terms in writing. By measuring frequencies and distribution, it has become a major method for everything from authorship attribution (note the case of the identity of the pseudonym Elisabeth Ferrante) and proximity to proving plagiarism.

Within the workshop, we took first steps in using the software R on a preassembled textual corpus of 19th-century English literature. We used Dr. Eder’s stylometry package for R which also can be found on his github page. You also find there two corpora to try out what it can offer you. Following these first steps I tried to apply stylometry to my own small corpus of teaching certificates getting mixed results. This might be in the nature of these texts, being epitomised by the most common three-word combinations being “Muhammad bin Muhammad” and “bin Muhammad bin”. Within the small sample, there was also little proximity to be found between the different texts.

Yet, as digital methods so often do, they make you think about your own methods and conditions of research. And R seems promising when you think of the possibilities in analysing premodern Arabic texts. As historical texts in particular were often to a large degree the product of recompilation of earlier sources, would it not be cool to find out where an author most probably copied from another one? This is basically what Sarah Savant’s Kitab Project promises to achieve.

But what would we need to enable the machine to help us with our texts? Clearly, it depends on what you work on: the period, the genre, and the fame of “your” authors all matter for how fast you can accumulate a relevant corpus. And it is generally better for you if you work with edited sources only, since chances are that you find a copy of those on shamela or another grey website. Even better, you might find them on the github page of Maxim Romanov’s project Open Arabic (current URL / new URL). This is by far the largest and best curated repository of Arabic texts you can hope for at the present moment.

Still, it suffers from what I’d like to call the “edition bias”, which distorts our perspective of Arabic literary of, more so, writerly heritage in much the same way as the better-known survival bias and the more recent “digitisation bias” do. It tends to over-represent works of the Abbasid period, for instance, over those of the Ottoman Period. This is not Maximov’s fault, I have to emphasise! It is simply the nature of modern academic tradition which, in turn, also has built upon the modern and traditional Arabic traditions in valuing and hierarchising literary works and other writings according to often implicit standards.

In other words, despite its many merits, I urge you to not to take the numbers it gives you for included works from different centuries as statistically sound representations of the ups-and-downs of actual literary production. I have before spoken about the size of Ibn Tulun’s written corpus: the surviving works alone range in the hundreds. So, what do we find on Open Arabic?

Ibn Tulun works in Open Arabic

Six works, only one of which has a considerable size! Even Ibn Abd al-Hadi (see IbnMibradHanbali), whose works were rarely copied in manuscript, has fourteen entries! One reason for the selection of works by both authors is a continuing interest in the Arab world in their religious and ethical writings. Other works, some of which have been introduced on this blog before, are currently not regarded as edition worthy (you see, there is the bias).

So, one main obstacle in applying stylometry to my own project is the lack of a sufficient corpus. As Dr. Eder explained, however, you would not need to transcribe everything in your corpus. A few passages per book might be enough to get you started. And it might be interesting to run these analyses not only when your corpus has been finalised but regularly in the process.

But I was wondering also whether we can actually use this method for cases like teaching certificates and other documentary sources. Even though Florian Sobieroj has shown in an article, that these texts never assumed a uniform format, we cannot get around the long chains of names connected into long chains of transmitters and interrupted by the likewise long titles of books. There is not much verbal action happening in those—nor in much of hadith literature.

In turn, this is connected to the question which parts of such texts were actually conceived of by their authors. Which parts of such texts could we use as the standard by which to evaluate the authorship of other passages or works? The basmalla? The tamhid? The justification of the work? The chains of transmission? The interpretation of their source material? This is a tricky question and I have no answer for it. And this comes before even problems connected to machine-readability are taken into account. These include the issue of the assimilation of the “wa” to the following word, the distinction search functions make between words with or without article, plural forms, let alone letters that become visible in writing only when the letter is present.

So, where is the silver lining of all those obstacles I keep ranting over? I would say that all of us can be the silver lining. We just need to share our own transcripts with each other and the Open Arabic (soon Open ITI) project in particular, thereby enlarging our shared corpus of machine-readable texts. These can even be partial ones like my contents list of Ibn Tulun’s biographical dictionary al-Ghuraf al-ʿaliyya. And the benefits of sharing your data go way beyond the applicability of stylometric analysis. It is perhaps the best entry point to understand better the opportunities digital humanities offer for our research and its dissemination. Or perhaps I am just idealistic in this matter…

Leave a Reply

Your email address will not be published. Required fields are marked *