Review “An Efficiency Comparison of Document Preparation Systems Used in Academic Research and Development”

Today someone email of of the mailing list that I subscribed about a article wrote by Markus Knauff and Jelica Nejasmic with the title “An Efficiency Comparison of Document Preparation Systems Used in Academic Research and Development” and published at PLOS ONE. This is a open access journal so if you want to read the article before read this post just click on the first link and enjoy the article.

At this post I will make some comments about the article.

How you can compare apples and oranges?

At the introduction we have

Microsoft Word is based on a principle called “What you see is what you get” (WYSIWYG), which means that the user immediately sees the document on the screen as it will appear on the printed page. LaTeX, in contrast, embodies the principle of “What you get is what you mean” (WYGIWYM), which implies that the document is not directly displayed on the screen and changes, such as format settings, are not immediately visible.

The authors are saying that Microsoft Word and LaTeX are different starting from their designs. So how do you want to compare it?

Novices vs Experts

About the division of the participants in groups we have

The participants were divided into 4 groups with 10 participants in each group: Word novices, Word experts, LaTeX novices, and LaTeX experts. Participants were classified as “novices” if they had less than 500 hours of experience with the respective program and “experts” if they had more than 1000 hours of experience with the respective program.

This classification of novices and experts is too much simplistic. With this separation will not be hard to find a Word expert that doesn’t know the difference of add a figure as embed element or as a link.

Note

Figures included as a link are automatically updated when another application change it. LaTeX only support this type of image inclusion so for a far comparison should be requested that Word users use only this type of figure inclusion.

Note

In the same way that with this criterion is easy to find Word expert that doesn’t know basic features of Word is also easy to find LaTeX expert that doesn’t know basic features of LaTeX.

The continuous text

Note

You will find the text used here.

The continuous text looks fair to me.

From the result, I still have questions of how much the LaTeX users know of the IDE they use.

The table text

Note

You will find the table used here.

The table has multi columns and multi lines that, in LaTeX, request the use of the packages multicols and multirow, respectively. Reproduce this table will be almost impossible for any LaTeX novice user and any LaTeX expert will probably use an third party tool, like Calc2LaTeX, to get this done.

The result is, more or less, what I expected. Word and LaTeX users had the same amount of orthographic and grammatical mistakes but the amount of formatting erros and typos is high among LaTeX users (probably because the requested table isn’t easy to be reproduce in LaTeX). I don’t know how the amount of written text was calculate so I will not make any comment.

The equation text

Note

You will find the equations used here.

Except for the equation numbering doesn’t start at one this should be easy for any Word or LaTeX expert. I expect that LaTeX novice have some issues with the last equation since it request the alignment of equations and the use of \nonumber to get ride of the equation numbering in the first two lines.

Note

In LaTeX, the last equation should be type as:

\begin{align}
  \frac{\partial}{\partial \phi_c(j)} \frac{1}{a(\phi_c)} &= \frac{-1}{(a
  (\phi_c))^2} \frac{\partial}{\partial \phi_c(j)} a(\phi_c) \nonumber \\
  &= \frac{-1}{(a(\phi_c))^2} \sum_{r \in \mathcal{R}} \exp(C(r)/c) x_r(j))
  \nonumber \\
  &= \frac{-1}{a(\phi_c)} p_c(x(j))
\end{align}

If you like you could replace ( with \left( and ) with \right).

As expect in this case LaTeX has a small advantage.

Usability questionnaire

This is the part that I’m little happy about since it confirms what many of us “know”:

  • “Word users rated their respective software as less efficient than LaTeX users”,
  • “LaTeX users rated the learnability of their respective software as poorer than Word users”.

Discussion

The author first said that

“Another characteristic of our study is that it is practically impossible to evaluate LaTeX without also evaluating the used editors. In fact, our research measured the efficiency of Word against LaTeX in combination with some editor interfaces.”

and next that

“our results suggest that LaTeX reduces the user’s productivity and results in more orthographical, grammatical, and formatting errors, more typos, and less written text than Microsoft Word over the same duration of time.”

From my experience, the fact that more orthographical and grammatical erros among LaTeX users is due the fact that many LaTeX IDE don’t have the spell check feature enable out of the box and many users don’t configure it. I would like to see how Word users performs with Word spell check disable.