Reproducible Research and Software Development Methods for Management tasks

Reproducible research is an important topic in today's IT research and it also has a big significance for our society and the ethics of conducting research. It is important to keep the underlying data, algorithms, and argumentation, so that others can follow the analysis of the data and understand the conclusions. Today's interest in SW containers like Docker is apart from being a great deployment mechanism also powered by this desire to allow the conservation of computational environments in an easily sharable form.

The concept of Reproducible research is closely related to the idea of literate programing that Donald Knuth launched with his seminal article in 1984. Literate programming environments allow the free mixing of code parts with extensive documentation parts. The source of a literate programming document contains all necessary data as well as the code for manipulating the data to produce results and plots. It also contains all the text and argumentation. Out of this source document one then can generate a nice publication, but also other derived information (e.g. a standalone software program and a data set, or an excel sheet to give to management).

Org mode for Emacs is one such environment, but there are also others (e.g. look at this list of links maintained by reproducibleresearch.net). Another open source environment in rapid development is given by Jupyter notebooks. They are very popular among scientists. These frameworks are not only great for publishing research in a reproducible manner, but also are becoming one of the best teaching aids, since students can easily experiment with self-contained documents.

Changes of source code should be tracked by revision control systems (RCS) like git. An optimal tracking is possible, if the differences from one version of a document to the next can be easily visualized. This usually requires that the document format is in a text based format and not some closed binary format. The RCS help us to track changes through time, and we also can add metadata in the form of comments to explain the motivations for the changes.

Humans will make errors, and each time one does something non-trivial, especially when it is of a repetitive nature, errors easily creep in. Also, doing changes in one part of a complex thing may cause other parts of our wonderful ivory tower computation to collapse. So, in SW engineering the ideas of unit and intergration tests has become an accepted paradigm.

Versioning. Documents should be versioned in a clearly defined way similar as to what we do with widely accepted standards like semantic versioning.

Templating and Repetition Repetitive parts in a document, e.g. if for a budget of many services a certain basic text/table/graphics structure needs to be repeated multiple times, we would like to generate it by a single code block that we call with different arguments - and not by copy/paste and manual editing. Senseless repetition or copy/paste is useless human toil and leads to errors. later changes to the basic structure should not involve us in making changes in a hundered places.

There exists a plethora of programming languages and libraries to deal with data in all of its forms. In particular, Python is a language which has found increasing adoption by the scientific community over the last decade. It is expressive and provides a wealth of libraries and also frameworks for interactive data analysis. Naturally there are other solutions as well, e.g. the R project.

Since I am a Python person, I will recommend the following libraries for tackling the kind of management related problems we are talking about

• Pandas: easy-to-use data structures and data analysis tools
• numpy: fundamental package for scientific computing. Used by Pandas
• matplotlib and seaborn for visualization

Outliners are ideal for jotting down thoughts and ideas. Some people may prefer mind-maps, but in the end the underlying data structures are mostly identical. The outliner lists are certainly more efficient for a proficient typist and are easily mixed with other parts of a text.

I have used outlines for years for the initial planning of most tasks. Actually the approach is now so engrained with me that I use lists as the starting point for almost all documents I write. Their inherent simple structure allows to express the main ideas as top level items, and then use Sub-items to further refine them or add newer information in a way that is transparent.

I also have used this approach for drawing up project and service budgets for our scientific computing section over the last years, But only now I implemented a better way to automatically derive a useful data structure from such a list in a new Emacs package org-listcruncher (you can get the package from MELPA).

Here is an example of a planning list for a budget of two (non-IT) services.

My rules for writing such a planning list are

1. Each line contains a tag defining wheter the line will become a table row. For this example I defined this as the string "item:". Rows without such a tag just serve as metadata.
2. A string following the output tag "item:" is taken as the description of the table row.
3. Each line can contain any number of key/value pairs in parentheses in the form (key1: val1, key2: val2, ...)
4. Lines of lower hierarchical order in the list inherit their default settings for key/values from the upper items.
5. The key value of a higher order item can be overwritten by a new new value for the same key in a lower order line.

Note that org-listcruncher allows the user to supply an own parsing function, so the rules for how the key/values, the description, and the tagging for a row can get arbitrarily defined by the end user. The present default function is just an easy example.

One can easily imagine using the same kind of approach to derive a table of ingredients and their amounts from the instructions of a cooking recipe that is given as a sequential list of steps. Just needs a bit a smarter parsing function.

We now would like to derive from the planning list a more suitable data format for doing computations. A table is an optimal format. The main function of org-listcruncher provides just this conversion. Based on the semantics defined in the previous section.

The following Org mode code block contains a single line of Emacs lisp for calling org-listcruncher's main function and uses the above list (which I named lstShipGalante) to create a table based on an "outer join" of all the keys found with the respective values arranged in columns.

The list from which we generate the table can later be further edited, and all derived results will change accordingly when recalculating the whole document. The audit trail is conserved in the list itself through the comments, and naturally it will also be conserved in the commits into the revision control system.

Now, as the data is in an easily accessible form, we can use Org mode's Babel feature to read in the table into a python code block and perform some Pandas data manipulation magic on it.

And another little code block lets us visualize cuts of the data with matplotlib and seaborn.

And we can generate the plots for our report.

I will only refer cursorily how to address the other points

• Org mode is part of Emacs which offers a number of powerful templating systems. Org itself supports Macros which can generate text parts.
• Versioning can easily be combined with org mode documents and integrated in the common ways with revision control systems like git (e.g. via tags)
• testing: Since org offers native inclusion of code snippets in different programming languages, tests can easily be formulated in the avaible test frameworks of these languages. One can also define simple tests in form of document code blocks putting out warnings into the document.