Build R project using Make

My default for building projects is to use make. I find it helps organise a projects workflow, even if there are other build tools available. It is succinct, and descriptive. A previous post showed how you can use make to build Haskell projects using GHC, Cabal and Stack. This blog show how to do the same for R projects.

Below is an R script that can be used to render R markdown to PDF. It is invoked from a parent Makefile which is what is read when you call make.

make.R

#!/usr/bin/env R
 
# Render R markdown to PDF.
# Invoke with:
# > R -q -f make.R --args my_report.Rmd
 
# load packages
require(rmarkdown)
 
# require a parameter naming file to render
if (length(args) == 0) {
    stop("Error: missing file operand", call. = TRUE)
} else {
    # read report to render from command line
    for (rmd in commandArgs(trailingOnly = TRUE)) {
        # render Rmd to PDF
        if ( grepl("\\.Rmd$", rmd) && file.exists(rmd)) {
            render(rmd, pdf_document())
        } else {
            print(paste("Ignoring: ", rmd))
        }
    }
}

The above will process Rmd files listed after the –args parameter. This script is invoked from the Makefile below. It declares an rule that describes how to build a PDF from Rmd. The rule calls R with the above R script (make.R).

Makefile

#!/usr/bin/env make
 
.PHONY: all clean
.SUFFIXES: .Rmd .pdf
 
DOCS = my-report
RMDS = $(patsubst %, %.Rmd, $(DOCS))
R = /usr/bin/R
 
Rmd: $(patsubst %.Rmd, %.pdf, $(RMDS))
 
.Rmd.pdf:
    @$(R) --quiet --file=make.R --args $<
 
all: clean Rmd
 
clean:
    @rm -f $(patsubst %.Rmd, %.pdf, $(RMDS))

Note Makefiles must use tabs for indentation, not spaces. My default tab setting is 8 spaces.

Example

Here is a quick example document we will render to PDF:

example.Rmd

---
title: "Example"
author: "Frank Jung"
date: "12 August 2016"
output: pdf_document
---
 
```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```
 
## R Markdown
 
This is an R Markdown document. Markdown is a simple formatting syntax for
authoring HTML, PDF, and MS Word documents. For more details on using R Markdown
see <http://rmarkdown.rstudio.com>.
 
When you click the **Knit** button a document will be generated that includes
both content as well as the output of any embedded R code chunks within the
document. You can embed an R code chunk like this:
 
```{r cars}
summary(cars)
```
 
## Including Plots
 
You can also embed plots, for example:
 
```{r pressure, echo=FALSE}
plot(pressure)
```
 
Note that the `echo = FALSE` parameter was added to the code chunk to prevent
printing of the R code that generated the plot.

Render using above make file with:

make clean all

Renders the PDF as example.pdf.

If you have not used a build tool for you project, I suggest having a look at make. The variant available on most systems is GNU Make. You'll find that there are many languages already supported. When I first started using make, they were primitive and I tended to ignore the implicit rules. Now, I write my own with confidence. I hope you too will experience the benefits of make in your projects ...

Render Jupyter Notebook to HTML or PDF

The following shows how to render a Jupyter Notebook into a static form, such as HTML or PDF document. 

Render to HTML

To render a Jupyter Notebook to HTML:

jupyter nbconvert --exec --to html --output document.html document.ipynb

You can also specify an output directory:

jupyter nbconvert --exec --to html --output document.html --output-dir target document.ipynb

The HTML has all images embedded. This makes sharing easy is the HTML is fully self contained.

Render to PDF

I have had mixed results converting a notebook directly to PDF. The conversion complained that embedded images were not found. Instead I suggest first converting it to LaTeX and then to a PDF.

To convert to Tex run:

jupyter nbconvert --exec --to latex --output document.tex document.ipynb

Now, it is an easy matter to convert TeX to PDF with pdflatex:

pdflatex document.tex

Build using Makefile

A generic Makefile that can be used in a Jupyter project to perform this task is:

Makefile

#!/usr/bin/env make
 
.PHONY: html pdf clean
.SUFFIXES: .ipynb .tex .pdf .html
.DEFAULT: html
 
IPYNB := $(wildcard *.ipynb)
TEXS := $(patsubst %.ipynb, %.tex, $(IPYNB))
HTMLS := $(patsubst %.tex, %.html, $(TEXS))
PDFS := $(patsubst %.tex, %.pdf, $(TEXS))
 
.ipynb.tex:
 -jupyter nbconvert --to=latex $<
 
.tex.pdf:
 -latexmk -f -gg -quiet -pdf \
  -interaction=nonstopmode -shell-escape \
  -pdflatex="pdflatex %O %S" $<
 
.ipynb.html:
 -jupyter nbconvert --exec --to html --output $@ $<
 
html: $(HTMLS)
 
pdf: $(PDFS)
 
clean:
 -latexmk -quiet -f -c $(TEXS)
 @$(RM) -rf $(wildcard *_files)
ifneq ("$(TEXS)", "")
 @$(RM) $(patsubst %.tex, %.*.*, $(TEXS))
 @$(RM) $(TEXS)
endif

Note: 

Use tabs (default is set to 8) and not spaces as make is whitespace sensitive.

Dependencies

Assuming you have Jupyter installed, then the other Debian Linux dependencies for this project are:

• latexmk
• texlive-latex-base
• jupyter nbconvert
• make

Build Haskell projects using Make

Make has been around for a long time. It does, however, have a moderate learning curve. Once you understand and overcome some of its idiosyncrasies, it is a very powerful generic build tool. Below are my variants for building Haskell projects using make. Most of my projects now use Stack. Even so, having a Makefile helps to coordinate building project artefacts. It becomes an organising script for the project.

A full description on building modules can be found in the chapter Using Make in the GHC documentation. For a single target, you can use make to build a Haskell project.

Make is whitespace sensitive. You must use tabs. The default is to use a tab setting of 8 spaces.

GHC

Below is a version using just GHC for compilation. We are defining a custom pattern to build from Haskell source (.hs) to an object (.o). This rule is cheating, as the end result is actually a executable, but as I didn't want to add a .exe extension, we are 'faking' it with testing just the object.

The build is dependent upon three other targets:

1. build tags for use in editors like Vim and Emacs
2. style to ensure consistent source code formatting
3. lint to check code for bad code smells

Makefile

#!/usr/bin/env make
 
.SUFFIXES:
.SUFFIXES: .o .hs
 
%:%.hs
        -ghc -Wall -O2 -threaded --make $<
 
.DEFAULT: build
 
SRCS    := $(wildcard *.hs)
TGTS    := $(patsubst %.hs, %, $(SRCS))
 
.PHONY: build
build:  check $(TGTS)
 
.PHONY: check
check:  tags style lint
 
.PHONY: all
all:    cleanall build
 
style:  $(SRCS)
        -stylish-haskell --config=.stylish-haskell.yaml --inplace $(SRCS)
 
lint:   $(SRCS)
        -hlint --color --show $(SRCS)
 
tags:   $(SRCS)
        -hasktags --ctags $(SRCS)
 
.PHONY: clean
clean:
        # replaces hs with o / hi in list of sources
        -$(RM) $(patsubst %.hs, %.o, $(SRCS))
        -$(RM) $(patsubst %.hs, %.hi, $(SRCS))
 
.PHONY: cleanall
cleanall: clean
        -$(RM) $(TGTS)

Cabal

Below we are using Cabal for compilation. This is essentially the same as for the build for GHC, but Cabal includes targets for checking, unit testing, performance benchmarks, documentation and execution. There is some setup involved to do all this, but that is described in the Cabal documentation. Cabal is useful if your project is using packages beyond the Prelude.

Makefile

#!/usr/bin/env make
 
TARGET := example
SUBS := $(wildcard */)
SRCS := $(wildcard $(addsuffix *.hs, $(SUBS)))
 
ARGS ?= -h
 
build: check
 @cabal build
 
.PHONY: all
all: check build test bench doc tags
 
.PHONY: style
style:
 @stylish-haskell -c .stylish-haskell.yaml -i $(SRCS)
 
.PHONY: lint
lint:
 @hlint --color $(SRCS)
 
.PHONY: check
check: lint style
 @cabal check
 
.PHONY: exec
exec: # Example:  make ARGS="112 12" exec
 @cabal exec $(TARGET) -- $(ARGS)
 
.PHONY: test
test:
 @cabal test
 
.PHONY: bench
bench:
 @cabal bench
 
.PHONY: tags
tags:
 -hasktags --ctags $(SRCS)
 
.PHONY: doc
doc:
 @cabal haddock
 
.PHONY: ghci
ghci:
 -ghci -Wno-type-defaults
 
.PHONY: clean
clean:
 @cabal clean

Stack

Stack is a more complete project management tool, built on top of Cabal. Where Cabal was used in the Makefile previously, it has now been replaced by calls to Stack:

Makefile

#!/usr/bin/env make
 
TARGET := example
SUBS := $(wildcard */)
SRCS := $(wildcard $(addsuffix *.hs, $(SUBS)))
 
ARGS ?= -h
 
build: check
 @stack build
 
.PHONY: all
all: check build test bench doc tags
 
.PHONY: style
style:
 @stylish-haskell -c .stylish-haskell.yaml -i $(SRCS)
 
.PHONY: lint
lint:
 @hlint --color $(SRCS)
 
.PHONY: check
check: lint style
 
.PHONY: exec
exec: # Example:  make ARGS="-h" exec
 @stack exec $(TARGET) -- $(ARGS)
 
.PHONY: test
test:
 @stack test
 
.PHONY: bench
bench:
 @stack bench
 
.PHONY: tags
tags:
 @hasktags --ctags $(SRCS)
 
.PHONY: install
install:
 @stack install
 
.PHONY: doc
doc:
 @stack haddock
 
.PHONY: ghci
ghci:
 @stack ghci --ghci-options -Wno-type-defaults
 
.PHONY: clean
clean:
 @stack clean
 
.PHONY: cleanall
cleanall: clean
 @$(RM) -rf .stack-work/

Other Examples

I hope these examples are a useful starting point. They could be extended. For example, by including a setup task. This would bootstrap a project, ensuring dependencies are met. Other tasks would be to package or deploy a target. 

You can find other Haskell project examples using make on my GitHub pages here.