shellpipes

An R package for working through Makefile and command lines. Designed to be used with pipeR.mk (part of makestuff).

The shellpipes code is here (the back end of the repo that hosts this page).

Installation

From inside R:

remotes::install_github("dushoff/shellpipes")

This will fail for old versions of remotes (before 2.2.0), because the default branch is now “main” instead of “master”. Adding , ref="main" before the last parenthesis may fix this problem, but you should probably update remotes instead:

install.packages("remotes")

… and then try again.

Alternative If you can’t install_github because of permission problems, try:

• download this zip file
• unzip it
• R CMD INSTALL shellpipes-main from the directory where you unzipped.

Overview

shellpipes is designed to:

• make working with makestuff pipelines easy and transparent
• make it relatively easy to use the same script both in a make pipeline and interactively

shellpipes functions work with a list of filenames that can be specified manually, but is typically specified by either:

• command-line arguments that include the word shellpipes
  • these are usually passed by make
• an rpcall() command
  • this command is often auto-generated by make and pasted into the script
  • the rpcall() is ignored if the command-line argument shellpipes is present (i.e., if the script is run using pipeR rules)
  • rpcall arguments are associated with a target; there can be more than one target per script. Any time you make <target>.Rout, the pipeR default rules will generate a file called <target>.Rout.args, which contains an rpcall statement that can be copy-pasted into the relevant script
  • rpcall statements supersede each other; if you are working with a script that produces multiple targets, you can put the rpcall for your current target last.

The idea is that shellpipes expects to read from files in the filename list, and to write to files with the same basename as the target. pipeR.mk recognizes that .Rout files are made using a shellpipe-friendly call, and that .rda, .rds, .Rout.* and other sorts of files are made as side-effects of .Rout.

A few example functions are listed below. Please see documentation from installed package (BB: feel free to tell me what’s the recommended way to put documentation online).

Setup

library(shellpipes) ## to load the package
startGraphics() ## not strictly necessary, but good practice; can also be used to set height and width
## startGraphics() does not do anything in interactive mode unless you over-ride a switch; it's meant to play nicely with interactive users

Reading files

• loadEnvironments() loads all environments (typically saved as .rda) from the filename list sequentially into the global environment; getEnvironment() and loadEnvironmentList() allow you to manipulate environments separately
• rdsRead() returns a single serialized object (typically saved as .rds) from the filename list; you can use pattern matching to select if necessary
• csvRead() reads a matched csv file using read_csv()

Saving files

• saveEnvironment() saves the whole global environment as an .rda (using the target name as a base by default)
• saveVars() saves selected variables to an .rda
• rdsSave() saves a single selected variable to an .rds