Skeleton tutorial

Raffaele A Calogero

Dissecting the skeleton.R

The skeleton function allows to control a bash script,, located in image in /bin.

The skeleton function has three parameters:

skeleton(group="docker", scratch.folder, data.folder)

The first step in the skeleton function is storing the working folder and grabbing the process time for subsequent performance evaluation.

  #storing the position of the home folder  
  home <- getwd()
  #running time 1
  ptm <- proc.time()

Then, it is tested if docker demon is running,

  #testing if docker is running
  test <- dockerTest()
    cat("\nERROR: Docker seems not to be installed in your system\n")

checking if data folder exists and setting it as working folder,

  #setting the data.folder as working folder
  if (!file.exists(data.folder)){
    cat(paste("\nIt seems that the ",data.folder, " folder does not exist\n"))

checking if scratch folder exists and creating a temporary folder.

  #check  if scratch folder exist
  if (!file.exists(scratch.folder)){
    cat(paste("\nIt seems that the ",scratch.folder, " folder does not exist\n"))
  tmp.folder <- gsub(":","-",gsub(" ","-",date()))
  scrat_tmp.folder=file.path(scratch.folder, tmp.folder)
  writeLines(scrat_tmp.folder,paste(data.folder,"/tempFolderID", sep=""))
  cat("\ncreating a folder in scratch folder\n")

Executing the docker command:

  #executing the docker job
    params <- paste("--cidfile ",data.folder,"/dockerID -v ",scrat_tmp.folder,":/scratch -v ", data.folder, ":/data -d sh /bin/", sep="")
    resultRun <- runDocker(group="sudo",container="", params=params)
    params <- paste("--cidfile ",data.folder,"/dockerID -v ",scrat_tmp.folder,":/scratch -v ", data.folder, ":/data -d sh /bin/", sep="")
    resultRun <- runDocker(group="docker",container="", params=params)

The scripts in is the following:

echo "skeleton 0.0.1"
#setting the scratch folder as workinng directory
#moving to scratch folder
#adding information to file or creating a file
if [ -f "$file" ]
        echo "skeleton 0.0.1" >> $SCRATCH_FOLDER/
        echo "skeleton 0.0.1" > $SCRATCH_FOLDER/
#writing the result file helloworld in data scratch
echo "hello world" > $SCRATCH_FOLDER/helloworld.txt
# creating the file indicating that run is finished 
echo "analysis is finished" > $SCRATCH_FOLDER/
#changing the properties of files and folders in /data/scratch 
chmod 777 -R $SCRATCH_FOLDER/*

It writes hello world in the helloworld.txt and moves helloworld.txt to the data folder together with the file, used to store information about the run, and the, used to tell to the R script when the doker job is finished. The scripts is a prototype for the handling of docker application(s).

Lets go back to the skeleton.R dissection:

The resultRun is used to check when the docker job is finished. The log of the docker job is saved with a name made of the first 12 letters of the docker job ID. Then, the docker container is deleted as well as the temporary folder and few other files:, dockerID, tempFolderID. Finally the home folder is restored as working directory.

 #when container ends
   #everything is copied to the input folder
    system(paste("mv ", scrat_tmp.folder,"/* ",data.folder, sep=""))
     #saving log and removing docker container <- readLines(paste(data.folder,"/dockerID", sep=""), warn = FALSE)
    system(paste("docker logs ", substr(,1,12), " &> ", substr(,1,12),".log", sep=""))
    system(paste("docker rm ",, sep=""))
    #removing temporary folder
    cat("\n\nRemoving the temporary file ....\n")
    system(paste("rm -R ",scrat_tmp.folder))
    system("rm -fR")
    system("rm -fR dockerID")
    system("rm  -fR tempFolderID")
    system(paste("cp ",paste(path.package(package="docker4seq"),"containers/containers.txt",sep="/")," ",data.folder, sep=""))

Then, the computing time is estimated and saved in the file

  #running time 2
  ptm <- proc.time() - ptm
  dir <- dir(data.folder)
  dir <- dir[grep("",dir)]
    con <- file("", "r") <- readLines(con)
    close(con)[length(] <- paste("user run time mins ",ptm[1]/60, sep="")[length(] <- paste("system run time mins ",ptm[2]/60, sep="")[length(] <- paste("elapsed run time mins ",ptm[3]/60, sep="")
  }else{ <- NULL[1] <- paste("run time mins ",ptm[1]/60, sep="")[length(] <- paste("system run time mins ",ptm[2]/60, sep="")[length(] <- paste("elapsed run time mins ",ptm[3]/60, sep="")