Skeleton tutorial

Raffaele A Calogero

Dissecting the skeleton.R

The skeleton function allows to control a bash script,, located in image in /bin.

The skeleton function has three parameters:

skeleton(group="docker", scratch.folder, data.folder)

The first step in the skeleton function is storing the working folder and grabbing the process time for subsequent performance evaluation.

  #storing the position of the home folder  
  home <- getwd()
  #running time 1
  ptm <- proc.time()

Then, it is tested if docker demon is running,

  #testing if docker is running
  test <- dockerTest()
    cat("\nERROR: Docker seems not to be installed in your system\n")

checking if data folder exists and setting it as working folder,

  #setting the data.folder as working folder
  if (!file.exists(data.folder)){
    cat(paste("\nIt seems that the ",data.folder, " folder does not exist\n"))

checking if scratch folder exists and creating a temporary folder.

  #check  if scratch folder exist
  if (!file.exists(scratch.folder)){
    cat(paste("\nIt seems that the ",scratch.folder, " folder does not exist\n"))
  tmp.folder <- gsub(":","-",gsub(" ","-",date()))
  scrat_tmp.folder=file.path(scratch.folder, tmp.folder)
  writeLines(scrat_tmp.folder,paste(data.folder,"/tempFolderID", sep=""))
  cat("\ncreating a folder in scratch folder\n")

Executing the docker command:

  #executing the docker job
    params <- paste("--cidfile ",data.folder,"/dockerID -v ",scrat_tmp.folder,":/scratch -v ", data.folder, ":/data -d sh /bin/", sep="")
    resultRun <- runDocker(group="sudo",container="", params=params)
    params <- paste("--cidfile ",data.folder,"/dockerID -v ",scrat_tmp.folder,":/scratch -v ", data.folder, ":/data -d sh /bin/", sep="")
    resultRun <- runDocker(group="docker",container="", params=params)

The scripts in is the following:

echo "skeleton 0.0.1"
#setting the scratch folder as workinng directory
#moving to scratch folder
#adding information to file or creating a file
if [ -f "$file" ]
        echo "skeleton 0.0.1" >> $SCRATCH_FOLDER/
        echo "skeleton 0.0.1" > $SCRATCH_FOLDER/
#writing the result file helloworld in data scratch
echo "hello world" > $SCRATCH_FOLDER/helloworld.txt
# creating the file indicating that run is finished 
echo "analysis is finished" > $SCRATCH_FOLDER/
#changing the properties of files and folders in /data/scratch 
chmod 777 -R $SCRATCH_FOLDER/*

It writes hello world in the helloworld.txt and moves helloworld.txt to the scratch folder together with the file, used to store information about the run, and the, used to tell to the R script when the doker job is finished. The scripts is a prototype for the handling of docker application(s).

Lets go back to the skeleton.R dissection:

The resultRun is used to check when the docker job is finished. The results are copied from the scratch to the data folder

 #when container ends copy results from scratch in data.folder
    system(paste("cp ", scrat_tmp.folder, "/* ", data.folder, sep=""))

Then, the computing time is estimated and saved in the file

  #running time 2
  ptm <- proc.time() - ptm
  dir <- dir(data.folder)
  dir <- dir[grep("",dir)]
    con <- file("", "r") <- readLines(con)
    close(con)[length(] <- paste("user run time mins ",ptm[1]/60, sep="")[length(] <- paste("system run time mins ",ptm[2]/60, sep="")[length(] <- paste("elapsed run time mins ",ptm[3]/60, sep="")
  }else{ <- NULL[1] <- paste("run time mins ",ptm[1]/60, sep="")[length(] <- paste("system run time mins ",ptm[2]/60, sep="")[length(] <- paste("elapsed run time mins ",ptm[3]/60, sep="")

The log of the docker job (describing the events observed during the execution of the container) is saved with a name made of the first 12 letters of the docker job ID. After saving the docker instance log, the scratch folder and the docker container are deleted. The dockerID and tempFolderID files are also deleted. The list of the dockers installed on the machine is saved in the data.folder.

   #saving log and removing docker container <- readLines(paste(data.folder,"/dockerID", sep=""), warn = FALSE)
  system(paste("docker logs ", substr(,1,12), " &> ",data.folder,"/", substr(,1,12),".log", sep=""))
  system(paste("docker rm ",, sep=""))
  #removing temporary folder
  cat("\n\nRemoving the temporary file ....\n")
  system(paste("rm -R ",scrat_tmp.folder))
  system("rm -fR")
  system("rm -fR dockerID")
  system("rm  -fR tempFolderID")
  system(paste("cp ",paste(path.package(package="docker4seq"),"containers/containers.txt",sep="/")," ",data.folder, sep=""))