Yog is a logging framework for R inspired by Apache Log4j and Python logging. It is based on R6 classes that focuses on extensibility, ease of use, and performance
Configure a new logger that writes all log messages of level info or more to a text file and of level error or more to a json file.
Setup:
tfi <- tempfile(fileext = ".info")
tfe <- tempfile(fileext = ".error")
l <- Logger$new(
"example Logger",
appenders = list(
AppenderFile$new(tfi, threshold = "info"),
AppenderFile$new(tfe, threshold = "error", layout = LayoutJson$new())
)
)
Usage:
l$info("Everything is okay")
#> INFO [14:10:21.496] Everything is okay
l$error("Everything is NOT okay")
#> ERROR [14:10:21.568] Everything is NOT okay
l$fatal("NOTHING is okay")
#> FATAL [14:10:21.592] NOTHING is okay
readLines(tfi)
#> [1] "INFO [2018-12-21 14:10:21.496] Everything is okay"
#> [2] "ERROR [2018-12-21 14:10:21.568] Everything is NOT okay"
#> [3] "FATAL [2018-12-21 14:10:21.592] NOTHING is okay"
read_json_lines(tfe)
#> level timestamp caller msg
#> 1 200 2018-12-21 14:10:21 eval Everything is NOT okay
#> 2 100 2018-12-21 14:10:21 eval NOTHING is okay
file.remove(tfi, tfe)
#> [1] TRUE TRUE
If you want custom logging configurations, you have to understand the structure of the logging process.
AppenderFile
uses LayoutFormat
by default to write human readable log events to a text file, but can also use LayoutJson
produce machine readable JSON lines logfiles.AppenderFormat
with LayoutFormat
you can only use the standard fields in your log message, while LayoutJson
supports custom fields in a quite natural manner. See examples 1 & 2Yog supports the standard log4j Log Levels outlined bellow. The Log Level of an event represents its severity. Log levels have numeric and character representations. Creation of custom log levels ist not yet supported by yog (but planned for the future).
Level | Name | Description |
---|---|---|
0 | off | A log level of 0/off tells a Logger or Appender to suspend all logging |
100 | fatal | Critical error that leads to program abort. Should always indicate a stop() or similar |
200 | error | A severe error that does not trigger program abort |
300 | warn | A potentially harmful situation, like warning() |
400 | info | An informatinal message on the progress of the application |
500 | debug | Finer grained informational messages that are mostly useful for debugging |
600 | trace | An even finer grained message than debug |
NA | all | A log level of NA/all tells a Logger or Appender to process all log events |
Yog comes with a pre-configured Root Logger. If you do not care about using multiple loggers, and do not want to deal with R6 classes you still use yog as a performant and easy to use logging solution.
FATAL("This is an important message about %s going wrong", "something")
#> FATAL [14:10:21.693] This is an important message about something going wrong
ERROR("A less severe error")
#> ERROR [14:10:21.695] A less severe error
WARN("something likely went wrong")
#> WARN [14:10:21.697] something likely went wrong
INFO("Everything is ok")
#> INFO [14:10:21.699] Everything is ok
DEBUG("Debug messages are hidden by default")
console_threshold("debug") # you must use lower case names here
DEBUG("unless we lower the threshold")
#> DEBUG [14:10:21.702] unless we lower the threshold
TRACE("Trace is even lower than debug")
You can also directly use Logger R6 objects to log. This package comes with a default root logger named like the package ("yog"
). This method of referring to the logger is more explicit (and therefore arguably better). The examples in this vignette will moslty use the notation below.
yog$fatal("This is an important message about %s going wrong", "->something<-")
#> FATAL [14:10:21.710] This is an important message about ->something<- going wrong
yog$trace("Trace messages are still hidden")
yog$appenders$console$set_threshold("trace")
yog$trace("Unless we lower the threshold")
#> TRACE [14:10:21.714] Unless we lower the threshold
(wip)
The root logger only logs to the console by default. If you want to redirect the output to a file you can just add a file appender to yog.
tf <- tempfile()
# Add a new appender to a logger. We don't have to supply a name, but that
# makes it easier to remove later.
yog$add_appender(AppenderFile$new(file = tf), name = "file")
# configure yog so that it logs everything to the file, but only info and above
# to the console
yog$set_threshold(NA)
yog$appenders$console$set_threshold("info")
yog$appenders$file$set_threshold(NA)
yog$info("Another informational message")
#> INFO [14:10:21.751] Another informational message
yog$debug("A debug message not shown by the console appender")
readLines(tf)
#> [1] "INFO [2018-12-21 14:10:21.751] Another informational message"
#> [2] "DEBUG [2018-12-21 14:10:21.753] A debug message not shown by the console appender"
# Remove the appender again
yog$remove_appender("file")
Each newly created Logger is child to a parent Logger. If no parent Logger is specified manually during a Logger’s creation, its parent is the Root Logger. A logger dispatches the LogRecords it creates not only to its own Appenders, but also to the Appenders of all its ancestral Appenders (ignoring the threshold and Filters of the ancestral Loggers, but not of the ancestral Appenders).
yog comes with simple formatting syntax. The configuration possibilites are limited, but should be sufficient for most users.
lg <- Logger$new(
"test",
appenders = list(cons = AppenderConsole$new()),
propagate = FALSE
)
lg$info("the default format")
#> INFO [14:10:21.771] the default format
lg$appenders$cons$layout$set_fmt("%L (%n) [%t] %c(): !! %m !!")
lg$info("A more involved custom format")
#> INFO (400) [14:10:21.774] eval(): !! A more involved custom format !!
More complex formats are possible with LayoutGlue
powered by the awesome glue package, at the cost of slightly worse performance.
# install.packages("glue")
# WIP
JavaScript Object Notation (JSON) is an open-standard file format that uses human-readable text to transmit data objects consisting of attribute–value pairs and array data types (wikipedia). JSON is the recommended text-based logging format when logging to files 1, as it is human- as well as machine readable. You should only log to a different format if you have very good reasons for it.
# install.packages("jsonlite")
tf <- tempfile()
# AppenderJson is just a shortcut for AppenderFile with LayotJson as default
# instead of Layout Format
lg <- Logger$new(
"test logger",
appenders = AppenderJson$new(file = tf),
propagate = FALSE
)
lg$info("JSON naturally supports custom log fields", field = "custom")
lg$info("You can serialize most R data types to JSON", numbers = 1:5)
lg$info("If you ever want to analyse your log files", use = "JSON")
JSON is easy to parse and analyse with R
read_json_lines(tf)
#> level timestamp caller
#> 1 400 2018-12-21 14:10:21 eval
#> 2 400 2018-12-21 14:10:21 eval
#> 3 400 2018-12-21 14:10:21 eval
#> msg field numbers use
#> 1 JSON naturally supports custom log fields custom NULL <NA>
#> 2 You can serialize most R data types to JSON <NA> 1, 2, 3, 4, 5 <NA>
#> 3 If you ever want to analyse your log files <NA> NULL JSON
It is also human readable, though maybe this vignett does not transport that fact very well because of the lack of horizontal space
cat(readLines(tf), sep = "\n\n")
#> {"level":400,"timestamp":"2018-12-21 14:10:21","caller":"eval","msg":"JSON naturally supports custom log fields","field":"custom"}
#>
#> {"level":400,"timestamp":"2018-12-21 14:10:21","caller":"eval","msg":"You can serialize most R data types to JSON","numbers":[1,2,3,4,5]}
#>
#> {"level":400,"timestamp":"2018-12-21 14:10:21","caller":"eval","msg":"If you ever want to analyse your log files","use":"JSON"}
Technically, the logger does not produce standard JSON files but JSON lines↩