R language as a derivative of the S language

• Bell Laboratories were unquestionably the source where R evolved, and specifically the S language. The S language developed from the Fortran libraries, that is the Statistical Computing Subroutines, is said to have given birth to the R language.
• Chambers (1977) documents in his book the achievements and breakthrough realized during his ten years at the Bell Labs.
• Whereas, the Statistics Research Department (SRD) was investigating Exploratory Data Analysis Tukey (1993), the Computer Science Research Department was developing UNIX OS and the C Compiler. (Ritchie and Thompson, 1978)
• At the time, the Statistics Research Department accessed the C language, based on the portable C compiler to develop a graphics facility.
• Becker and Chambers (1977) wrote a subroutine library, GRZ to produce flexible, portable, device-independent graphics, whose parameters controlled graphical layout and rendering. The GRZ system provided for graphic input as well as output and became a cornerstone for graphics in S.
• It should be noted that S like R is:
  • a collection of functions residing in separate single overlays
  • based on formal grammar with as few restrictions as possible, subscripting and object naming.
  • facilitates portability using two character sets (BCD and ASCII) and two system calls (Batch and Time-sharing)
  • enabled with detailed online documentation
  • promoted with the X window system developed by (Scheifler and Gettys, 1986)

GUI:

• The graphical user interface is a form of user interface that allows users to interact with electronic devices through graphical icons and audio indicator such as primary notation, instead of text-based user interfaces, typed command labels or text navigation. Lawrence and Verzani (2018)
• Leeuw (2010) pointed out that the GUI revolution reached the statistical software more or less at the arrival of the Macintosh computers. Around 1985, after the long dominance of the big three statistical packages, that is BMDP, SPSS, and SAS, a program with a truly innovative GUI, DataDesk (Velleman, 1997), was released.
• The R capability of linking to other languages permitted the release of the R Commander (Fox, 2005), which facilitated a GUI for R. Ihaka and Gentleman (1996)

IDE

• The IDE or integrated development environment can be defined as a software that combines, in one place, all the tools needed for developing a project.
• On a more basic level, they provide interfaces for users to write code, organize text groups, and automate programming redundancies. The IDEs combine the functionality of multiple programming processes into one, instead of a bare-bones code editor Allaire (2012),Racine (2012) and Studio (2018).

Why IDE?

• The process of writing, creating, and testing projects requires that developers employ a variety of tools. Text editors, code libraries, bug tracking software, compilers, and test platforms are among the most common development tools.
• An integrated development environment combines several of these development-related technologies into a single framework. When all utilities are represented on the same workbench, developers don’t have to spend hours learning how to operate them separately.
• Most IDE capabilities, such as intelligent code completion and automatic code creation, are designed to save time by eliminating writing out complete character sequences. The integrated toolset aims to make project development easier while also detecting and reducing code errors and typos.
• Some other popular IDE features assist developers in streamlining their workflow and problem-solving. IDEs parse code as it is written, identifying problems caused in real time. Most IDEs also include syntax highlighting, which employs visual clues to discern grammar in the text editor.

Common Features of IDE

• Text editor: Virtually every IDE will have a text editor designed to write and manipulate source code. Some tools may have visual components to drag and drop front-end components, but most have a simple interface highlighting language-specific syntax.
• Debugger: Debugging tools assist users in identifying and remedying errors within source code. They often simulate real-world scenarios to test functionality and performance. Programmers and software engineers can usually test the various code segments and identify errors before the application is released.
• Compiler: Compilers are components that translate programming language into a form machines can process, such as binary code. The machine code is analyzed to ensure its accuracy. The compiler then parses and optimizes the code to optimize performance.
• Code completion: Code complete features assist programmers by intelligently identifying and inserting common code components. These features save developers time writing code and reduce the likelihood of typos and bugs.
• Programming language support: IDEs are typically specific to a single programming language, though several also offer multi-language support. As such, the first step is to figure out which languages you will be coding in and narrow your prospective IDE list down accordingly. Examples include Ruby, Python, and Java IDE tools.
• Integrations and plugins: With the name integrated development environment, it is no surprise that integrations need to be considered when looking at IDEs. Your IDE is your development portal, so incorporating all your other development tools will improve development workflows and productivity. Poor integrations can cause numerous issues and lead to many headaches.

Reproducibility

WHy:

• Standard to judge scientific claims. It opens claims to scrutiny, allowing us to keep what works and discard what doesn’t. It requires the complete and open exchange of data, procedures, and materials. Scientific conclusions that are not replicable should be abandoned or modified.
• Reproducibility enhances replicability. If other researchers are able to clearly understand how a finding was originally made, then they will be able to conduct comparable research in meaningful attempts to replicate the original findings.
• Limits duplication & encourages cumulative knowledge development.
• New knowledge and dependable analyses. Dynamic reproducible documents in particular can make changing things much easier. Changes made to one part of a research project have a way of cascading through the other parts. For example, if you used data imputation or matching methods you may need to rerun these models. You then have to update your main statistical analyses, and recreate the tables and graphs you used to present the results, thus making it easier for yourself as well as others to reproduce your research.

Galton’s data on the heights of parents and their children

Galton (1886) presented these data in a table, showing a cross-tabulation of 928 adult children born to 205 fathers and mothers, by their height and their mid-parent’s height. He visually smoothed the bivariate frequency distribution and showed that the contours formed concentric and similar ellipses, thus setting the stage for correlation, regression and the bivariate normal distribution.

plot with regression line with data ellipses and lowess smooth

  data(Galton)

  with(Galton,
  {
    sunflowerplot(parent,child, xlim=c(62,74), ylim=c(62,74))
    reg <- lm(child ~ parent)
    abline(reg)
    lines(lowess(parent, child), col="blue", lwd=2)
    if(require(car)) {
    dataEllipse(parent,child, xlim=c(62,74), ylim=c(62,74), plot.points=FALSE) }
  })

Halley’s Life Table

• Heywood (1985), Seal (1980): In 1693 the famous English astronomer Edmond Halley studied the birth and death records of the city of Breslau, which had been transmitted to the Royal Society by Caspar Neumann.
• He produced a life table showing the number of people surviving to any age from a cohort born the same year. -He also used his table to compute the price of life annuities. \[\frac{P_{k+1}}{P_k}~~~\forall~~~P_{k+1}=P_k-D_k \]
• This method had the great advantage of not requiring a general census but only knowledge of the number of births and deaths and of the age at which people died during a few years.

    data(HalleyLifeTable)

    # plot survival vs. age
    par(mfrow=c(1,3))
    plot(number ~ age, data=HalleyLifeTable, type="h", ylab="Number surviving")
    # population pyramid is transpose of this -s>
    plot(age ~ number, data=HalleyLifeTable, type="l", xlab="Number surviving")
    with(HalleyLifeTable, segments(0, age, number, age, lwd=2))
    # conditional probability of survival, one more year
    plot(ratio ~ age, data=HalleyLifeTable, ylab="Probability survive one more year")

Macdonell’s Data on Height and Finger Length of Criminals, Gosset (1908)

• In the second issue of Biometrika, Macdonell (1902) published an extensive paper, On Criminal Anthropometry and the Identification of Criminals in which he included numerous tables of physical characteristics about 3000 non-habitual male criminals serving their sentences in England and Wales.
• His Table III (p. 216) recorded a bivariate frequency distribution of height by finger length.
• His main purpose was to show that Scotland Yard could have indexed their material more efficiently, and find a given profile more quickly.
• Gösset (1908) used these data in two classic papers in 1908, in which he derived various characteristics of the sampling distributions of the mean, standard deviation and Pearson’s r.
• He said, “Before I had succeeded in solving my problem analytically, I had endeavoured to do so empirically.” Among his experiments, he randomly shuffled the 3000 observations from Macdonell’s table, and then grouped them into samples of size \(4, 8,\cdots\) calculating the sample means, standard deviations and correlations for each sample. (Hanley et al., 2008)
• Original data looks

Naive contour plots and Galton smoothed 2-D of height and finger

• Smoother 2-D frequencies’ contour plot by Galton
• Worked with a mathematician the iso-density contours of a bivariate Gaussian distribution

Bivariate kernel density estimate of height and finger

Sampling distributions of sample sd (s) and z=(ybar-mu)/s

• Note that Gosset used a divisor of n (not n-1) to get the sd.
• He also used Sheppard’s correction for the ‘binning’ or grouping.
• With concatenated height measurements
• 750 samples of size n=4 (as Gosset did)

## [1] 5.4196250 0.2131819

Is the future of statistical programming now?

How do embrace the RStudio Integrated Development Environment?

Is RStudio really an enabler of Data Analytics?

If S is to R, then R will be to - ?

What if Gosset had R, where would statistics be?