Getting Started with the NYT COVID-19 Data

Prerequisites and Setup

R Packages

You will need to install the following packages:

Some of these packages require package sf, which cannot be installed on the GC R Studio Server. You will need to work on your own machine.

Create Your R Studio Project

You will work with a repository that contains the NYT data as a submodule, so the process of creating an R Studio project will be different than usual.

  1. Open your terminal and cd to wherever you keep your git-controlled projects.
  2. Clone my repository along with its submodule:

    git clone --recursive https://github.com/homerhanumat/nyt-covid-analysis.git
  3. In R Studio ask to create a new Project, in an existing directory. Specify the directory that was created by the clone in the previous step.

Updating

The purpose of having the NYT data as a submodule in our repository is to permit you to keep you to track your analysis work separately from the data on which your analysis is based. With your terminal’s working directory set to the root directory of your project, git commands such as add, commit and (should you choose to create a remote on Github) push will apply to your project only and will have no effect on the contents of the covid-19-data folder within it.

Always remember that the covid-19-data directory in your repo is a git submodule that links to the NYT repo. Don’t modify anything in this directory!

The New York Times updates its repository every day, and you will want to keep up with those changes. The submodule allows you to accomplish this without interfering with any of your analysis work. In order to update the data, take the following to steps:

  1. Run the following command in your terminal:

    git submodule update --remote
  2. Then source the import.R file.

The data tables in your data directory will be updated. To do your analysis, you’ll load them into your Global Environment:

We now illustrate a couple of things you can do with the data.

Line Graphs

Looking at states with a threshold of confirmed cases, make line graphs of the number of cases against the number of days since the threshold was reached.

Have a look:

The line graph

Too many!

Make a function out of this:

Give it a try:

Kentucky Map for a Day

Note

The following plots use R packages that require package sf, which cannot be installed on the GC R Studio Server.

mapview

There is also mapview:

2020-04-05

LS0tCnRpdGxlOiAiR2V0dGluZyBTdGFydGVkIHdpdGggdGhlIE5ZVCBDT1ZJRC0xOSBEYXRhIgpkYXRlOiAiYHIgU3lzLkRhdGUoKWAiCm91dHB1dDoKICBybWRmb3JtYXRzOjpyZWFkdGhlZG93bjoKICAgIGhpZ2hsaWdodDoga2F0ZQogICAgdG9jX2RlcHRoOiAyCiAgICBjb2RlX2Rvd25sb2FkOiB0cnVlCi0tLQoKCmBgYHtyIHNldHVwLCBpbmNsdWRlID0gRkFMU0V9CmxpYnJhcnkodGlkeXZlcnNlKQojbGlicmFyeShyZWFjdGFibGUpCiNsaWJyYXJ5KHBsb3RseSkKI2RldnRvb2xzOjppbnN0YWxsX2dpdGh1YigiVXJiYW5JbnN0aXR1dGUvdXJibm1hcHIiKQojbGlicmFyeSh1cmJubWFwcikKI2xpYnJhcnkobWFwdmlldykKbG9hZCgiZGF0YS9ueXRfY291bnRpZXMucmRhIikKbG9hZCgiZGF0YS9ueXRfc3RhdGVzLnJkYSIpCmtuaXRyOjpvcHRzX2NodW5rJHNldCgKICBzaXplID0gInNtYWxsIiwKICBlcnJvciA9IFRSVUUsCiAgZmlnLmFsaWduID0gImNlbnRlciIsCiAgdGlkeSA9IEZBTFNFCikKYGBgCgojIyBQcmVyZXF1aXNpdGVzIGFuZCBTZXR1cAoKIyMjIFIgUGFja2FnZXMKCllvdSB3aWxsIG5lZWQgdG8gaW5zdGFsbCB0aGUgZm9sbG93aW5nIHBhY2thZ2VzOgoKYGBge3IgZXZhbCA9IEZBTFNFfQppbnN0YWxsLnBhY2thZ2VzKGMoCiAgInRpZHl2ZXJzZSIsCiAgInJlYWN0YWJsZSIsCiAgIm1hcHZpZXciCikpCmRldnRvb2xzOjppbnN0YWxsX2dpdGh1YigiVXJiYW5JbnN0aXR1dGUvdXJibm1hcHIiKQpgYGAKCgpTb21lIG9mIHRoZXNlIHBhY2thZ2VzIHJlcXVpcmUgcGFja2FnZSAqKnNmKiosIHdoaWNoIGNhbm5vdCBiZSBpbnN0YWxsZWQgb24gdGhlIEdDIFIgU3R1ZGlvIFNlcnZlci4gIFlvdSB3aWxsIG5lZWQgdG8gd29yayBvbiB5b3VyIG93biBtYWNoaW5lLgoKCiMjIyBDcmVhdGUgWW91ciBSIFN0dWRpbyBQcm9qZWN0CgpZb3Ugd2lsbCB3b3JrIHdpdGggYSByZXBvc2l0b3J5IHRoYXQgY29udGFpbnMgdGhlIE5ZVCBkYXRhIGFzIGEgc3VibW9kdWxlLCBzbyB0aGUgcHJvY2VzcyBvZiBjcmVhdGluZyBhbiBSIFN0dWRpbyBwcm9qZWN0IHdpbGwgYmUgZGlmZmVyZW50IHRoYW4gdXN1YWwuCgoxLiBPcGVuIHlvdXIgdGVybWluYWwgYW5kIGBjZGAgdG8gd2hlcmV2ZXIgeW91IGtlZXAgeW91ciBnaXQtY29udHJvbGxlZCBwcm9qZWN0cy4KMi4gQ2xvbmUgbXkgcmVwb3NpdG9yeSBhbG9uZyB3aXRoIGl0cyBzdWJtb2R1bGU6CgogICAgYGBgCiAgICBnaXQgY2xvbmUgLS1yZWN1cnNpdmUgaHR0cHM6Ly9naXRodWIuY29tL2hvbWVyaGFudW1hdC9ueXQtY292aWQtYW5hbHlzaXMuZ2l0CiAgICBgYGAKICAgIAozLiAgSW4gUiBTdHVkaW8gYXNrIHRvIGNyZWF0ZSBhIG5ldyBQcm9qZWN0LCBpbiBhbiBleGlzdGluZyBkaXJlY3RvcnkuICBTcGVjaWZ5IHRoZSBkaXJlY3RvcnkgdGhhdCB3YXMgY3JlYXRlZCBieSB0aGUgY2xvbmUgaW4gdGhlIHByZXZpb3VzIHN0ZXAuCgojIyMgVXBkYXRpbmcKClRoZSBwdXJwb3NlIG9mIGhhdmluZyB0aGUgTllUIGRhdGEgYXMgYSBzdWJtb2R1bGUgaW4gb3VyIHJlcG9zaXRvcnkgaXMgdG8gcGVybWl0IHlvdSB0byBrZWVwIHlvdSB0byB0cmFjayB5b3VyIGFuYWx5c2lzIHdvcmsgc2VwYXJhdGVseSBmcm9tIHRoZSBkYXRhIG9uIHdoaWNoIHlvdXIgYW5hbHlzaXMgaXMgYmFzZWQuICBXaXRoIHlvdXIgdGVybWluYWwncyB3b3JraW5nIGRpcmVjdG9yeSBzZXQgdG8gdGhlIHJvb3QgZGlyZWN0b3J5IG9mIHlvdXIgcHJvamVjdCwgZ2l0IGNvbW1hbmRzIHN1Y2ggYXMgYGFkZGAsIGBjb21taXRgIGFuZCAoc2hvdWxkIHlvdSBjaG9vc2UgdG8gY3JlYXRlIGEgcmVtb3RlIG9uIEdpdGh1YikgYHB1c2hgIHdpbGwgYXBwbHkgdG8geW91ciBwcm9qZWN0IG9ubHkgYW5kIHdpbGwgaGF2ZSBubyBlZmZlY3Qgb24gdGhlIGNvbnRlbnRzIG9mIHRoZSBgY292aWQtMTktZGF0YWAgZm9sZGVyIHdpdGhpbiBpdC4KCkFsd2F5cyByZW1lbWJlciB0aGF0IHRoZSBgY292aWQtMTktZGF0YWAgZGlyZWN0b3J5IGluIHlvdXIgcmVwbyBpcyBhIGdpdCBzdWJtb2R1bGUgdGhhdCBsaW5rcyB0byB0aGUgTllUIHJlcG8uICBEb24ndCBtb2RpZnkgYW55dGhpbmcgaW4gdGhpcyBkaXJlY3RvcnkhCgpUaGUgTmV3IFlvcmsgVGltZXMgdXBkYXRlcyBpdHMgcmVwb3NpdG9yeSBldmVyeSBkYXksIGFuZCB5b3Ugd2lsbCB3YW50IHRvIGtlZXAgdXAgd2l0aCB0aG9zZSBjaGFuZ2VzLiAgVGhlIHN1Ym1vZHVsZSBhbGxvd3MgeW91IHRvIGFjY29tcGxpc2ggdGhpcyB3aXRob3V0IGludGVyZmVyaW5nIHdpdGggYW55IG9mIHlvdXIgYW5hbHlzaXMgd29yay4gICBJbiBvcmRlciB0byB1cGRhdGUgdGhlIGRhdGEsIHRha2UgdGhlIGZvbGxvd2luZyB0byBzdGVwczoKCjEuIFJ1biB0aGUgZm9sbG93aW5nIGNvbW1hbmQgaW4geW91ciB0ZXJtaW5hbDoKCiAgICBgYGAKICAgIGdpdCBzdWJtb2R1bGUgdXBkYXRlIC0tcmVtb3RlCiAgICBgYGAKCjIuIFRoZW4gc291cmNlIHRoZSBgaW1wb3J0LlJgIGZpbGUuCgpUaGUgZGF0YSB0YWJsZXMgaW4geW91ciBgZGF0YWAgZGlyZWN0b3J5IHdpbGwgYmUgdXBkYXRlZC4gIFRvIGRvIHlvdXIgYW5hbHlzaXMsIHlvdSdsbCBsb2FkIHRoZW0gaW50byB5b3VyIEdsb2JhbCBFbnZpcm9ubWVudDoKCmBgYHtyIGV2YWwgPSBGQUxTRX0KbG9hZCgiZGF0YS9ueXRfY291bnRpZXMucmRhIikKbG9hZCgiZGF0YS9ueXRfc3RhdGVzLnJkYSIpCmBgYAoKCldlIG5vdyBpbGx1c3RyYXRlIGEgY291cGxlIG9mIHRoaW5ncyB5b3UgY2FuIGRvIHdpdGggdGhlIGRhdGEuCgoKIyMgTGluZSBHcmFwaHMKCkxvb2tpbmcgYXQgc3RhdGVzIHdpdGggYSB0aHJlc2hvbGQgb2YgY29uZmlybWVkIGNhc2VzLCBtYWtlIGxpbmUgZ3JhcGhzIG9mIHRoZSBudW1iZXIgb2YgY2FzZXMgYWdhaW5zdCB0aGUgbnVtYmVyIG9mIGRheXMgc2luY2UgdGhlIHRocmVzaG9sZCB3YXMgcmVhY2hlZC4KCmBgYHtyfQp0aHJlc2hvbGQgPC0gNTAwCmBgYAoKCmBgYHtyfQp0X3N0YXRlcyA8LQogIG55dF9zdGF0ZXMgJT4lIAogIGdyb3VwX2J5KHN0YXRlKSAlPiUgCiAgZmlsdGVyKGNhc2VzID49IHRocmVzaG9sZCkgJT4lIAogIGFycmFuZ2UoZGF0ZSkgJT4lIAogIG11dGF0ZShkYXlfbnVtYmVyID0gcm93X251bWJlcigpKSAlPiUgCiAgYXJyYW5nZShzdGF0ZSkKYGBgCgpIYXZlIGEgbG9vazoKCmBgYHtyfQpyZWFjdGFibGU6OnJlYWN0YWJsZSh0X3N0YXRlcywgc2VhcmNoYWJsZSA9IFRSVUUpCmBgYAoKVGhlIGxpbmUgZ3JhcGgKCgpgYGB7cn0KZ2dwbG90KHRfc3RhdGVzLCBhZXMoeCA9IGRheV9udW1iZXIsIHkgPSBjYXNlcykpICsKICBnZW9tX2xpbmUoYWVzKGNvbG9yID0gc3RhdGUpKQpgYGAKClRvbyBtYW55IQoKTWFrZSBhIGZ1bmN0aW9uIG91dCBvZiB0aGlzOgoKYGBge3J9Cm1ha2VfZ3JhcGggPC0gZnVuY3Rpb24odGhyZXNob2xkID0gNTAwLCBpbmNsdWRlZCkgewogIHAgPC0KICAgIG55dF9zdGF0ZXMgJT4lIAogICAgZmlsdGVyKHN0YXRlICVpbiUgaW5jbHVkZWQpICU+JSAKICAgIGdyb3VwX2J5KHN0YXRlKSAlPiUgCiAgICBmaWx0ZXIoY2FzZXMgPj0gdGhyZXNob2xkKSAlPiUgCiAgICBhcnJhbmdlKGRhdGUpICU+JSAKICAgIG11dGF0ZShkYXlfbnVtYmVyID0gcm93X251bWJlcigpKSAlPiUgCiAgICBnZ3Bsb3QoYWVzKHggPSBkYXlfbnVtYmVyLCB5ID0gY2FzZXMsCiAgICAgICAgICAgICAgIGxhYmVsMSA9IHN0YXRlLCBsYWJlbDIgPSBkYXRlLCAKICAgICAgICAgICAgICAgbGFiZWwzID0gY2FzZXMsIGxhYmVsNCA9IGRlYXRocykpICsKICAgICAgZ2VvbV9saW5lKGFlcyhjb2xvciA9IHN0YXRlKSkgKwogICAgICBsYWJzKHggPSAiZGF5cyBzaW5jZSB0aHJlc2hvbGQgcmVhY2hlZCIsCiAgICAgICAgICAgeSA9ICJjb25maXJtZWQgY2FzZXMiKQogICAgcGxvdGx5OjpnZ3Bsb3RseShwLCB0b29sdGlwID0gYygibGFiZWwxIiwgImxhYmVsMiIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICJsYWJlbDMiLCAibGFiZWw0IikpICU+JSAKICAgICAgcGxvdGx5Ojpjb25maWcoZGlzcGxheU1vZGVCYXIgPSBGQUxTRSkKfQpgYGAKCkdpdmUgaXQgYSB0cnk6CgpgYGB7cn0KaW5jbHVkZWQgPC0gYygiQ2FsaWZvcm5pYSIsICJXYXNoaW5ndG9uIiwgIk5ldyBZb3JrIiwgIkxvdWlzaWFuYSIpCm1ha2VfZ3JhcGgoaW5jbHVkZWQgPSBpbmNsdWRlZCkKYGBgCgojIyBLZW50dWNreSBNYXAgZm9yIGEgRGF5CgojIyMgTm90ZQoKVGhlIGZvbGxvd2luZyBwbG90cyB1c2UgUiBwYWNrYWdlcyB0aGF0IHJlcXVpcmUgcGFja2FnZSAqKnNmKiosIHdoaWNoIGNhbm5vdCBiZSBpbnN0YWxsZWQgb24gdGhlIEdDIFIgU3R1ZGlvIFNlcnZlci4KCiMjIyBEYXRhCgpHZXQgdGhlIGRhdGE6CgpgYGB7cn0KIyMgZ2V0IHRoZSBtb3N0IHJlY2VudCBhdmFpbGFibGUgZGF5OgpkYXkgPC0KICBueXRfY291bnRpZXMgJT4lIAogIHB1bGwoZGF0ZSkgJT4lIAogIHVuaXF1ZSgpICU+JSAKICBzb3J0KGRlY3JlYXNpbmcgPSBUUlVFKSAlPiUgCiAgLlsxXQojIyBkZXZ0b29sczo6aW5zdGFsbF9naXRodWIoIlVyYmFuSW5zdGl0dXRlL3VyYm5tYXByIikKY291bnRpZXNfc2YgPC0gdXJibm1hcHI6OmdldF91cmJuX21hcCgiY291bnRpZXMiLCBzZiA9IFRSVUUpCmt5IDwtCiAgbnl0X2NvdW50aWVzICU+JSAKICBmaWx0ZXIoZGF0ZSA9PSBkYXkgJiBzdGF0ZSA9PSAiS2VudHVja3kiKSAlPiUgCiAgcmlnaHRfam9pbihjb3VudGllc19zZiAlPiUgZmlsdGVyKHN0YXRlX25hbWUgPT0gIktlbnR1Y2t5IiksCiAgICAgICAgICAgICBieSA9IGMoImZpcHMiID0gImNvdW50eV9maXBzIikpICU+JSAKICBtdXRhdGUoY2FzZXMgPSBpZmVsc2UoaXMubmEoY2FzZXMpLCAwLCBjYXNlcyksCiAgICAgICAgIGRlYXRocyA9IGlmZWxzZShpcy5uYShkZWF0aHMpLCAwLCBkZWF0aHMpKQpgYGAKCiMjIyBnZ3Bsb3QyCgoqKmdncGxvdDIqKiBoYXMgbmF0aXZlIHN1cHBvcnQgZm9yIHNpbXBsZSBmZWF0dXJlczoKCgpgYGB7cn0KZ2dwbG90KCkgKwogIGdlb21fc2Yoa3ksCiAgICAgICAgICBtYXBwaW5nID0gYWVzKGZpbGwgPSBjYXNlcywgZ2VvbWV0cnkgPSBnZW9tZXRyeSksCiAgICAgICAgICBjb2xvciA9ICJ3aGl0ZSIsIHNpemUgPSAwLjA1KSArCiAgY29vcmRfc2YoZGF0dW0gPSBOQSkgKwogIGxhYnMoZmlsbCA9ICJDb25maXJtZWQgQ2FzZXMiLAogICAgICAgdGl0bGUgPSBwYXN0ZTAoIktlbnR1Y2t5IENPVklELTE5IENhc2VzIGFzIG9mICIsIGRheSkpCmBgYAoKU3VyZSB3b3VsZCBiZSBuaWNlIHRvIGdldCAqKnBsb3RseSoqIHdvcmtpbmcgd2l0aCB0aGlzLgoKIyMjIG1hcHZpZXcKClRoZXJlIGlzIGFsc28gKiptYXB2aWV3Kio6CgpgYGB7cn0Ka3kgJT4lIAogIHNmOjpzdF9hc19zZigpICU+JSAKICAjIyBpbnN0YWxsLnBhY2thZ2VzKCJtYXB2aWV3IikKICBtYXB2aWV3OjptYXB2aWV3KHpjb2wgPSBjKCJjYXNlcyIsICJkZWF0aHMiLCAiY291bnR5X25hbWUiKSkKYGBgCgoKCgoKCgoKCgoKCg==