R Notebook

https://www.r-bloggers.com/how-to-map-your-google-location-history-with-r/ https://towardsdatascience.com/analyzing-my-google-location-history-d3a5c56c7b70

I’ve just been on sabbatical for 6 months mostly in New York (at NYU) and London (at UCL). Returning back, one thing I was really reminded of was how frustrating I find walking in Sydney, because the traffic lights all require use of pedestrian push buttons, and most roads are in used for 2 way traffic. In both London and New York it’s relatively easy to take back streets where cars are much slower and often one way, and the cycle lanes, etc. are much better (meaning no cyclists on the pavement). I’m always amazed more people don’t run/cycle in, particularly from the active area of Bondi - but it’s really quite unpleasant to do so. So, I was curious, is Sydney a slower walking city than London/NYC?

To find out, I took my Google Takeout timeline data (my journeys), and compared average walking speeds for the 3 cities.

library(jsonlite)

package 㤼㸱jsonlite㤼㸲 was built under R version 3.5.3

library(dplyr)

package 㤼㸱dplyr㤼㸲 was built under R version 3.5.3
Attaching package: 㤼㸱dplyr㤼㸲

The following objects are masked from 㤼㸱package:stats㤼㸲:

    filter, lag

The following objects are masked from 㤼㸱package:base㤼㸲:

    intersect, setdiff, setequal, union

library(maptools)

package 㤼㸱maptools㤼㸲 was built under R version 3.5.3Loading required package: sp
package 㤼㸱sp㤼㸲 was built under R version 3.5.3Checking rgeos availability: FALSE
    Note: when rgeos is not available, polygon geometry     computations in maptools depend on gpclib,
    which has a restricted licence. It is disabled by default;
    to enable gpclib, type gpclibPermit()

setwd("C:/Users/125295_admin/Oxygen Enterprise/Teaching student data/Takeout")
library(revgeo)

package 㤼㸱revgeo㤼㸲 was built under R version 3.5.3

library(psych)

package 㤼㸱psych㤼㸲 was built under R version 3.5.2

library(stringr)
library(tidyr)

package 㤼㸱tidyr㤼㸲 was built under R version 3.5.2

library(doBy)

package 㤼㸱doBy㤼㸲 was built under R version 3.5.2

library(ggplot2)

package 㤼㸱ggplot2㤼㸲 was built under R version 3.5.3
Attaching package: 㤼㸱ggplot2㤼㸲

The following objects are masked from 㤼㸱package:psych㤼㸲:

    %+%, alpha

The JSON file is very large. I used a program called Sublime to delete the first chunk of it. The epoch time for 31 Aug 2019 12.34 GMT is 1567254840 If you want Aug 2019 search for first entry: “156 delete everything above that, ensuring you keep the opening {”locations" : [ {

BUT that file doesn’t seem to contain all the details. So, instead, we want to merge multiple json files for the months

Instead, I merged the files by just appending the (take the august file and copy everything between the top and bottom parts from sept/oct/nov, etc. into that one, maknig ure to add a comma…you could this programatically)

#temp <- as.data.frame(fromJSON("Location History/Semantic Location History/2020/2020_FEBRUARY.json"))
temp <- as.data.frame(fromJSON("Location History/merge.json"))

Error: lexical error: invalid char in json text.
                                       Location History/merge.json
                     (right here) ------^

First off we’re going to convert the lat/long to something that looks more typical

activity$lat <- activity$startLocation.latitudeE7/1E7
activity$lon <-  activity$startLocation.longitudeE7/1E7

Then we’re going to take those locations and find an address using revgeo (it has terms of use, and it’s slow). This is messy because I’m being lazy to wrangle it into the shape I find easier to work with, there are more elegant ways to do this

table(unlist(activity$cities))


  London New York     none   Sydney 
     470      476     1658      215

then we want to find out how long each trip is…this is all assuming these are accurate representations

then compute the average speed we probably want to convert distance to something more sensible first It’s probably sensible to cut out very short distances

Then, summarise the data by city. You can filter results e.g. to get rid of implausible data points. Use principles to do that (e.g. average human walking times), or rules of thumb for outliers

LS0tDQp0aXRsZTogIlIgTm90ZWJvb2siDQpvdXRwdXQ6IGh0bWxfbm90ZWJvb2sNCi0tLQ0KDQpodHRwczovL3d3dy5yLWJsb2dnZXJzLmNvbS9ob3ctdG8tbWFwLXlvdXItZ29vZ2xlLWxvY2F0aW9uLWhpc3Rvcnktd2l0aC1yLw0KaHR0cHM6Ly90b3dhcmRzZGF0YXNjaWVuY2UuY29tL2FuYWx5emluZy1teS1nb29nbGUtbG9jYXRpb24taGlzdG9yeS1kM2E1YzU2YzdiNzANCg0KSSd2ZSBqdXN0IGJlZW4gb24gc2FiYmF0aWNhbCBmb3IgNiBtb250aHMgbW9zdGx5IGluIE5ldyBZb3JrIChhdCBOWVUpIGFuZCBMb25kb24gKGF0IFVDTCkuIFJldHVybmluZyBiYWNrLCBvbmUgdGhpbmcgSSB3YXMgcmVhbGx5IHJlbWluZGVkIG9mIHdhcyBob3cgZnJ1c3RyYXRpbmcgSSBmaW5kIHdhbGtpbmcgaW4gU3lkbmV5LCBiZWNhdXNlIHRoZSB0cmFmZmljIGxpZ2h0cyBhbGwgcmVxdWlyZSB1c2Ugb2YgcGVkZXN0cmlhbiBwdXNoIGJ1dHRvbnMsIGFuZCBtb3N0IHJvYWRzIGFyZSBpbiB1c2VkIGZvciAyIHdheSB0cmFmZmljLiBJbiBib3RoIExvbmRvbiBhbmQgTmV3IFlvcmsgaXQncyByZWxhdGl2ZWx5IGVhc3kgdG8gdGFrZSBiYWNrIHN0cmVldHMgd2hlcmUgY2FycyBhcmUgbXVjaCBzbG93ZXIgYW5kIG9mdGVuIG9uZSB3YXksIGFuZCB0aGUgY3ljbGUgbGFuZXMsIGV0Yy4gYXJlIG11Y2ggYmV0dGVyIChtZWFuaW5nIG5vIGN5Y2xpc3RzIG9uIHRoZSBwYXZlbWVudCkuIEknbSBhbHdheXMgYW1hemVkIG1vcmUgcGVvcGxlIGRvbid0IHJ1bi9jeWNsZSBpbiwgcGFydGljdWxhcmx5IGZyb20gdGhlIGFjdGl2ZSBhcmVhIG9mIEJvbmRpIC0gYnV0IGl0J3MgcmVhbGx5IHF1aXRlIHVucGxlYXNhbnQgdG8gZG8gc28uIFNvLCBJIHdhcyBjdXJpb3VzLCBpcyBTeWRuZXkgYSBzbG93ZXIgd2Fsa2luZyBjaXR5IHRoYW4gTG9uZG9uL05ZQz8NCg0KVG8gZmluZCBvdXQsIEkgdG9vayBteSBHb29nbGUgVGFrZW91dCB0aW1lbGluZSBkYXRhIChteSBqb3VybmV5cyksIGFuZCBjb21wYXJlZCBhdmVyYWdlIHdhbGtpbmcgc3BlZWRzIGZvciB0aGUgMyBjaXRpZXMuDQoNCmBgYHtyfQ0KbGlicmFyeShqc29ubGl0ZSkNCmxpYnJhcnkoZHBseXIpDQpsaWJyYXJ5KG1hcHRvb2xzKQ0Kc2V0d2QoIkM6L1VzZXJzLzEyNTI5NV9hZG1pbi9PeHlnZW4gRW50ZXJwcmlzZS9UZWFjaGluZyBzdHVkZW50IGRhdGEvVGFrZW91dCIpDQpsaWJyYXJ5KHJldmdlbykNCmxpYnJhcnkocHN5Y2gpDQpsaWJyYXJ5KHN0cmluZ3IpDQpsaWJyYXJ5KHRpZHlyKQ0KbGlicmFyeShkb0J5KQ0KbGlicmFyeShnZ3Bsb3QyKQ0KYGBgDQoNClRoZSBKU09OIGZpbGUgaXMgdmVyeSBsYXJnZS4gSSB1c2VkIGEgcHJvZ3JhbSBjYWxsZWQgU3VibGltZSB0byBkZWxldGUgdGhlIGZpcnN0IGNodW5rIG9mIGl0Lg0KVGhlIGVwb2NoIHRpbWUgZm9yIDMxIEF1ZyAyMDE5IDEyLjM0IEdNVCBpcyAxNTY3MjU0ODQwDQpJZiB5b3Ugd2FudCBBdWcgMjAxOSBzZWFyY2ggZm9yIGZpcnN0IGVudHJ5OiAiMTU2DQpkZWxldGUgZXZlcnl0aGluZyBhYm92ZSB0aGF0LCBlbnN1cmluZyB5b3Uga2VlcCB0aGUgb3BlbmluZyANCnsNCiAgImxvY2F0aW9ucyIgOiBbIHsNCiANCg0KQlVUIHRoYXQgZmlsZSBkb2Vzbid0IHNlZW0gdG8gY29udGFpbiBhbGwgdGhlIGRldGFpbHMuIFNvLCBpbnN0ZWFkLCB3ZSB3YW50IHRvIG1lcmdlIG11bHRpcGxlIGpzb24gZmlsZXMgZm9yIHRoZSBtb250aHMNCg0KSW5zdGVhZCwgSSBtZXJnZWQgdGhlIGZpbGVzIGJ5IGp1c3QgYXBwZW5kaW5nIHRoZSAodGFrZSB0aGUgYXVndXN0IGZpbGUgYW5kIGNvcHkgZXZlcnl0aGluZyBiZXR3ZWVuIHRoZSB0b3AgYW5kIGJvdHRvbSBwYXJ0cyBmcm9tIHNlcHQvb2N0L25vdiwgZXRjLiBpbnRvIHRoYXQgb25lLCBtYWtuaWcgdXJlIHRvIGFkZCBhIGNvbW1hLi4ueW91IGNvdWxkIHRoaXMgcHJvZ3JhbWF0aWNhbGx5KQ0KYGBge3J9DQojdGVtcCA8LSBhcy5kYXRhLmZyYW1lKGZyb21KU09OKCJMb2NhdGlvbiBIaXN0b3J5L1NlbWFudGljIExvY2F0aW9uIEhpc3RvcnkvMjAyMC8yMDIwX0ZFQlJVQVJZLmpzb24iKSkNCnRlbXAgPC0gYXMuZGF0YS5mcmFtZShmcm9tSlNPTigiTG9jYXRpb24gSGlzdG9yeS9TZW1hbnRpYyBMb2NhdGlvbiBIaXN0b3J5L21lcmdlLmpzb24iKSkNCmFjdGl2aXR5IDwtIGFzLmRhdGEuZnJhbWUoZmxhdHRlbih0ZW1wJHRpbWVsaW5lT2JqZWN0cy5hY3Rpdml0eVNlZ21lbnQpKQ0KDQpgYGANCg0KRmlyc3Qgb2ZmIHdlJ3JlIGdvaW5nIHRvIGNvbnZlcnQgdGhlIGxhdC9sb25nIHRvIHNvbWV0aGluZyB0aGF0IGxvb2tzIG1vcmUgdHlwaWNhbA0KYGBge3J9DQphY3Rpdml0eSRsYXQgPC0gYWN0aXZpdHkkc3RhcnRMb2NhdGlvbi5sYXRpdHVkZUU3LzFFNw0KYWN0aXZpdHkkbG9uIDwtICBhY3Rpdml0eSRzdGFydExvY2F0aW9uLmxvbmdpdHVkZUU3LzFFNw0KYGBgDQoNClRoZW4gd2UncmUgZ29pbmcgdG8gdGFrZSB0aG9zZSBsb2NhdGlvbnMgYW5kIGZpbmQgYW4gYWRkcmVzcyB1c2luZyByZXZnZW8gKGl0IGhhcyB0ZXJtcyBvZiB1c2UsIGFuZCBpdCdzIHNsb3cpLiBUaGlzIGlzIG1lc3N5IGJlY2F1c2UgSSdtIGJlaW5nIGxhenkgdG8gd3JhbmdsZSBpdCBpbnRvIHRoZSBzaGFwZSBJIGZpbmQgZWFzaWVyIHRvIHdvcmsgd2l0aCwgdGhlcmUgYXJlIG1vcmUgZWxlZ2FudCB3YXlzIHRvIGRvIHRoaXMNCmBgYHtyfQ0KYWN0aXZpdHkkbG9va3VwIDwtIG1hcHBseShmdW5jdGlvbih4LHkpIHJldmdlbyh4LHkpLA0KICBhY3Rpdml0eSRsb24sIGFjdGl2aXR5JGxhdCkNCg0KI3RoZW4gSSBuZWVkIHRvIGNoZWNrIGlmIHRoYXQgY29udGFpbnMgc3lkbmV5LCBsb25kb24sIG9yIG5ldyB5b3JrDQphY3Rpdml0eSRjaXRpZXMgPC0gbGFwcGx5KGFjdGl2aXR5JGxvb2t1cCwgZnVuY3Rpb24oeCkgc3RyX2V4dHJhY3RfYWxsKHVubGlzdCh4KSwgcmVnZXgoImxvbmRvbnxzeWRuZXl8bmV3IHlvcmsiLCBpZ25vcmVfY2FzZSA9IFRSVUUpLCBzaW1wbGlmeSA9IEZBTFNFKSkNCg0KYWN0aXZpdHkkY2l0aWVzIDwtIGxhcHBseShhY3Rpdml0eSRjaXRpZXMsIHVuaXF1ZSkgI2dldCByaWQgb2YgYW55IGR1cGxpY2F0ZSBjaXRpZXMNCg0KYWN0aXZpdHkkY2l0aWVzIDwtIGxhcHBseShhY3Rpdml0eSRjaXRpZXMsIHVubGlzdCkgI2ZsYXR0ZW4gdGhlIGxpc3QNCg0KdGFibGUodW5saXN0KGFjdGl2aXR5JGNpdGllcykpDQoNCmFjdGl2aXR5JGNpdGllcyA8LSB1bmxpc3QocmVwbGFjZShhY3Rpdml0eSRjaXRpZXMsICFzYXBwbHkoYWN0aXZpdHkkY2l0aWVzLCBsZW5ndGgpLCJub25lIikpICNwdXQgc29tZXRoaW5nIGluIHRoZSBlbXB0eSB2YWx1ZXMNCmBgYA0KDQp0aGVuIHdlIHdhbnQgdG8gZmluZCBvdXQgaG93IGxvbmcgIGVhY2ggdHJpcCBpcy4uLnRoaXMgaXMgYWxsIGFzc3VtaW5nIHRoZXNlIGFyZSBhY2N1cmF0ZSByZXByZXNlbnRhdGlvbnMNCmBgYHtyfQ0KI3RoZW4gZmluZCB0aGUgZHVyYXRpb24NCmFjdGl2aXR5JHN0YXJ0IDwtIHRlbXAkdGltZWxpbmVPYmplY3RzLmFjdGl2aXR5U2VnbWVudCRkdXJhdGlvbiRzdGFydFRpbWVzdGFtcE1zDQphY3Rpdml0eSRlbmQgPC0gdGVtcCR0aW1lbGluZU9iamVjdHMuYWN0aXZpdHlTZWdtZW50JGR1cmF0aW9uJGVuZFRpbWVzdGFtcE1zDQoNCmFjdGl2aXR5JGR1cmF0aW9uIDwtIGFzLm51bWVyaWMoYWN0aXZpdHkkZW5kKS1hcy5udW1lcmljKGFjdGl2aXR5JHN0YXJ0KSAjZHVyYXRpb24gaW4gbXMuLi4NCg0KYWN0aXZpdHkkZHVyYXRpb24gPC0gYWN0aXZpdHkkZHVyYXRpb24vMTAwMC82MCAjdGhpcyBjb252ZXJ0cyBpdCB0byBtaW51dGVzDQoNCmRlc2NyaWJlKGFzLm51bWVyaWModW5saXN0KGFjdGl2aXR5JGR1cmF0aW9uKSkpDQoNCmBgYA0KDQoNCnRoZW4gY29tcHV0ZSB0aGUgYXZlcmFnZSBzcGVlZA0Kd2UgcHJvYmFibHkgd2FudCB0byBjb252ZXJ0IGRpc3RhbmNlIHRvIHNvbWV0aGluZyBtb3JlIHNlbnNpYmxlIGZpcnN0DQpJdCdzIHByb2JhYmx5IHNlbnNpYmxlIHRvIGN1dCBvdXQgdmVyeSBzaG9ydCBkaXN0YW5jZXMgDQpgYGB7cn0NCmFjdGl2aXR5JHNwZWVkIDwtIGFjdGl2aXR5JGR1cmF0aW9uLyhhY3Rpdml0eSRkaXN0YW5jZS8xMDAwKSAjbWludXRlcy9rbSwgaS5lLiwgdGhlIG51bWJlciBvZiBtaW51dGVzIGVhY2gga20gdGFrZXMgb3IgdGhlIHBhY2UgKFRISVMgSVMgTk9UIFRIRSBTQU1FIEFTIGttIHBlciBob3VyKQ0KZGVzY3JpYmUoYWN0aXZpdHkkc3BlZWQpIA0KDQpgYGANCg0KVGhlbiwgc3VtbWFyaXNlIHRoZSBkYXRhIGJ5IGNpdHkuICBZb3UgY2FuIGZpbHRlciByZXN1bHRzIGUuZy4gdG8gZ2V0IHJpZCBvZiBpbXBsYXVzaWJsZSBkYXRhIHBvaW50cy4gVXNlIHByaW5jaXBsZXMgdG8gZG8gdGhhdCAoZS5nLiBhdmVyYWdlIGh1bWFuIHdhbGtpbmcgdGltZXMpLCBvciBydWxlcyBvZiB0aHVtYiBmb3Igb3V0bGllcnMNCmBgYHtyfQ0KI3RoZW4gZG8gaXQgYnkgY2l0eQ0Kc3VtbWFyeUJ5KHNwZWVkfmNpdGllcywgbmEub21pdChzdWJzZXQoYWN0aXZpdHksIGFjdGl2aXR5VHlwZSA9PSAiV0FMS0lORyIsIHNlbGVjdD1jKCJzcGVlZCIsImNpdGllcyIpKSkpDQoNCnggPC0gc3Vic2V0KGFjdGl2aXR5LCBhY3Rpdml0eVR5cGUgPT0gIldBTEtJTkciICYgc3BlZWQgPiA4ICYgc3BlZWQgPCAyMCAmIGRpc3RhbmNlID4gODAwLCBzZWxlY3Q9Yygic3BlZWQiLCJjaXRpZXMiKSkNCg0KDQpkZXNjcmliZUJ5KHgkc3BlZWQsIHgkY2l0aWVzKSAgI3R5aGlzIHNob3VsZCBnaXZlIG1lIGRlc2NyaXB0aXZlIHN0YXRzIGJ5IGNpdHkuLi5ub3Qgc3VyZSB3aHkgaXQgaXNuJ3Qgd29ya2luZw0KDQpnZ3Bsb3QoeCwgYWVzKHggPSBzcGVlZCkpICsgDQogIGdlb21faGlzdG9ncmFtKGFlcyhjb2xvdXIgPSBjaXRpZXMpLCBmaWxsID0gIndoaXRlIiwNCiAgICAgICAgICAgICAgICAgcG9zaXRpb24gPSAiaWRlbnRpdHkiKQ0KDQojeW91IGRvbid0IHdhbnQgdGhpcyBieSBjb3VudCB5b3Ugd2FudCBkZW5zaXR5IChiZWNhdXNlIHRoZXkgYWxsIGhhdmUgZGlmZmVyZW50IGJhc2VsaW5lcykNCg0KYGBgDQoNCg==