R Markdown

Here is a sample of the dataset:

Pings

We have four data dimensions: location, time, pings, devices

1. Location: proportion of the records with trailing zeros coordinates

Pings are mostly given with 6 decimal places

Some pings have fewer than 6 decimal places in their coordinates

Assuming that the number of decimal places can be between 2 and 6 (we will count 0 and 1 decimal place as 2), the fraction of points where the longitude has exactly 6 decimal places should be 90%. Indeed, the 6th decimal place can be 0, 1, 2, …, 9. In 9 out of 10 cases, the longitude has 6 decimal places and in one case, i.e., when the 6th decimal place is 0, the number of decimal places is 5 or less. By the same logic, the fraction of points whose longtitude has 5 decimal places should be 9% (90% of the remaining 10%), the fraction of points whose longitude has 4 decimal places should be 0.9% etc.

The same logic is applied to latitude

However, we observe different frequencies. South Park I:

Science Park II:

One North:

There are too many pings with abnormally few decimal places. There presence can be explained by

• Rounding errors due to glitches in GPS.

• IP-based geolocation. There are pings whose location was collected via Wi-Fi rather than via a satellite. In that case, the coordinates we have are those of the physical LAN port of Wi-Fi router and they are imprecise.

• Data doctoring.

To estimate the number of incorrect pings in our dataset, we notice that the expected fraction of pings where both longitude and latitude have 6 decimal places should be 81% (90% of 90%). The difference between the observed fraction and the expected fraction is due to the presence of incorrect data points.

Science Park I

Estimated percentage of incorrect data points is 81% - 66% = 15%

Science Park II

Estimated percentage of incorrect data points is 81% - 59% = 22%

One North

Estimated percentage of incorrect data points is 81% - 61% = 20%

Now we will sort all pings into several bands:

• band 2 - pings where latitude OR longitude has only 2 decimal places

• band 3 - pings where latitude OR longitude has only 3 decimal places

• band 4 - pings where latitude OR longitude has only 4 decimal places

• band 5 - pings where latitude OR longitude has only 5 decimal places

• band 6 - pings where both latitude AND longitude have 6 decimal places

If we believe that errors that we have are just rounding errors, then we can remove pings with at least one coordinate given with not enough decimal places.

Science Park I

Science Park II

One North

Devices

Need a plan!

1. Count unique points where each device pinged. The definition of a unique point will be with different resolution, e.g., if as a unique point we count pairs (lon, lat), where both coordinates are rounded to the 4th decimal place, then we are essentially counting the number of different squares \(11\times 11\) m that this device has been to.

2. Stationary devices are devices that pinged from very few \(11\times 11\) m squares.

3. Devices that pinged always from one \(11\times 11\) cm square are probably geolocated based on IP rather than based on satellite.

## `summarise()` has grouped output by 'did', 'lon'. You can override using the
## `.groups` argument.

## `summarise()` has grouped output by 'did', 'lon'. You can override using the
## `.groups` argument.

Science Park

One North