Lab9
Junsheng
30 March 2017
Markdown
tenders <- fread("tabula-tender-bids-from-mar-2012-to-jan-2017.csv", header = F, fill = T)
colnames(tenders) <- c("centre", "stall", "area", "trade", "bid", "month") #table does not have headers
tenders <- tenders[centre != "",] #remove empty rows
tenders <- tenders[!1135,] #remove 'lockup' row
tenders <- tenders[!1749,] #remove 'market' row
tenders[1:1134,type:="cooked",] #set type to cooked for rows before 1135
tenders[1135:1748,type:="lockup",] #set type to cooked for rows between 1135 and 1738
tenders[1749:nrow(tenders),type:="market",] #vice versa
tenders[,bidNum:=as.numeric(gsub(bid, pattern="\\$|,", replacement = "")),] #convert formatted price to numeric
tenders[,date:=as.Date(paste0("01-",month),"%d-%b-%Y"),] # same for date
tenders[,priceM2:=bidNum/as.numeric(area),] # price per m2
head(tenders) #everything ok?## centre stall area trade bid month
## 1: AMOY STREET FOOD CENTRE 1-Nov 5.65 COOKED FOOD $3,701.11 Oct-15
## 2: AMOY STREET FOOD CENTRE Jan-48 5.65 COOKED FOOD $4,048.00 Mar-16
## 3: AMOY STREET FOOD CENTRE Jan-48 5.65 COOKED FOOD $3,200.00 Jun-16
## 4: AMOY STREET FOOD CENTRE Jan-50 5.65 COOKED FOOD $2,728.00 Jun-16
## 5: AMOY STREET FOOD CENTRE Jan-60 5.65 COOKED FOOD $3,188.00 Jun-16
## 6: AMOY STREET FOOD CENTRE Jan-68 5.65 HALAL COOKED FOOD $1,800.00 Jun-13
## type bidNum date priceM2
## 1: cooked 3701.11 0015-10-01 655.0637
## 2: cooked 4048.00 0016-03-01 716.4602
## 3: cooked 3200.00 0016-06-01 566.3717
## 4: cooked 2728.00 0016-06-01 482.8319
## 5: cooked 3188.00 0016-06-01 564.2478
## 6: cooked 1800.00 0013-06-01 318.5841tenders[,list(price=mean(priceM2)),by = centre] # avg price per centre## centre price
## 1: AMOY STREET FOOD CENTRE 357.68944
## 2: BEO CRESCENT MARKET 26.64691
## 3: BERSEH FOOD CENTRE 90.11038
## 4: BLK 665 BUFFALO ROAD 217.51288
## 5: BLK 1 JALAN KUKOH 41.65605
## ---
## 110: BLK 50A MARINE TERRACE 335.56357
## 111: BLK 511 BEDOK NORTH STREET 3 16.66667
## 112: BLK 630 BEDOK RESERVOIR ROAD NA
## 113: BLK 724 ANG MO KIO AVENUE 10.52632
## 114: BLK 82 TELOK BLANGAH DRIVE NAtenders[type=="cooked",list(price=mean(priceM2)),by=centre] #avg price per centre for only cooked food## centre price
## 1: AMOY STREET FOOD CENTRE 357.68944
## 2: BEO CRESCENT MARKET 167.28756
## 3: BERSEH FOOD CENTRE 90.11038
## 4: BLK 665 BUFFALO ROAD 284.37981
## 5: BLK 1 JALAN KUKOH 41.65605
## 6: BLK 11 TELOK BLANGAH CRESCENT 235.44803
## 7: BLK 112 JALAN BUKIT MERAH 104.61073
## 8: BLK 115 BUKIT MERAH VIEW 135.12743
## 9: BLK 117 ALJUNIED AVENUE 2 234.98865
## 10: BLK 120 BUKIT MERAH LANE 1 177.64957
## 11: BLK 127 TOA PAYOH LORONG 1 95.18510
## 12: BLK 137 TAMPINES STREET 11 222.83814
## 13: BLK 14 HAIG ROAD 410.02825
## 14: BLK 159 MEI CHIN ROAD 78.00092
## 15: BLK 16 BEDOK SOUTH ROAD 136.16792
## 16: BLK 16 BEDOK SOUTH ROAD S(460016) 132.83100
## 17: BLK 162 ANG MO KIO AVENUE 4 180.99548
## 18: BLK 17 UPPER BOON KENG ROAD 194.90581
## 19: BLK 20 GHIM MOH ROAD 513.84615
## 20: BLK 216 BEDOK NORTH STREET 1 174.55859
## 21: BLK 22 TOA PAYOH LORONG 7 131.35614
## 22: BLK 226D ANG MO KIO AVENUE 1 57.25534
## 23: BLK 226H ANG MO KIO STREET 22 183.22581
## 24: BLK 22A HAVELOCK ROAD 430.37634
## 25: BLK 22B HAVELOCK ROAD 47.00941
## 26: BLK 254 JURONG EAST STREET 24 246.08449
## 27: BLK 29 BENDEMEER ROAD 462.96959
## 28: BLK 3 CHANGI VILLAGE ROAD 322.31801
## 29: BLK 32 NEW MARKET ROAD 401.87481
## 30: BLK 320 SHUNFU ROAD 57.54103
## 31: BLK 335 SMITH STREET 110.74070
## 32: BLK 335SMITH STREET 167.01754
## 33: BLK 36 TELOK BLANGAH RISE 81.17021
## 34: BLK 37A TEBAN GARDENS ROAD 232.65864
## 35: BLK 38A BEO CRESCENT S(169982) 115.10791
## 36: BLK 41A CAMBRIDGE ROAD 41.17647
## 37: BLK 44 HOLLAND DRIVE 159.79962
## 38: BLK 453A ANG MO KIO AVENUE 10 296.33929
## 39: BLK 49 SIMS PLACE 211.67353
## 40: BLK 4A JALAN BATU 99.00368
## 41: BLK 503 WEST COAST DRIVE 345.78877
## 42: BLK 51 OLD AIRPORT ROAD 281.37586
## 43: BLK 527 ANG MO KIO AVENUE 10 269.13978
## 44: BLK 531A UPPER CROSS STREET 145.96395
## 45: BLK 6 JALAN BUKIT MERAH 522.91550
## 46: BLK 6 TANJONG PAGAR PLAZA 188.89338
## 47: BLK 628 ANG MO KIO AVENUE 4 355.98991
## 48: BLK 7 EMPRESS ROAD 201.58730
## 49: BLK 726 CLEMENTI WEST STREET 2 492.29040
## 50: BLK 75 TOA PAYOH LORONG 5 173.61111
## 51: BLK 79 CIRCUIT ROAD 161.33099
## 52: BLK 79 TELOK BLANGAH DRIVE 306.36792
## 53: BLK 79A CIRCUIT ROAD 210.80532
## 54: BLK 80 CIRCUIT ROAD 32.41667
## 55: BLK 85 REDHILL LANE 409.15114
## 56: BLK 89 CIRCUIT ROAD 72.00328
## 57: BLK 90 WHAMPOA DRIVE 58.52843
## 58: BLK 91 WHAMPOA DRIVE 141.73077
## 59: BLK 93 TOA PAYOH LORONG 4 113.60317
## 60: BUKIT TIMAH MARKET 235.85086
## 61: COMMONWEALTH CRESCENT MARKET 67.41825
## 62: DUNMAN FOOD CENTRE 72.00269
## 63: EAST COAST LAGOON FOOD VILLAGE 278.05815
## 64: GEYLANG SERAI MARKET 160.89246
## 65: GOLDEN MILE FOOD CENTRE 142.53171
## 66: KALLANG ESTATE MARKET 73.33333
## 67: MARKET STREET FOOD CENTRE 297.63294
## 68: MAXWELL FOOD CENTRE 305.69219
## 69: NEWTON FOOD CENTRE 280.92754
## 70: NORTH BRIDGE ROAD MARKET 75.28231
## 71: SERANGOON GARDEN MARKET 164.66147
## 72: TAMAN JURONG MARKET & FOOD CENTRE 71.19260
## 73: TIONG BAHRU MARKET 200.58236
## 74: ZION RIVERSIDE FOOD CENTRE 196.54989
## centre pricetenders[,list(count=length(stall)),by=centre] #number of bids per centre## centre count
## 1: AMOY STREET FOOD CENTRE 62
## 2: BEO CRESCENT MARKET 22
## 3: BERSEH FOOD CENTRE 103
## 4: BLK 665 BUFFALO ROAD 50
## 5: BLK 1 JALAN KUKOH 3
## ---
## 110: BLK 50A MARINE TERRACE 8
## 111: BLK 511 BEDOK NORTH STREET 3 1
## 112: BLK 630 BEDOK RESERVOIR ROAD 19
## 113: BLK 724 ANG MO KIO AVENUE 1
## 114: BLK 82 TELOK BLANGAH DRIVE 17Number of bids over time
Generally the number of bids have been increasing from 2012 to 2016. There seems to be some consistent spikes in number of bids in the mid to 3rd quarter of the each year such as May 12,Oct 14,May 16 and Aug 16 which I suspect might be the time which tenders expire and need renewal.
ggplot(tenders[,list(count=length(stall)),by=date], aes(date, count)) + geom_line() + ggtitle("Number of bids over time") + theme(axis.text.x = element_text(angle = 90)) + scale_x_date(date_breaks = "4 month", date_labels = "%d %b %y")
Change in average price per m2 over time
The average price per m2 has been increasing over the years slowly. There is a spike of the prices in Jun 13 which is perhaps the reason why there is less of a spike in the number of bids for year of 2013.
ggplot(tenders[,list(price=mean(priceM2)),by = date], aes(date, price)) + geom_line() + ggtitle("Change in average price per m2 over time") + theme(axis.text.x = element_text(angle = 90)) + scale_x_date(date_breaks = "4 month", date_labels = "%d %b %y")
Bids for each hawker centre
The bar chart of bids for each hawker centre across years allow me to estimate the size of the hawker centres. The greater the number of bids for a hawker centre, the bigger the hawker centre. In the graph, Blk 335 Smith Street has the greatest number of bids which is not that surprising as there is where the Chinatown complex lies with lots of stalls. Some other hawker centres with significant number of bids are Berseh Food Centre, Blk 117 Aljunied Avenue 2, Commonwealth Cresent market, Golden Mile Food Complex and Taman Jurong Market & Food Centre.
ggplot(tenders[,list(count = length(stall)),by = centre], aes(centre, count)) + geom_bar(stat='identity') + theme(axis.text.x = element_text(angle=90,vjust = 1, size = 8)) + ggtitle("Bids for each hawker centre")
Reading in Hawker Center KML from data.gov.sg
# Reading in KML file of all hawker centres from data.gov.sg
h <- readOGR("HAWKERCENTRE.kml","HAWKERCENTRE")## OGR data source with driver: KML
## Source: "HAWKERCENTRE.kml", layer: "HAWKERCENTRE"
## with 116 features
## It has 2 fieldsplot(h) + title("Hawker centres in Singapore")
## numeric(0)# performing join of the data from HAWKERCENTRE.kml and tender by name and centre respectively
h.t <- data.frame(toupper(h$Name),h@coords[,1:2])
colnames(h.t) <- c("name","lon","lat") # column header names in tender.sp
tenders.sp <- merge(tenders,h.t,by.x="centre",by.y="name",all.x=T)Solving the issues of poor matching of name and centre by performing a geocoding with Google’s services
Quick check of results
ggplot(tenders.sp, aes(x=lon,y=lat,size=priceM2,color=type)) + geom_point(alpha = 0.3) + coord_fixed()
Density plots of hawker centres in Singapore
## density plot
ggplot(tenders.sp, aes(x=lon,y=lat)) + geom_point() + geom_density2d() + coord_fixed() + ggtitle("2D density plot of bids for each hawker centres in Singapore")
ggplot(tenders.sp, aes(x=lon,y=lat)) + geom_point() + geom_hex() + coord_fixed() + ggtitle("Hex plot of bids for each hawker centre in Singapore") + scale_fill_gradient(low = "#9ecae1", high = "#08519c" ,name = "Count",na.value=NA)
number of bids per trade
Looking at the all the trades, it seems that most of them are clustered around the centre and south of Singapore like Cooked Food, Fresh Seafood and many others. There are some trades that are quite uniform like Beancakes & Noodles, Halal Frozen Good and many others. A few are quite dispersed like Beef, Eggs, Halal Mutton and others.
ggplot(tenders.sp, aes(x=lon,y=lat)) + geom_point() + geom_density2d() + coord_fixed() + ggtitle("2D density plot of number of bids per trade in Singapore") + facet_wrap(~trade,scales = "fixed") + labs(x = "Longitude", y = "Latitude")
number of bids per type
Cooked bids seems to be clustered around the center and south with a few dispersed to the edge of Singapore. For lockup, they are the most tightly clustered in the center and south of Singapore. Market bids do have a number of clusters around Singapore but its distribution is the most uniform compared to the rest of the types.
ggplot(tenders.sp, aes(x=lon,y=lat, colour = type)) + geom_point() + geom_density2d() + coord_fixed() + ggtitle("2D density plot of bids for each type in Singapore") + facet_wrap(~type,scales = "fixed") + labs(x = "Longitude", y = "Latitude")
number of bids over the years
It can be observed that the hawker centres have been expanding outwards from the center of Singapore while continue to be most clustered in center and south of Singapore.
tenders_year.sp <- tenders.sp[,year:=paste("20",substr(date, 3, 4))]
ggplot(tenders_year.sp, aes(x=lon,y=lat, colour = year)) + geom_point() + geom_density2d() + coord_fixed() + ggtitle("2D density plot of bids over the years in Singapore") + facet_wrap(~year,scales = "fixed") + labs(x = "Longitude", y = "Latitude")
Analysis of spatial point patterns
centres.sp <- tenders.sp[lat>0,list(lon=lon[1],lat=lat[1],price=mean(priceM2),count=.N),by = centre] # creating a table with one row for each centre
centres.sp[is.na(price),price:=0,] #NA to 0
coordinates(centres.sp) <- c('lon','lat') #specify coordinates as lon and lat for x and y
centres.ppp <- unmark(as.ppp(centres.sp))
plot(centres.ppp) # in a bounded box
sg <- readOGR(".","sg-all")## OGR data source with driver: ESRI Shapefile
## Source: ".", layer: "sg-all"
## with 1 features
## It has 13 fieldssg.window <- as.owin(sg)
centres.ppp <- centres.ppp[sg.window]
plot(centres.ppp) # with outline of Singapore
Studying the clustering of centres
plot(Kest(centres.ppp)) # studying the clustering of centres by plotting Ripley's reduced second moment function K(r)
Density plots of centres
plot(density(centres.ppp,0.01))
contour(density(centres.ppp,0.01))
Investigating the relationship between cluttering of hawker centres and the underlying population in that area
pop <- as.im(readGDAL("sg-pop.tif")) # read in data in sg-pop.tif as a spatial object## sg-pop.tif has GDAL driver GTiff
## and has 37 rows and 58 columnsplot(rhohat(centres.ppp,pop))
plot(rhohat(centres.ppp,pop,weights = centres.sp$price))
The dip at the upper end of the population density might be due to the lack of space to build hawker centres when the place is heavily clustered with residential estates. In addition, there seems to be a trend of concentrating food places in giantic mall like hubs.
plot(pop)
plot(centres.ppp,add=T, col="White")
Looking at the scatter plot of hawker centers on a heatmap of population, there 2 places that are very dense in population which are Choa Chu Kang and Sengkang areas. It might be because these towns are relatively new where the government has been building many residential estates to meet the growing demand of housings while not populating the place with enough hawker centres yet.