Research question

Can a deep learning model learn to identify roof types based on street-level images?

Design

I found sample locations using City of Boston parcel records, which specify roof type for residential housing. I selected 1600 homes with flat roofs and 1600 with gabled (pointed) roofs, then downloaded the Google Street View images for each location using the Google Street View API. I randomly reserved 200 of each category to test the model’s accuracy.

Input images

Here’s what the flat-roof images looked like:

…

And here’s what the gabled-roof images looked like:

The model

The model is a convolutional neural network (“CNN” or “ConvNet”) built by modifying VGG16, a deep learning model that was trained to classify a wide range of image types (e.g. dogs, birds, flowers).

Results

The model’s predictions in the test dataset were highly accurate (92% correct, area under the curve = 0.97). The model was usually quite confident in its predictions (close to 0 or 1); many of its mistakes occurred when it lacked confidence.

Here are some test images it classified correctly:

…

And some it misclassified:

Comments

Identifying roof types based on the GSV images is more difficult than it sounded to me originally. In some images, roofs aren’t visible; in others, multiple buildings appear. While the model makes some mistakes a human wouldn’t, it gets some very difficult classifications right.

A caveat: most of the flat-roofed houses in the Boston dataset are in South Boston, and I didn’t attempt to match the sample locations based on characteristics such as size, age, etc. Thus, it’s possible the model has learned to pick up on additional cues that are not generally related to roof type.

Although the classification task is simple, it is not entirely trivial. Roof type might determine a building’s resilience towards different extreme weather events. In tropical regions, roof type may be associated with health risks such as mosquito-borne diseases. While cities like Boston usually have accurate data on building construction, other cities (including many that have been imaged by GSV) do not; a model like this one could help generate a fast, free, reasonably accurate property survey.