Open data plataforms

Service plataforms enables customers and users to download the data via a huge number of open standards (CSV, JSON, etc.) so there is no risk of lock-in at the data level.

• Socrata is a software-as-a-service platform that provides a cloud-based solution for open data publishing and visualization

• CKAN is an open source project, developed by the Open Knowledge Foundation, that lets users provision open data catalogs and, in some cases, visualizations and APIs.

• Succesuful cases: Colombia open data porta, Mexico open data portal, Paraguay Open Data portal, etc.

Basic functions

• get_socrata_metada: fetch a detailed metada of open datasets (extract json and basic list of datasets available from a Socrata domain

• search_data: Shows a list object with with available datasets by keywords/tags. The list contains four things:

• eda_opendata: Generate a basic report in rmarkdown or shiny with selected datasets from an open data portal.

Future: a basic recommendation system.

get_socrata_metada

Description fetch a detailed metada of open datasets (extract json and basic list of datasets available from a Socrata domain

Usage

get_socrata_metada(url)

Arguments

• url - A Socrata URL. This simply points to the site root.

Details a R data frame containing a listing of datasets along with detailed metadata. Next field are preserved for every open dataset:

id, name, attribution, averageRating, category, createdAt, description, displayType, downloadCount, hideFromCatalog, hideFromDataJson, indexUpdatedAt, licenseId, newBackend, numberOfComments, oid, provenance, publicationAppendEnabled, publicationDate, publicationGroup, publicationStage, rowsUpdatedAt, rowsUpdatedBy, tableId, totalTimesRated, viewCount, viewLastModified, viewType, columns, grants, license, metadata, owner, query, rights, tableAuthor, tags, flags, attributionLink, blobFilename, blobFileSize, blobId, blobMimeType, rowClass, previewImageId, childViews, displayFormat, iconUrl, rowIdentifierColumnId, ratings, disabledFeatureFlags, accessLevel, landingPage, issued, @type, modified, keyword, contactPoint, publisher, identifier, title, distribution, theme

Value The function returns a dataframe with detailed information about datasets.

search_data

Description Shows a list object with with available datasets by keywords/tags. The list contains four things:

• tabular datasets: a dataframe with four columns (dataset name, attribution (public agency), number of rows and columns size of dataset (Mb) and elapsed time for R export.
• geo: a dataframe with general information about the geo file.
• href: a dataframe with general information about the href file.
• blobby: a dataframe with general information about the href file.

Usage

search_data(metadata, tags)

Arguments

• metadata - a dataframe get from get_metadata function
  
• tags a character vector with keywords and/or tags required for searching in dataset name, tag field or public agency field

Details The selected sample is drawn according to a selection-rejection (list-sequential) algorithm

Value The function returns a list of 4 dataframes. The 4 dataframes contain information of tabular, geo, href and blobby data of open data portal.

eda_opendata

Description Generate a basic report in rmarkdown or shiny with selected datasets from an open data portal.

Usage

eda_opendata(metadata, ids, tags)

Arguments

• metadata - a dataframe get from get_metadata function
• ids - a character vector with datasets id required. if ids and tags arguments are NULL all datasets are analysed
  
• tags - a character vector with tags/keywords required for analysis. if ids and tags arguments are NULL all datasets are analysed.

Details The selected sample is drawn according to a selection-rejection (list-sequential) algorithm

Value The function returns a list of 4 dataframes. The 4 dataframes contain information of tabular, geo, href and blobby data of open data portal.

Type data

• Most datasets to (2017-03-04) are of tabular type.
• A frecuency analysis of data type in colombia open is show below:

Open Data in Colombia is new!

• Every day is added new open data to “datos abiertos” portal:

Main topics

• Some common topics in Colombia open data are:

View Count Distribution

• Distribution of number of views of Colombian datasets:

Summary view counts

• A descriptive statistics summary of view counts is presented below:

Most frequently viewed datasets:

Most downloaded datasets