The Dataverse can be accessed via an API, which has a user-contributed wrapper written in R. We'll take a look at these and see if we can get them to work.
library(devtools)
## Warning: package 'devtools' was built under R version 2.15.2
# install_github(repo = 'dvn', username = 'leeper')
library(dvn)
There is a demonstration dataverse available here: https://dvn-demo.iq.harvard.edu/dvn/
Note: You must use https for DVN API calls.
The first step is to set our options to use this dataverse:
options(dvn = "https://dvn-demo.iq.harvard.edu/dvn/")
options(dvn.user = "foo")
options(dvn.pwd = "bar")
It's worth noting that this method of authentication is not secure. Plain-text passwords should never be used!!! If there is an OAuth-style authentication method, or a key based method, that would be far preferable.
Let's see if we can search this dataverse using the API. First, we try it on the website and try to search.
We can see that there are two results here. Let's reproduce this with the API.
dvSearch("fudan")
## Error: subscript out of bounds
This didn't work. Let's try this with the browser option on and see what happens.
dvSearch("fudan", browser = TRUE)
It pulls from the following location:
https://dvn-demo.iq.harvard.edu/dvn/api/metadataSearch/fudan
Which contains the following:
<MetadataSearchResults>
<searchQuery>fudan</searchQuery>
<searchHits></searchHits>
</MetadataSearchResults>
I guess the API entry point is different for the search box itself?
Also worth noting is that while using this package, there is a lot of this error in displaying the documentation:
Error in fetch(key) : internal error -3 in R_decompress1
I do not know why this, but it goes away if I restart R.
Let's try and create a study instead. First we use dvBuildMedata to make the Study's metadata. We also need to open a dvServiceDoc, which I guess will authenticate me somehow.
I created a user/pass and a dataverse on the website.
md <- dvBuildMetadata(title = "Foo Study", creator = "Storer, Alex", creator = "Durbin, Phil",
publisher = "IQSS", date = "2013-10-13", description = "A foo study", subject = "Foo",
subject = "Trash", subject = "Test")
dsd <- dvServiceDoc()
This will let us create a new study, in theory. We pass the results from the dvServiceDoc and dvBuildMetadata commands.
dvCreateStudy(dsd, md)
## [1] ""
This prints the following to the screen, and does not create a study:
> dvCreateStudy(dsd,md)
* About to connect() to dvn-demo.iq.harvard.edu port 443 (#0)
* Trying 140.247.115.100... * connected
* Connected to dvn-demo.iq.harvard.edu (140.247.115.100) port 443 (#0)
* SSL connection using EDH-RSA-DES-CBC3-SHA
* Server certificate:
* subject: C=US; ST=California; L=Santa Clara; O=Oracle Corporation; OU=GlassFish; CN=dvn-1.hmdc.harvard.edu
* start date: 2012-04-05 18:22:31 GMT
* expire date: 2022-04-03 18:22:31 GMT
* issuer: C=US; ST=California; L=Santa Clara; O=Oracle Corporation; OU=GlassFish; CN=dvn-1.hmdc.harvard.edu
* SSL certificate verify result: self signed certificate (18), continuing anyway.
> POST /dvn/api/data-deposit/v1/swordv2/collection/dataverse/<service xmlns="http://www.w3.org/2007/app" xmlns:atom="http://www.w3.org/2005/Atom"><workspace><atom:title type="text">Demo</atom:title><collection href="https://dvn-demo.iq.harvard.edu/dvn/api/data-deposit/v1/swordv2/collection/dataverse/baz"><atom:title type="text">Baz</atom:title><collectionPolicy xmlns="http://purl.org/net/sword/terms/">Demo deposit terms of use: HTTP/1.1
Host: dvn-demo.iq.harvard.edu
Accept: */*
Content-Type: application/atom+xml
Content-Length: 549
< HTTP/1.1 505 HTTP Version Not Supported
< Date: Wed, 09 Oct 2013 19:16:42 GMT
< Connection: close
Maybe the problem is that I need the name of the dataverse, and not just the entire dsd XML? Let's try that. Instead of parsing the XML using an R library and returning a data frame with the results (which would be a nice dvn feature!!) I'm going to be a lazy bum.
snippet <- dvCreateStudy("baz", md)
This prints the following to the screen, and does create a study.
* About to connect() to dvn-demo.iq.harvard.edu port 443 (#0)
* Trying 140.247.115.100... * connected
* Connected to dvn-demo.iq.harvard.edu (140.247.115.100) port 443 (#0)
* SSL connection using EDH-RSA-DES-CBC3-SHA
* Server certificate:
* subject: C=US; ST=California; L=Santa Clara; O=Oracle Corporation; OU=GlassFish; CN=dvn-1.hmdc.harvard.edu
* start date: 2012-04-05 18:22:31 GMT
* expire date: 2022-04-03 18:22:31 GMT
* issuer: C=US; ST=California; L=Santa Clara; O=Oracle Corporation; OU=GlassFish; CN=dvn-1.hmdc.harvard.edu
* SSL certificate verify result: self signed certificate (18), continuing anyway.
> POST /dvn/api/data-deposit/v1/swordv2/collection/dataverse/baz HTTP/1.1
Host: dvn-demo.iq.harvard.edu
Accept: */*
Content-Type: application/atom+xml
Content-Length: 549
< HTTP/1.1 401 Unauthorized
< X-Powered-By: Servlet/3.0 JSP/2.2 (GlassFish Server Open Source Edition 3.1.2 Java/Sun Microsystems Inc./1.6)
< Server: GlassFish Server Open Source Edition 3.1.2
< WWW-Authenticate: Basic realm="SWORD2"
< Content-Length: 0
< Date: Wed, 09 Oct 2013 20:43:37 GMT
<
* Connection #0 to host dvn-demo.iq.harvard.edu left intact
* Issue another request to this URL: 'https://dvn-demo.iq.harvard.edu/dvn/api/data-deposit/v1/swordv2/collection/dataverse/baz'
* Re-using existing connection! (#0) with host dvn-demo.iq.harvard.edu
* Connected to dvn-demo.iq.harvard.edu (140.247.115.100) port 443 (#0)
* Server auth using Basic with user 'foo'
> POST /dvn/api/data-deposit/v1/swordv2/collection/dataverse/baz HTTP/1.1
Authorization: Basic Zm9vOmJhcg==
Host: dvn-demo.iq.harvard.edu
Accept: */*
Content-Type: application/atom+xml
Content-Length: 549
< HTTP/1.1 201 Created
< X-Powered-By: Servlet/3.0 JSP/2.2 (GlassFish Server Open Source Edition 3.1.2 Java/Sun Microsystems Inc./1.6)
< Server: GlassFish Server Open Source Edition 3.1.2
< Location: https://dvn-demo.iq.harvard.edu/dvn/api/data-deposit/v1/swordv2/edit/study/doi:10.5072/FK2/OCIXR
< Last-Modified: Wed, 09 Oct 2013 16:43:39 -0400
< Content-MD5: 745c86e15fbb4bbb12147f43e648e019
< Content-Type: application/atom+xml;type=entry;charset=UTF-8
< Transfer-Encoding: chunked
< Date: Wed, 09 Oct 2013 20:43:39 GMT
<
* Connection #0 to host dvn-demo.iq.harvard.edu left intact
It is worth noting that this study is not in the Baz dataverse, as I tried to suggest, but has no dataverse associated with it, at least online. The study, however, thinks it is associated with the Baz dataverse.
Furthermore, the only way to add files is by adding a zip file. From an R standpoint, this is not particularly useful, as you'd probably have an open data set or something like that, right? Anyway, I guess I can find a zip file to test this with.
The fact that the searching functions don't return a clear string that I can use in the adding functions is quite frustrating…
Here are the studies for baz:
dvUserStudies("baz")
## [1] "<feed xmlns=\"http://www.w3.org/2005/Atom\"><title type=\"text\">Baz</title><dataverseHasBeenReleased xmlns=\"http://purl.org/net/sword/terms/state\">false</dataverseHasBeenReleased><generator uri=\"http://www.swordapp.org/\" version=\"2.0\"/><entry xml:base=\"https://dvn-demo.iq.harvard.edu/dvn/api/data-deposit/v1/swordv2/edit/study/doi:10.5072/FK2/MA2JD\"><id>https://dvn-demo.iq.harvard.edu/dvn/api/data-deposit/v1/swordv2/edit/study/doi:10.5072/FK2/MA2JD</id><title type=\"text\">Foo Study</title><link href=\"https://dvn-demo.iq.harvard.edu/dvn/api/data-deposit/v1/swordv2/edit-media/study/doi:10.5072/FK2/MA2JD\" rel=\"edit-media\"/></entry><entry xml:base=\"https://dvn-demo.iq.harvard.edu/dvn/api/data-deposit/v1/swordv2/edit/study/doi:10.5072/FK2/OCIXR\"><id>https://dvn-demo.iq.harvard.edu/dvn/api/data-deposit/v1/swordv2/edit/study/doi:10.5072/FK2/OCIXR</id><title type=\"text\">Foo Study</title><link href=\"https://dvn-demo.iq.harvard.edu/dvn/api/data-deposit/v1/swordv2/edit-media/study/doi:10.5072/FK2/OCIXR\" rel=\"edit-media\"/></entry></feed>"
Which of those is the objectid, I wonder, for adding a file? Maybe doi:10.5072/FK2/OCIXR?
study = "doi:10.5072/FK2/OCIXR"
fname = "/Users/astorer/Desktop/rintro-cf13.zip"
dvAddFile(study, fname)
## [1] "<entry xmlns=\"http://www.w3.org/2005/Atom\"><bibliographicCitation xmlns=\"http://purl.org/dc/terms/\">Storer, Alex; Durbin, Phil, 2013-10-13, \"Foo Study\", http://dx.doi.org/10.5072/FK2/OCIXR V1 [Version]</bibliographicCitation><generator uri=\"http://www.swordapp.org/\" version=\"2.0\"/><id>https://dvn-demo.iq.harvard.edu/dvn/api/data-deposit/v1/swordv2/edit/study/doi:10.5072/FK2/OCIXR</id><link href=\"https://dvn-demo.iq.harvard.edu/dvn/api/data-deposit/v1/swordv2/edit/study/doi:10.5072/FK2/OCIXR\" rel=\"edit\"/><link href=\"https://dvn-demo.iq.harvard.edu/dvn/api/data-deposit/v1/swordv2/edit/study/doi:10.5072/FK2/OCIXR\" rel=\"http://purl.org/net/sword/terms/add\"/><link href=\"https://dvn-demo.iq.harvard.edu/dvn/api/data-deposit/v1/swordv2/edit-media/study/doi:10.5072/FK2/OCIXR\" rel=\"edit-media\"/><link href=\"https://dvn-demo.iq.harvard.edu/dvn/api/data-deposit/v1/swordv2/statement/study/doi:10.5072/FK2/OCIXR\" rel=\"http://purl.org/net/sword/terms/statement\" type=\"application/atom+xml; type=feed\"/><treatment xmlns=\"http://purl.org/net/sword/terms/\">no treatment information available</treatment><link href=\"http://dx.doi.org/10.5072/FK2/OCIXR\" rel=\"alternate\"/></entry>"
Yep, that worked! It unzipped the file into its constituents and looks reasonable.