Project 4: NoSQL Migration

Assignment

The task assigned was to take information from a relational database and migrate it to a NoSQL database of our choosing. I chose to migrate the ‘flights’ database we loaded to MySQL in Week 0. For the NoSQL database, I selected MongoDB. I downloaded and installed MongoDB from their website: https://www.mongodb.com/download-center?jmp=nav#community

Overview of Approach

I started with an internet search on SQL-to-MongoDB migration strategies. I was curious whether the migration should be done manually, and if so, what tranformations were needed and how to designate the keys for the key-value pairs.

In the course of my research, I came across the web site for a Ruby gem called ‘mongify’ (http://mongify.com/) which “is a data translator system for moving your SQL data to MongoDB”. I decided I would try to execute my migration using mongify.

First, I had to download and install Ruby for Windows: http://rubyinstaller.org/ Next, I downloaded the mongify gem. This proved to be a very time-consuming endeavor because the mongify Ruby gem requires several other Ruby gems, which in turn require other Ruby gems, etc. But eventually it was installed and ready to use. Especially critical were the ‘mysql’ and ‘mysql2’ gems needed for Ruby to communicate with MySQL.

Executing the Migration

The first step in the mongify process is to create a database.config file designating the Ruby gem to use as the adapter as well as the MySQL host, username, password and database. The file also specifies the name of the MongoDB host and database:

database.config

Once the database.config file is created, the database translation file can be generated using the mongify translation database.config > translation.rb command. This takes the MySQL database specified (‘flights’) and converts it to a MongoDB database file called ‘translation.rb’ which can be used to generate a MongoDB:

Finally, to convert the data into a MongoDB database, I ran the mongify process database.config translation.rb command:

View Data in MongoDB

Once the migration process was complete, I opened the MongoDB CLI and viewed the databases. Then I set the active database to ‘flights’ and viewed the collections included in the migrated databases:

Collections in ‘flights’ database

Lastly, I viewed the first 20 records of each of the collections using the find command:

airlines

airports

flights

planes

weather