## and converting the json data to a dataset.

#### install any packages that might be necessary

if(!require(jsonlite)) install.packages("jsonlite")
## Loading required package: jsonlite

#### Assumes the api-key is in a csv called ‘NYT_API_KEY’ in the same directory

mykey <- read.csv("NYT_API_KEY.csv")
api_key <- mykey$api_key #### Get the URL’s of the NYT Community API NYT_API_Base_URL <- "http://api.nytimes.com/svc/community/v3/user-content/" NYT_API_Recent_comments <- paste0(NYT_API_Base_URL, "recent.json?api-key=", api_key) # get comments on a specific date say on 11-8-2008 (president Obama becoming # president) NYT_API_Comments_by_date <- paste0(NYT_API_Base_URL, "by-date.json?api-key=", api_key, "&date=2008-11-08") # comments by user ID a random number say 2364811 NYT_API_Comments_by_UserID <- paste0(NYT_API_Base_URL, "user.json?api-key=", api_key, "&userID=2364811") # url on website for Data Scientists to prove their skills and make money (2011) # (birth of kaggle perhaps) comm_url <- "http://bits.blogs.nytimes.com/2011/11/03/a-site-for-data-scientists-to-prove-their-skills-and-make-money/" NYT_API_Comments_by_Url <- paste0(NYT_API_Base_URL, "url.json?api-key=", api_key, "&url=", comm_url) #### Convert the json data from these Uri’s to a dataset # to get the data we actually want, after experimenting we have to drill down a # bit and extract the column that holds the data Recent_Comments <- fromJSON(NYT_API_Recent_comments) rec_comm_df <- Recent_Comments$results$comments rec_comm_df ## list() # turns out that there were no recent comments... Comments_by_Date <- fromJSON(NYT_API_Comments_by_date) comm_date_df <- Comments_by_Date$results$comments comm_date_df ## commentID parentID statusID lft rgt commentSequence ## 1 1130431 NA 2 20 21 10 ## 2 381590 NA 2 380 381 188 ## 3 381626 NA 2 94 95 45 ## 4 381635 NA 2 2 3 1 ## 5 381638 NA 2 10 11 5 ## 6 381652 NA 2 78 79 37 ## 7 381663 NA 2 4 5 2 ## 8 381669 NA 2 382 383 189 ## 9 381671 NA 2 96 97 46 ## 10 381676 NA 2 98 99 47 ## 11 381698 NA 2 6 7 3 ## 12 381701 NA 2 8 9 4 ## 13 381704 NA 2 100 101 48 ## 14 381711 NA 2 10 11 5 ## 15 381718 NA 2 12 13 6 ## 16 381721 NA 2 12 13 6 ## 17 381722 NA 2 16 17 7 ## 18 536965 NA 2 116 117 43 ## 19 539436 NA 2 8 9 2 ## 20 537689 NA 2 200 201 77 ## 21 529855 NA 2 8 9 2 ## 22 3088221 NA 2 14 15 6 ## 23 529856 NA 2 10 11 3 ## 24 3070344 NA 2 10 11 5 ## 25 538765 NA 2 8 9 1 ## commentBody ## 1 The professional and readers' reviews for this musical have been over-the-top bad. I can only imagine some kind of gang mentality has set in, to inspire such a disproportionately cruel attack on a good show. There are many delightful moments, genuine laughs and exhilarating music. Crazy love is celebrated in all its mad glory. This is a light and fizzy comedy with heart. The actors give stellar performances, with Patina Renea Miller a particularly bright light in the constellation. ## 2 Yes, condoms should be free for all, but only cold heartless people would cut back on researching a cure for AIDS. There may not be a cure for the common cold, but people don’t die from colds.<br /><br />Yes, voluntary family planning, is good. But blaming all our problems on over-population is just plain wrong. There are not too many people on this planet. There are too many cars, trucks, airplanes, polluting power-plants and cows.<br /><br />The most fascist country on earth, enforced a one child per family rule, and their population still grew from 1 billion, to 1.2 billion. If becoming as fascist as China, isn’t going to stop climate change, will you kill billions of people, just so you can keep driving your cars?<br /><br />It is all very well, to say we need to eat less beef, or promote vegetarianism. But as I said earlier, eating less beef has always led to larger herds. A strictly enforced herd reduction program, can only work, if it subsidizes the slaughter of bulls and breeding cows. Farmers will never just bury their animals, because beef consumption has gone down. Beef consumption needs to increase, in the short term, for it to go down in the long term.<br /><br />Now would also be a good time, to try to increase our trust of government. If we trusted government, we might accept surveillance cameras. That would make us safer, and eventually we wouldn’t need as many police.<br /><br />But to gain that trust, the government would have to pass better laws. It should be illegal for government employees (especially politicians) to lie. Cannabis (Marijuana is a derogatory word, and fabricated concept) should be fully legal and only lightly taxed. And most importantly, there should be effective civilian oversight, of all police, and police training institutions.<br /><br /> ## 3 If there's one truism in our energy future, its that one technology won't do the job. The government is not good at choosing "the best" and it's worse when it comes to orchestrating a suite. Moreover, technologies and how they fit together will change over time.<br /><br />What the government can be good at is setting goals. It also can be good at enforcing the goal. What's nimble and capable of changing direction quickly is a good industrial manager looking for a profit. Put them together and you have a recipe of relatively rapid and successful change.<br /><br />I'm with Krupp on this one, after a 30 long period resisting the idea. Carbon cap and trade (including a provision for individuals to get into the trading game if they have invested in measurable carbon saving practices and equipment) which at once puts a value on what is not produced and provides a way for the value to be used productively.<br /><br />For too long we've covered up the true cost of our industry and development. The public has subsidized the profits of those who orchestrate the use of natural resources. By valuing those resources and, in effect, charging for them at their true market value, the innovators and, as importantly, the investors willing to take the risks, who move away from conventional energy sources, will reap benefits and the capital with which to continue searching for even more efficient and productive ways in which to sustain our industry. Let profit be the driving force.<br /><br />What drives the resistance to converting to reusable or regenerative energy sources are two: 1) Why give up a good deal as long as it can be sustained/protected, and 2) the cost of conversion. The solution to the former is to add a measure of the true cost of doing business as usual. The solution to the latter is to use revenue from the former to drive more realistic cost/benefit calculations. Further, by adopting cap along with trade, managers who want to convert to regenerative energy sources, can make the switch when the cost of not doing so becomes insupportable, or after they've fully prepared for the switch.<br /><br />I don't subscribe to an "Apollo Project" model as even a way to start. The Apollo model requires that we define the solution from the beginning. In reality, we can't do that. The solution we think we see now will undoubtedly metamorphose over time and the government is notorious for being unable to allow unsuccessful projects to be shelved. No one on Capitol Hill has a good enough crystal ball to know now what will eventually work, what will be the best mix of complementary technologies. Industry, on the other hand, will continually adjust as the impact on its bottom line reveals itself.<br /><br />Cap and trade levels the field, provides revenue for successful ventures to continue research as well as take products to market. I agree with others that seed money for research probably does have to come from government sources. However, if it does, the tax paying public should reap some reward. Government subsidy should be looked on as an investment to be repaid or rewarded by profit participation whether that is returned to the treasury or is returned to a revolving research fund.<br /><br />Apollo is not a good model even though it's sure to appeal to politicians with visions of glory in their heads. ## 4 Young boys make the best uses of sticks. When deprived of violent toys by their mothers, they turn them into guns, swords and knives. Even when they've had virtually no exposure to guns swords and knives.<br /><br />The simple stick I use to turn the compost is always finding its way across the yard at the hand of my 2 year old boy. He wields it like Friar Tuck might.<br /><br />The bush which gave its life and from where I harvested the stick didn't know that its purpose would be to serve Dad's need to sustain and son's need to play, but then they never do. ## 5 I'm horrified to see support for tar sands and can only hope that the Obama team has taken a cold hard look at it. On the other hand, there are still 72 days left for Bush et al. to do their darndest, which can be pretty awful.<br /><br />re tar sands:<br /><a href="http://www.newyorker.com/reporting/2007/11/12/071112fa_fact_kolbert" target="_blank">http://www.newyorker.com...</a><br /><br />re awfulness:<br />This about rules being put in place to make it hard for new administration to make progress:<br /><a href="http://greeninc.blogs.nytimes.com/2008/10/31/bushs-last-minute-rule-making-has-big-environmental-implications/" target="_blank">http://greeninc.blogs.nytimes.com...</a><br /><br />Wonderful Krugman again today, suggest you real the whole thing!<br /><a href="http://www.nytimes.com/2008/11/07/opinion/07krugman.html" target="_blank">http://www.nytimes.com...</a><br />"But it would be fair for the new administration to point out how conservative ideology, the belief that greed is always good, helped create this crisis. What F.D.R. said in his second inaugural address — “We have always known that heedless self-interest was bad morals; we know now that it is bad economics” — has never rung truer.<br /><br />And right now happens to be one of those times when the converse is also true, and good morals are good economics. Helping the neediest in a time of crisis, through expanded health and unemployment benefits, is the morally right thing to do; it’s also a far more effective form of economic stimulus than cutting the capital gains tax."<br /> ## 6 While editing a book, Opposing Viewpoints: Energy Alternatives (Greenhaven Press, 2006), I read many articles. I read that during Jimmy Carter's presidency in the late 1970s, the U.S. faced the disabling Arab oil embargo. President Carter and his advisors were deeply concerned about this country's dependence on the volatile Mideast for energy resources. Carter recognized the value of federal funding for renewable energy resources and was slated to speak on July 5, 1979, about this country's desperate need to reduce our dependence on oil (<a href="http://www.americanrhetoric.com/speeches/jimmycarterundeliveredenergyspeech.htm)." target="_blank">http://www.americanrhetoric.com/speeches/jimmycarterundeliveredenergyspeech.htm).</a> This speech ended with a quote and a plea. “Franklin Roosevelt once said: ‘Lives of nations are determined, not by the count of years, but by the lifetime of the human spirit. The life of a man is three-score years and ten, a little more, a little less. But the life of a nation is the fullness of its will to live.’ As a people, as a nation, let us meet the challenge of the energy crisis with the fullness of our will to live.”<br /><br />Pres. Carter never gave that speech, but instead a less strong one about energy resources: Energy and the National Goals - A Crisis of Confidence (<a href="http://www.americanrhetoric.com/speeches/jimmycartercrisisofconfidence.htm)." target="_blank">http://www.americanrhetoric.com/speeches/jimmycartercrisisofconfidence.htm).</a> The Republicans did their worst to smear Carter as weak and unable to handle difficult national topics, and he was defeated by Ronald Reagan. After Reagan became president in 1980, he removed the solar panels from the White House roof. Like Reagan, everyone in govt. forgot the energy crisis, and soon after the auto makers brought out gas guzzling SUVs. Auto makers have produced huge numbers of trucks and SUVs with low fuel efficiency even while losing money. Now let's see, who is suffering? Unemployed auto workers, the gullible public who have added tons of CO2 to the atmosphere... Here’s a question: How many barrels of oil has this country used since 1980? How many would we have used with high-fuel efficiency vehicles? Then ask yourself and the Bush administration, “Who made a profit from this?”<br /><br />Finally this past year, the effects of global climate change became so obvious that most people couldn't keep up their denial. And somehow the government released the report they’ve hidden for five years recognizing that humans could cause global climate. But the greedy ones will continue to blame others while seeking to destroy the environment—and our children’s and grandchildren’s futures--to make heaps of money. Perhaps they’re building rockets to go to Mars.<br /><br />I’m incredibly grateful that the American people took their Constitutional rights seriously and voted out some of the party and senators and representatives who lied while dipping their hands into our pockets. It’s important for us all to ensure that lies are not tolerated, no matter how many times they are repeated.<br /><br /> ## 7 May I add the humble rock (stone, pebble)?<br /><br />Collecting them is communicable; I inherited from my mother, and have infected many others.<br /><br />In addition, over time I've discovered a few practical uses:<br />sharpening stones (smoother ones with water as lubricant)<br />Paperweight<br />Ice crusher<br />Trail marker<br />Tent/tar weight<br /><br />One will often find them arranged at the beach, in the woods, just for fun. ## 8 \tI propose a labeling program for the environmental impact of all sorts of consumer goods and services that we buy.<br />\tAs you know, there are more threats to Earth's biosphere than global warming. For example, there are depletion of ocean fisheries, killing of coral reefs, polluting the oceans, destruction of tropical forest, etc. They all have the effect of seriously reducing biodiversity, which may well lead to a poorer, more sterile world.<br />\tIt seems that the biggest driver of these threats is wasteful consumption. And I think many people would be willing to be more cautious about what they buy and consume if they knew the true environmental costs.<br />\tTherefore, we could use some kind of environmental impact labeling program, so that people could easily make wise choices of what they buy. It seems to me that all kinds of things could be labeled, from food to furniture to manufactured goods of all sorts, as well as fuels and even travel and transportation.<br />\tWe already have uniform labeling for nutrition and for energy efficiency of appliances, and that seems to be useful. Why not extend the practice to more general environmental impact?<br /><br />\tOf course, it would also be good if our new president encouraged us to consume wisely to protect the Earth.<br /><br /> ## 9 I sent this letter yesterday to the Obama transition team ;<br /><br />Dear President-elect Obama ;<br /><br />Our nation faces some serious challenges in the next 4 years.<br /><br />I would like to mention one subject which is seldom openly talked about, but<br /> a subject which could have enormous impact on our country’s ENERGY FUTURE ( and “national security ) ;that is the subject of the “Unacknowledged Special Access Projects”, or “USAP’s”, which are the highly secret “black budget” programs into which billions of (off the official books ) dollars are poured each year. I refer specifically to those “black” projects which fund, and HAVE funded for many years now , research into the nature, make up, power and propulsion methods of…wait for it …UFO’s.<br /><br />This is a subject for which there is a veritable MOUNTAIN of highly credible first hand witness testimonyMUCH of it from highly placed corporate, military and law enforcement leaders , many at “command” , “executive” and/or ‘CEO” level. (Example; the 400 witnesses gathered, on videotape, by Dr Steven Greer’s “Disclosure Project”)<br /><br />It is my informed belief, that buried within these black budget projects, is the closely held secret of very advanced energy production technology; technology which, if brought to light and put to use, could radically change the USA’s, (and the world’s for that matter) “energy problem” which, as you realize, has been a major issue in this years campaign. I , and millions of others who are knowledgeable on this subject, believe NOW is the time to have these “secrets”, (as long as our national security is not jeopardized), to be brought into the open, studied OPENLY, and used.<br /><br />Many of these “USAP’s” have operated for years in a highly illegal way, completely free from congressional or executive oversight, They have, in effect, constituted a secret ”shadow government” which has operated through many presidential administrations, since the founding of the CIA after WW II. If you doubt me on any of this…please ask former President Clinton, who was briefed on this subject, or Mr Laurence Rockefeller, or former CIA chief James Woolsey, among many others.<br /><br />(Many names ARE available. There are also many in congress who are very aware of this subject…some support disclosure….and some…would like to prevent it for various , mostly political, reasons. I encourage you, as a former Senator yourself , with access to these people , to “ask around”. )<br /><br />The illegal extra-governmental activities of the people involved in these “USAP’s” should be brought to light, and the individuals directly involved must be given some sort of immunity from prosecution in exchange for their cooperation…this would help to speed the pace of disclosure.<br /><br />We are in the 21st century…I believe the American people are prepared to hear the truth about this subject, especially if there is a huge dividend in highly advanced energy technology to be gained from disclosure.<br /><br />I would like to ask that you make the “discovery” of this technology, and the disclosure of the long-held secret of so-called “UFO’s” a TOP priority as part of your overall energy plan.<br /><br />Thank you for your time, and may God bless America (and your new administration !)<br /><br />Bob Welch<br /><br />Ex-Fleetwood Mac/<br />Capitol Records solo artist<br /><br />( Please see 2 addendums, enclosed )<br /><br /> ## 10 We need to go smart, but that doesn't mean we can't go BIG. This financial crisis has presented an opportunity for important federal investments in green energy and technology.<br /><br />Why not propose an "energy cost stabilization program", that will keep energy prices within an acceptable but moderately high range? All revenues could be split between investments in domestic clean energy, and an "energy buffer fund" to provide relief in the event of disruptive spikes in energy costs. ## 11 World's first hoe, pick and shovel... ## 12 All hail the stick! The magic wand is the kid's mind.<br /><br />May decades ago, back when my kids were so young-and-all, they had (over my protests) manufactured toys, including a Barbie for my little girl and a GI Joe for my little boy. They grew, and these plastic denizens of the world of play were set aside; but before that, the kids increasingly returned to their immense collection of stuffed animals, inventing entire worlds and histories for them to act out. The last use of the Barbie and GI Joe was not for my eyes. I heard the kind of laughter one associates with barracks or locker rooms: peeking in, I saw Barbie and GI Joe in flagrante. Kids grow, but imagination keeps growing too. ## 13 uhh, jonathan (#15)? "There aren't going to be any revolutions or breakthroughs".<br /><br />maybe you could kindly get out of the way? ## 14 Sticks were OK in my childhood though I mostly favored forked sticks to which I attached strips of innertube and a leather pouch and terrorized rabbits, squirrels, and birds. Never could hit anything with those slingshots other than my knuckles from time to time. Ouch!<br /><br />I was more of a rock kid. I still have some arrowheads - somewhere - found in years long ago as well as some interesting fossils.<br /><br />As an adult, I'm still not much on sticks but I have about 30-40 tons of flat-rock limestone in my landscaping that are loaded with fossils. Some say I have rocks in my head but plainly, they're in my yard. ## 15 I once wrote an entire blog on sticks. I have pictures of a few sticks on my latest blog. And stones.<br /><br />The simplest things in life just happen to be some of the most necessary, and the most often taken for granted. ## 16 Someone needs to collect their <a href="http://www.350.org/">election success prize</a>. ## 17 Well OK -- but I would be careful with <a href="http://www.cleanupge.org/video/Bessette3.mov">anything that's been in the Hudson.</a> ## 18 This excruciating self indulgent mess extolled by Ms. Dargis as beyond the best film of the year if nothing else confirms this: formal film criticism is a dead duck.<br /><br />The readers’ comments both pro and con were spot-on and for me provided a soothing catharsis having just endured the relentless narcissistic tedium of Synecdoche, New York.<br /><br />Mr. Kaufman go away on a long, long vacation and just stay there alright? Thank you. Oh yeah and leave your laptop at home!<br /><br /> ## 19 The kind of light hearted-ness that will keep you on the edge of your seat. Full of chuckles rather than full blown laughter, however there are a few parts that the entire audience will enjoy. Perfect to crack the serious person in everyone...<br /><br /> ## 20 What I loved about this movie was that we had to figure things out. If anyone has seen the other movies by Spike Lee, you know that you cannot fall asleep.<br /><br />Miracle of St. Anna inspired me to research the 92nd Infantry Regiment. I want to know about veterans of the war. I see them differently. There has always been a them and us mentality that has been inside of me. I wonder more about the soldiers. African Americans could not be leaders of anyone but African Americans because who else respected them during these times.<br /><br />The movie was filled with issues of racism, division, love, war, and lots of lost.<br /><br />Lee obviously believes that we should be left with questions in the end of his movies.<br /><br />In the end, the scene of Angelo and Hector meeting again was beachy. It was also ironic being filmed at a beach. Lee pokes fun at the scene by filming it on a beach.So I am grateful that he doesn't take our minds and thoughts for granted, but initiates conversation and keeping my thirst for knowledge active. ## 21 Bold and colourful. Rich scenery. Funny....lmao. Lots of adult humour. Cheesy and dumb...ultimately fun. Oh, and the little kids there (not mine-mine are in their 20's) liked it too. Granted I got an odd look 'cause I know all the words to Move it, move it...Mr. Dargis? I think Moto Moto was right on....would you have prefered Peppi Le Pew approach? I bet Marc and Justin not only know the character, but have already used him personally. I suggest you watch colour blind...<br /><br />P.S. saw it on IMAX screen, maybe it was TOO big, so I guess I am going to have to see it again on a regular theater screen, just to be sure of it. ## 22 I've been visiting the Island annually since 1974. The Inverurie hotel was our first accomodation, since closed as a hotel and is now a condominium complex, I think. Also stayed at the Castle Harbour I(now known as The Pointe). Lived through the ups-and-downs of The Mariott Southhampton. While decisions were being made by management, Wyndam took over its operation. Fabian forced us out of there and we came upon The Pompano Beach Club, a nice, quiet resort with 50+ guest rooms in all ranges. In fact, we've already booked our annual Christmas vacation. Enjoy your Bermuda vacation! ## 23 I didn't see the first one, but I needed a laugh and went to see Escape 2 Africa - by myself &amp; no kids in tow. I'm 50 years old and laughed harder and more times than all the kids around me, who seemed to be getting bored at some of the slower parts. But I found it to be fun. ## 24 I have to agree with "Anonymous" on this. I was just at Pappagallo last week and was extremely disappointed. My friend and I, women in our 50's, were well dressed, spoke italian with the waiters, and were, I felt, treated like country bumpkins. The wait staff was aloof and supercilious with us, I thought. Food was mediocre and overpriced...and the service was slow and not just with us. At the table next to us a very elegant older couple waited 15 minutes before the water and the wine were served. The gentlemen finally exclaimed "Senta!" to flag down the waiter.<br />Our primary waiter (there were 3) dropped something on the shelf separating the 2 dining areas and almost hit me on the head. There were no verbal apologies but we were appeased with an extra glass of wine for both of us.<br />The menu had some interesting things on it, I'd say. Not necessarily "tourist fare". Maybe I guess we ordered the wrong things? But I can say that my friends green lasagne, a house speciality, was surprisingly dry, as if she had ordered grilled lasagne!<br />We stayed long enough to have ordered a second bottle of wine to drown out our dismal dinner but the waiter never bothered to ask us if we wanted one. A 17 euro antipasto of culatello (yes, it is an expensive ham) was listed as being served with a parmesan mousse and tomato chutney...about a tablespoon of each showed up on the plate. The fried items were definitely not what they should have been. For 125 euro the dinner was a bust. Fortunately,we found a hole in the wall in the University Quarter that made up for the Pappagallo fiasco. The food and service there was great. We ate there 4 times! ## 25 torture and torquemada might be synonyms but to reduce it to a hilarious level where you find it rather so dull that you ckuckle when a pendulum blade slices a body apart is the triumph which can be duly acclaimed by this franchise.<br /><br />the drama is restricted to a cat mouse game where we know the dead and go into a retrospect to find how they got to the mortuary ,<br /><br />well does it matter as they say get on with the autopsy and the inquest and leave the audience alone .<br /><br />the biggest surprise is the acting is actually good which demonstrates they took it seriously too.<br /><br />the rest of the set was made from the leftovers of the 4 prequels and the franchise is just preaching now about corrupt governments and cops and the immorality of the judiciary and the legal system being effete as if that is going to shock anyone in this age of the wicked and lying .the only bshock is watching this movie to the end with nothing to tell anyone . ## commentTitle createDate updateDate ## 1 Don't Believe the Anti-Hype 1226121700 1226168034 ## 2 1226089635 1226126002 ## 3 1226092591 1226126002 ## 4 1226093658 1226126002 ## 5 1226093929 1226126002 ## 6 1226094412 1226126002 ## 7 1226095135 1226126002 ## 8 1226095839 1226126002 ## 9 1226095999 1226126002 ## 10 1226096205 1226126002 ## 11 1226098247 1226126002 ## 12 1226098412 1226126002 ## 13 1226099082 1226126002 ## 14 1226101857 1226126002 ## 15 <NA> 1226111198 1226126002 ## 16 1226115257 1226126002 ## 17 1226116052 1226126002 ## 18 Brilliant! Not the film!! The readers' comments! 1226124365 1226168538 ## 19 Lightly Sprinkled 1226125185 1226168538 ## 20 It's not that easy 1226126143 1226168538 ## 21 Go for the colours and stay for the laughs.... 1226128072 1226168538 ## 22 1226132498 1226167666 ## 23 Enjoyed it! 1226137147 1226168538 ## 24 1226139248 1226167847 ## 25 the seance of the righteous torquemada 1226144481 1226168538 ## approveDate userID userDisplayName userTitle userURL ## 1 1226121700 57327516 <NA> <NA> ## 2 1226122402 57316519 Donald Renne <NA> <NA> ## 3 1226122402 32467 ryarnell <NA> <NA> ## 4 1226122402 35502097 Michael May <NA> <NA> ## 5 1226122402 57076816 Susan Anderson <NA> <NA> ## 6 1226122402 55988219 Barbara <NA> <NA> ## 7 1226122402 57076816 Susan Anderson <NA> <NA> ## 8 1226122402 38035315 John F. Schivell <NA> <NA> ## 9 1226122402 55995857 Bob Welch <NA> <NA> ## 10 1226122402 29142684 Mark E <NA> <NA> ## 11 1226122402 57084036 David B. Benson <NA> <NA> ## 12 1226122402 34046114 David Ocampo G. <NA> <NA> ## 13 1226122402 54501617 chuck <NA> <NA> ## 14 1226122402 57291432 H.R. <NA> <NA> ## 15 1226122402 57326821 Loraine ## 16 1226122402 34088373 mjc <NA> <NA> ## 17 1226122402 34088373 mjc <NA> <NA> ## 18 1226124365 16947225 johnbaylin <NA> <NA> ## 19 1226125185 54632532 orcakidf <NA> <NA> ## 20 1226126143 54249115 lifeblessing <NA> <NA> ## 21 1226128072 57327799 lamo421 <NA> <NA> ## 22 1226132498 57170036 <NA> <NA> ## 23 1226137147 36471646 young1345 <NA> <NA> ## 24 1226139248 49337072 <NA> <NA> ## 25 1226144481 52812272 jbz7879 ## userLocation editorsSelection ## 1 0 ## 2 Vancouver Canada 0 ## 3 Beavercreek, OR 0 ## 4 Chicago 0 ## 5 Boston 0 ## 6 Boston 0 ## 7 Boston 0 ## 8 Princeton, NJ 0 ## 9 Nasille TN 0 ## 10 Los Angeles, CA 0 ## 11 Pullman 0 ## 12 Chicago IL 0 ## 13 berkeley CA 0 ## 14 Planet Earth & not afraid to admit it 0 ## 15 Utah 0 ## 16 wrong side of the river 0 ## 17 wrong side of the river 0 ## 18 0 ## 19 0 ## 20 0 ## 21 0 ## 22 0 ## 23 0 ## 24 0 ## 25 havering,london,england 0 ## recommendationCount replyCount assetID ## 1 0 0 255681 ## 2 1 0 311636 ## 3 0 0 309578 ## 4 4 0 317677 ## 5 3 0 316510 ## 6 1 0 308947 ## 7 2 0 317677 ## 8 7 0 311636 ## 9 0 0 309578 ## 10 0 0 309578 ## 11 2 0 317677 ## 12 2 0 317677 ## 13 0 0 309578 ## 14 4 0 317677 ## 15 2 0 317677 ## 16 0 0 316510 ## 17 0 0 317677 ## 18 0 0 118471 ## 19 0 0 315731 ## 20 1 0 157553 ## 21 0 0 40173 ## 22 0 0 277 ## 23 0 0 40173 ## 24 0 0 7073 ## 25 0 0 224095 ## assetURL ## 1 http://www.nytimes.com/show/21843/Romantic-Poetry/overview ## 2 http://dotearth.blogs.nytimes.com/2008/11/05/the-president-and-the-planet/ ## 3 http://dotearth.blogs.nytimes.com/2008/11/04/what-would-an-energy-moon-shot-look-like/ ## 4 http://dotearth.blogs.nytimes.com/2008/11/07/new-inductee-at-toy-hall-of-fame-the-stick/ ## 5 http://dotearth.blogs.nytimes.com/2008/11/07/in-case-you-missed-it/ ## 6 http://dotearth.blogs.nytimes.com/2008/11/04/the-presidency-and-the-climate-challenge/ ## 7 http://dotearth.blogs.nytimes.com/2008/11/07/new-inductee-at-toy-hall-of-fame-the-stick/ ## 8 http://dotearth.blogs.nytimes.com/2008/11/05/the-president-and-the-planet/ ## 9 http://dotearth.blogs.nytimes.com/2008/11/04/what-would-an-energy-moon-shot-look-like/ ## 10 http://dotearth.blogs.nytimes.com/2008/11/04/what-would-an-energy-moon-shot-look-like/ ## 11 http://dotearth.blogs.nytimes.com/2008/11/07/new-inductee-at-toy-hall-of-fame-the-stick/ ## 12 http://dotearth.blogs.nytimes.com/2008/11/07/new-inductee-at-toy-hall-of-fame-the-stick/ ## 13 http://dotearth.blogs.nytimes.com/2008/11/04/what-would-an-energy-moon-shot-look-like/ ## 14 http://dotearth.blogs.nytimes.com/2008/11/07/new-inductee-at-toy-hall-of-fame-the-stick/ ## 15 http://dotearth.blogs.nytimes.com/2008/11/07/new-inductee-at-toy-hall-of-fame-the-stick/ ## 16 http://dotearth.blogs.nytimes.com/2008/11/07/in-case-you-missed-it/ ## 17 http://dotearth.blogs.nytimes.com/2008/11/07/new-inductee-at-toy-hall-of-fame-the-stick/ ## 18 http://movies.nytimes.com/movie/385312/Synecdoche-New-York/overview ## 19 http://movies.nytimes.com/movie/387815/Role-Models/overview ## 20 http://movies.nytimes.com/movie/450565/Miracle-at-St-Anna/overview ## 21 http://movies.nytimes.com/movie/335737/Madagascar-Escape-2-Africa/overview ## 22 http://www.nytimes.com/travel/guides/caribbean-and-bermuda/bermuda/hotels.html ## 23 http://movies.nytimes.com/movie/335737/Madagascar-Escape-2-Africa/overview ## 24 http://www.nytimes.com/travel/guides/europe/italy/bologna/41852/ristorante-al-pappagallo/restaurant-detail.html ## 25 http://movies.nytimes.com/movie/453137/Saw-V/overview ## status commentType replies ## 1 approved comment NULL ## 2 approved comment NULL ## 3 approved comment NULL ## 4 approved comment NULL ## 5 approved comment NULL ## 6 approved comment NULL ## 7 approved comment NULL ## 8 approved comment NULL ## 9 approved comment NULL ## 10 approved comment NULL ## 11 approved comment NULL ## 12 approved comment NULL ## 13 approved comment NULL ## 14 approved comment NULL ## 15 approved comment NULL ## 16 approved comment NULL ## 17 approved comment NULL ## 18 approved comment NULL ## 19 approved comment NULL ## 20 approved comment NULL ## 21 approved comment NULL ## 22 approved userSuggestion NULL ## 23 approved comment NULL ## 24 approved userReview NULL ## 25 approved comment NULL Comments_by_UserID <- fromJSON(NYT_API_Comments_by_UserID) comm_user_df <- Comments_by_UserID$results$comments comm_user_df ## commentID status commentSequence userID userSubmittedDisplayName ## 1 4570862 approved 10 2364811 Jim ## 2 1129038 approved 20 2364811 ## userSubmittedTitle userSubmittedURL userSubmittedLocation ## 1 NA NA NYC ## 2 NA NA ## commentTitle ## 1 hardcoded comment title ## 2 ok ## commentBody createDate updateDate ## 1 time to Google for "cell phone signal blocker"... 1316723787 1316728099 ## 2 great 1211305399 1211305399 ## approveDate recommendations replyCount replies editorsSelection parentID ## 1 1316728099 11 0 NULL FALSE NA ## 2 1211305399 0 0 NULL FALSE NA ## commentType ## 1 comment ## 2 comment ## asset.assetURL ## 1 http://cityroom.blogs.nytimes.com/2011/09/22/cell-phone-service-on-some-subway-platforms-starts-next-week/ ## 2 <NA> ## asset.vendorID asset.source asset.assetID asset.createDate ## 1 P12Rq7kT2AvJyomlFgcylQ== url 3267087 1316722025 ## 2 <NA> <NA> NA <NA> ## asset.updateDate ## 1 1319743568 ## 2 <NA> ## asset.taxonomies ## 1 70457, 125749, 199832, blogs, cityroom, Cellphone Service Coming to the Subway ## 2 NULL ## asset.taxonomy asset.labels ## 1 blogs/cityroom/Cellphone Service Coming to the Subway NYT_Blogs, 54906 ## 2 <NA> NULL ## asset.text asset.properties ## 1 NULL 1, 199832, 2, 199832, 3, 199832 ## 2 NULL NULL ## asset.all-properties ## 1 no-group, comment-list-sort, auto-moderation, reached-max-com, comment-style, Not in a group, Sort order to use when displaying comment listings, auto-moderation on or off, If we reached the max number of COMS per parent taxonomy, To use inline comments or overflow page, 2, sort by approve date, oldest first, 3, sort by approve date, newest first, 4, sort by number of recommendations, 5, sort by editors' selections, 6, sort by replies, 7, auto-moderation on, 8, auto-moderation off, 9, reached the max number of COMS, 10, did not reach the max number of COMS, 11, articles having inline comments or overflow page, 12, comments on overflow page ## 2 no-group, comment-list-sort, auto-moderation, reached-max-com, comment-style, Not in a group, Sort order to use when displaying comment listings, auto-moderation on or off, If we reached the max number of COMS per parent taxonomy, To use inline comments or overflow page, 2, sort by approve date, oldest first, 3, sort by approve date, newest first, 4, sort by number of recommendations, 5, sort by editors' selections, 6, sort by replies, 7, auto-moderation on, 8, auto-moderation off, 9, reached the max number of COMS, 10, did not reach the max number of COMS, 11, articles having inline comments or overflow page, 12, comments on overflow page ## asset.assetTitle labels permid userDisplayName ## 1 Cellphone Service Coming to the Subway NULL 10 NA ## 2 <NA> NULL 20 NA ## userTitle userURL userLocation ## 1 NA NA NA ## 2 NA NA NA ## picURL ## 1 https://s3.amazonaws.com/pimage.timespeople.nytimes.com/2364/811/cropped-2364811.jpg?0.8428952772024576 ## 2 https://s3.amazonaws.com/pimage.timespeople.nytimes.com/2364/811/cropped-2364811.jpg?0.8428952772024576 ## timespeople sharing ## 1 1 0 ## 2 1 0 Comments_by_Url <- fromJSON(NYT_API_Comments_by_Url) comm_Url_df <- Comments_by_Url$results$comments comm_Url_df ## commentID status commentSequence userID ## 1 4980043 approved 4 65293063 ## 2 4836825 approved 3 57456855 ## 3 4826359 approved 2 40220672 ## 4 4809969 approved 1 60361520 ## userDisplayName userLocation userTitle ## 1 banana slug Santa Cruz, CA NA ## 2 Barbara Duck - The Medical Quack Huntington Beach, California NA ## 3 RAPIER CITY OF ANGELS NA ## 4 Edward D. Weinberger Manhattan NA ## userURL commentTitle ## 1 NA n/a ## 2 NA hardcoded comment title ## 3 NA hardcoded comment title ## 4 NA hardcoded comment title ## commentBody ## 1 I disagree with your highly optimistic outlook of this site Mr. Rapier. I would be amazed to find any contributing evidence that would suggest a contest such as the Heritage Health Prize, where a bank offers$5000 so that they can more effectively deny loans to medical patients that need this financial support the most, is beneficial to all parties involved. People will die without proper medical funding just so that a "data scientist" from Canada can buy a new tv?
## 2 We have pros and cons here and if you are just looking for number crunching then that is not bad;however you will have mismatched data submitted with someone trying to make a buck to associate mismatched data and unfortunately when it is taken down to and individual level you have issues, discrimination by the algorithm.<br/><br/>When it comes to the Heritage program, the site looks like a gaming site, not intelligent healthcare research, and I wrote about it exactly that way.  That particular project is not the works of scientists, but rather a lot of gamers mixed into that know nothing about healthcare.  We also have the old adults that believe there's a magic formula number out there to predict re-admittance to the hospital.  Developer are just there for a buck and the crunching of the numbers may come up with some non targeted information but again when you go beyond the group level and use it for decision making processes at the individual level, well even Netflix's algorithm is only 60%...what a waste in my opinion.  Koogle is just way out of line for this project as well as the gamers working on it.  Former HHS Leavitt though is pretty funny promoting it as he's not what I would call digitally literate and has been sucked in here to believing a magic formula based on numbers only is possible.  <br/><br/><a href="http://ducknetweb.blogspot.com/2011/08/so-whats-going-on-over-at-heritage.html" title="http://ducknetweb.blogspot.com/2011/08/so-whats-going-on-over-at-heritage.html" target="_blank">http://ducknetweb.blogspot.com/2011/08/so-whats-going-on-over-at-heritag...</a><br/><br/>If you don't believe me, listen to my Twitter bud NYU professor Siefe tell you more about it, how numbers don't lie but people do and the marketing spins that are attached today.  Math is no longer 100% proof when it comes to proving accuracy today.  <br/><br/><a href="http://ducknetweb.blogspot.com/2011/01/numbers-dont-lie-but-people-do.html" title="http://ducknetweb.blogspot.com/2011/01/numbers-dont-lie-but-people-do.html" target="_blank">http://ducknetweb.blogspot.com/2011/01/numbers-dont-lie-but-people-do.html</a><br/><br/>This is a big part of the reason you have all the Occupy movements going on as numbers and algorithms discriminate and deny.  You can't touch them, see them or talk to them, they just run on servers 24/7 making decisions and throwing the data up on the screen.
## 3                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                Mr. Weinberger, your comment is valid and such a model is pervasive because of the mindset of the people currently in control of this planet.  However, from the other side of the coin, it could morph and be the beginning of an infrastructure for joint communal effort where everyone can win equitably according to the level of their effort and contribution.
## 4                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Yet another profession succumbs to the "starving actor" economic model.  Just like the expected hourly rate of the average actor wanna be in New York is about 8 cents an hour, the expected earnings of the people entering these contests is depressed by the massive numbers, including the armies of well educated people in India and China, of contestants.  Only a few people can win, so a lot of work will be in vain.
##   createDate updateDate approveDate recommendations replyCount replies
## 1 1322764264 1322767938  1322764338               0          0    NULL
## 2 1320707258 1320715649  1320712049               0          0    NULL
## 3 1320525409 1320525618  1320525618               0          0    NULL
## 4 1320306739 1320306916  1320306916               1          0    NULL
##   editorsSelection parentID parentUserDisplayName depth commentType
## 1            FALSE       NA                    NA     1     comment
## 2            FALSE       NA                    NA     1     comment
## 3            FALSE       NA                    NA     1     comment
## 4            FALSE       NA                    NA     1     comment
##   trusted recommendedFlag reportAbuseFlag permID
## 1       0              NA              NA      4
## 2       0              NA              NA      3
## 3       0              NA              NA      2
## 4       0              NA              NA      1
##                                                                                                      picURL
## 1                                            https://graphics8.nytimes.com/images/apps/timespeople/none.png
## 2 https://s3.amazonaws.com/pimage.timespeople.nytimes.com/5745/6855/cropped-57456855.jpg?0.5102049501869466
## 3                                            https://graphics8.nytimes.com/images/apps/timespeople/none.png
## 4                                            https://graphics8.nytimes.com/images/apps/timespeople/none.png
##   timespeople sharing
## 1           0       0
## 2           1       0
## 3           1       0
## 4           1       1