PADM6001

Data In Organizations

This is a class about research method for public administrators. Much of the class has focused on collecting novel data using surveys and interviews, but one of the important lessons I am hoping to demonstrate is that there is lots of data all around us. There are some questions that require a survey to answer, but others might be addressed by looking harder at the bits and numbers floating around us at any moment. And just as importantly, I’d like to demonstrate how being able to collect and manage data, combined with a research mentality can benefit you personally, professionally. Research skills aren’t just useful to write papers, but can have applications throughout life.

There’s a really famous example of how data, and the use of data, can transform an organization. It’s from baseball, which everyone in the class may not know anything about, but you don’t need to know much about the background to follow what happened. It centers on a book called Moneyball that Michael Lewis wrote about Billy Beane and the Oakland A’s. The A’s happened to be (and are) my favorite team (I grew up in Northern California), but that’s not why I’m telling you about it. The A’s faced long odds to succeed, because they did not have the money to outspend their rivals.

 

 

They made a movie about it too, and the tailor will give you a bit more of the back story.

 

 

What they used instead of money was data. They were an early mover, one of the very first teams to see how data could be used to identify skills in players that were under appreciated in the marketplace. While other teams could outspend the A’s, the A”s could spend smarter. They identified skills, such as getting on base, as being cheaper than other skills like driving in runs. And the data showed that some of these undervalued skills were a better predictor of which teams would win games. So they bucked the conventional wisdom, and got a lot of laughter for it, and invested in the undervalued skills they identified. And what happened? They won a lot of games. They didn’t win the World Series (sadly for me, and them) but they way over performed relative to the money they spent on players. Between 2000 and 2006, the A’s won the second most games in all of baseball (second to the Yankees), despite spending a fraction as much. Since then other teams have copied the A’s methods of analysis and have caught up.

Every organization can learn from what the Oakland A’s did. Moneyball has become a shorthand term for using data to optimize performance, and if you Google “Moneyball for…” you’ll get lots of suggestions. Government needs to play Moneyball, so do businesses, schools, hospitals, everyone. The term Moneyball has become a shorthand for using data to get better performance.

In order to play Moneyball though, a few things are required. You have to have data. Sports are great for researchers. We know the outcome people care about (wins and losses) and we count a lot that happens during the game. We count hits and runs of course, but now we track the location of pitches and the speed players run and where they’re standing at every moment. That provides fine grained data about everything one could imagine using to predict which team wins, and which one loses. The A’s just used this data better than everyone else.

That raises an important point about using data in any organization. You need to have an accurate measure of the outcomes you care about (wins, revenue, grades, etc.) and you need to measure the inputs that might affect them (spending, hours spent studying). Without that data, you’ll never know how different activities or strategies impact performance. If you don’t measure it, you’ll probably think everything is okay. That’s the way we’ve always done it, can’t be improved. That’s how every organization in government thought for a long time, but things are changing.

 

 

You might think that doesn’t help you though. You might feel like you’re in the same boat is Milgram at the start, within an organization or department that doesn’t count anything. Unless you’ve thought about it, you probably can’t fully appreciate all of the data generated by your organization, or that might just be waiting to get counted. Milgram had arrest rates, factors that predicted recidivism, crime locations, and so much more just waiting to be used for criminal justice to play Moneyball. What does your organization have going on that is just waiting to be turned into useful data? Every sale, delivery, partnership, program, event, everything can be counted and turned into information that may help increase productivity.

I want to be clear though, that idea can be taken too far. If I work as a manger in a call center, and I want to insure my employees are working hard, the easiest thing to count is how many calls they complete. Great, job done right? Wrong, counting just the number of calls they complete, that incentives them to just get off the phone as quickly as possible. Now, rather than improving performance, I’ve created a system that makes them serve customers worse. Counting for the sake of counting can be damaging, and we have to ensure that our measures are an operationalization of the outcomes we care about. Test scores are useful, but they aren’t the same thing as students learning. Arrests are useful, but they aren’t the same thing as public safety. There’s a wonderful book written by Cathy O’Neil called Weapons of Math Destruction that highlights a lot of these challenges in the growth of data

 

 

The Need for Experiments

Beyond data, you need a plan to use it. You could collect data about the millimeters of movement every employee at your organization does, but that will be useless unless its analyzed and utilized effectively. That’s where experiments in learning organizations come in.

If you keep doing what you’ve always done, you’ll keep getting the same results. But how do we know that change will improve whatever we care about? The best way to learn something in academic research is by conducting an experiment. You set up a treatment and control group, and give one whatever you’re testing, you can see if there’s an impact. That might be impractical in an organization though, (how would you set up a control department to compare your results?). But by tracking performance with clear metrics, we can see if there’s an impact over time, before and after the experiment. By tracking baseline data during period A, and comparing it to the same metrics collected during the experiment over period B, you can begin to draw conclusions about whether the experiment worked.

 

  Does studying in the evening or morning improve your grades? Does a big breakfast reduce the number of calories you eat? Does only answering emails during certain blocks of time improve your productivity? These are all small questions that can be answered with personal experiments.

You can think about this in an organization context too. Does allowing employees to work from home one day a week improve productivity? Does free coffee increase employee work rate? What happens if weekly meetings are cancelled and replaced with an email? These can all be tested. We could implement them without a test first, but small nimble experiments allow us to keep learning and seeking ways to make improvements.

What are the characteristics of a good experiment?

Its specific. Not “walk more”, but contains explicit details: “get 12000 steps every day”. You can’t evaluate a hypothesis if you don’t actually test it.

Its temporary. Don’t throw out all your forks to see if eating only with chopsticks increases your mental agility. Use the chopsticks for a week and see what happens first.

Its measurable. If you want to eat Keto and see if it improves your health, you have to know how you’ll measure health. Lost weight? Increased distance running? Better mood? Those measures need to be set out before you start.

Learning organizations are constantly doing experiments. Facebook shifts things around on your newsfeed to see how you react. Online newspapers change their headlines to see what gets the most clicks. Amazon tests different ads to see what generates purchases. These are all small tests, but they speak to a desire to understand how they can improve. Imagine the improvement you might see over a year if you tried doing something different every week. A lot of those tests would fail, but imagine the gains from the 2 or 3 that worked?

You have more data in your life now than was imaginable a century ago. Every minute you’re generating data, and just as importantly, it’s being counted.

Types of Data

There are two types of micro data that every individual is producing every minute: Passive and Active data. We’ll review both kinds, and give some examples.

Active data is collected, well, actively. Someone has to do the work of entering it in order for it to exist. For example, if you track your calories, you have to do that actively. There’s no app or device that can automatically figure out what you’ve eaten. There are ways for it to be simpler (using pictures or recipes, scanning barcodes), but you have to be active in the data collection. There are lots of ways we can build active data about ourselves using self-recorded measures.

Schedule

You could keep track of how you structure your day. When did you work/study, when do you eat your meals, do you take a nap? Or, simpler than a detailed schedule of your day, one can actively record whether certain activities were performed (whether you exercised, whether you drove or biked to work, whether you gardened, etc.).

Mood trackers

There are lots of apps or websites that will help you track your moods throughout the day.

  • Moodpanda
  • In Flow
  • Moodscope
  • AskMeEvery
  • Eating
  • Food diaries can help you collect rich data about your diet
  • Meal Snap – App ($2)

The contrast to active data collection is passive data collection, This doesn’t take daily activity to be collected. Everyday we wear and carry devices that are passively building data about us. One of the most common is a fitness tracker. I wear a FitBit, which counts my steps, sleep, and activity everyday without me thinking about it actively. I have to choose to wear it and I actively chose to purchase it, but beyond that the data is collected passively. Some passive data is collected without us directly choosing to have it done. For instance, credit card companies keep detailed records of your spending, logging every transaction you make. Your smartphone tracks your overall activity, and sends you reports on your weekly usage.

Sensors like FitBit, mobile phones

  • Most fitness trackers combine steps, activity, sleep, gps, heartrate…
  • Fitbit
  • Nike Fuelband
  • Jawbone UP
  • Senkefei ($14)
  • Seegar ($20)
  • Arbily ($25)
  • AmazFit Smarwatch ($80)
  • Phones can track steps too, apps can track sleep

Administrative Data which are logged via activities. They collect passively, but with less direct permission than passive. An indirect record of activity

  • RescueTime -a program you can download to your computer to track how much you work/play/socialize
  • Credit Card Transactions
  • Apple/Android time tracking

There are surely many other ways to track the data you generate about yourself. THink of every hour you do in a dat, and imagine it being tracked (text messages sent, netflix watched, emails ignored) and there’s probably a way to get that data collected either actively or passively.

To sum up

Data is of growing importance to organizations. But beyond collecting data, using data is the real goldmine. Nimble experiments are an important part of being a learning individual or working at a learning organization. Hopefully the final project can help to demonstrate that.

Oh, and by the way, I wrote this document in Markdown.