Hi, it’s Lukas. After I quit my data analyst job in market research company, I spent some time volunteering in a community based in south of Portugal. It was a indeed refreshing experience, which I highly recommend to everyone, who wants to get out of the regular worldly routine. When coming back home, I decided that unless I find a very fine job, I would earn my bread by freelancing. But… I found 2 problems:

  1. I didn’t know if my skills were actually worth anything in freelance market
  2. I didn’t have portfolio. As I worked mostly in corporate environment, most of the results I achieved were protected by contracts and couldn’t be shared.

So, as a saying goes I decided to “shoot two rabbits by one shot”: Research the data-related freelance job market and create a report, hence creating some real-world examples to my portfolio.

Below is provided analysis of data related job listings for the period of 1 year using R. Listings were scraped from one of the leading freelancing jobs sites using rvest package. Scraped data consists of over 11,000 listings. To make it usable, I cleaned and transformed data.

Skill demand

First, I decided to check, which skills have the highest demand. Every job listing in the website provides a set of skills that are required for a particular positions. To extract the exact skills, I used text mining technique with the help of R’s tidytext package. Of course, there are other and probably easier ways to do it, but I wanted to show that I understand and am able to perform tokenization. Below is the word-cloud showing the popularity of the skills among the listings. I was a bit surprised that Python skills have way higher demand then R (it didn’t even made it to the top 50). Though many more broad skills, like “Statistics”, “Data mining” or “Machine learning” may use R as a tool.

Hourly rates

Even more interesting than skill demand is the pay. Part of the retrieved data had information on freelancers bids for hourly wages. I selected 20 skills that have the highest demand and sorted them out by hourly rates. To me it seems logical that the highest pay goes to developers of various sorts. Then a bit below one finds internet marketing, sales and research. And at the bottom - more manual kind of jobs, like data entry, web search and so on. As I have some R programming skills, I would hope to get a higher pay. Although, it is not clear if R is a match for Python here, as it definitely is not that popular. Maybe I should consider learning Python. :/

Project budgets

Rest of the jobs (actually a majority ~85%) instead of hourly rates are paid per project. Here also, I selected 20 skills that have the highest demand and sorted them out by the size of project budget bids. It was quite surprising that Excel projects turns out to have bigger budgets then Python or MySQL, as the hourly rates are higher for the latter. This could mean that Excel projects tend to be more extensive.

Competition

This is other important aspect of the market that I want to have a look into. In the Bubble chart below I included already explored information on hourly rates and the number of jobs available for particular skills as well as the competition. The competition is measured by the number of freelancers applying for the jobs. So basically, what I want to look for is the biggest bubble in the bottom-right corner of the chart. This would mean the lowest competition, highest hourly wage and lots of jobs to apply for. I am really starting to consider learning Python. :D It looks also interesting, that not only wages are highest for mobile app developers, but the competition as well.

Python jobs. Hourly rates

Python jobs already caught my eye, as they are on demand, with relatively low competition and good wages. I would like to look more closely on what is actually being offered. Histogram below shows, that predominant wages are 15-25 USD/hour. Number of observations are rather small, because most of the jobs are paid per project and not per hour, so one needs to be cautious before making any generalised conclusions.

Python jobs. Project budgets

Density plot below shows, that except few large and very large projects, most of the budgets are under $1000.

Let’s zoom in a little bit. We can see, that most of projects are actually under $600 with most of them being under $300.

In density plot we can’t see the actual number of projects, so here is the same data in a form of histogram. About 2/3 of all Python projects are under $250. So having in mind, that average wage rate is around $20/hour, most projects would take 2-10 hours to accomplish. Another ~20% of the projects (68 in count) have budgets between $250 and $800, these would be the 3-5 working days projects. While projects above $800 (43) constitute another ~13%, most of which (~7%, 22) are projects between $2000 and $5000. The latter ones would be the 2 weeks to a month or more projects. This is for the period of 1 year.

To sum up the findings, I should say the numbers does not look promising to me. The number of projects offered are rather small, with total yearly market value of $243k (for Python jobs). Most of the projects have rather small budgets. If I would win most of the projects that I would like to undertake, I could earn solid enough income, but I see that as rather unlikely as I do not have much experience in data-freelancing. Maybe to come back to office work is not such a bad idea for now… :)

Python jobs. Further exploration

After exploring the data a little bit, I have a feeling that Python is a very general skill, that has many diferent types of more specific skills and hence the difference in wages and budgets. So I tested this. Again I used some text mining to extract most common bigrams (combinations of two words) from the titles of the listings. Besides some general terms, I was able to extract some more specific skills, that actually overlaps with the skills from the previous analysis. In the bubble chart below, we can see that the high budget projects are usually asociated with Machine Learning, Web Applications and Artificial Intelligence Machines. There is relatively large number of Web Scraping projects. However, as there is not much data here, it is quite difficult to make any firm conclusions, although we get some hints.

Sum up

I have to admit, I thought there would be more jobs related to Python/R programming, data analysis, machine learning, because these are the areas I’d like to work in (in many times R and Python skills are interchangable). Of course, it does not mean, that it is not possible to work full-time as freelance data analyst, but it doesn’t make the things easier neither. My feeling would be to work on a bigger budget projects, so that I wouldn’t need to look for a new project every couple of days. Therefore, I see that Machine Learning and AI would be the way to go (these are the most interesting areas too!). So my decision for now is to learn more and do some projects on those topics. And for now, I will look for a office work, that is most in-line with these goals.

Thank you!

Lukas