Data Codebook

Description of variables, units of measurements, data category and the source of data, can be found on the tables below:

Codebook Varible Description

Codebook Varible Units of measurement

Codebook Varible Source of data

• Raw data in original .csv and .xls formats can be found on the following Link
• Data wrangling code used on this work, can be found on the following Link
• Already cleaned up .RData files can be found on the following Link

Objective

The main purpose of this work is to create models that are able to predict the rig activity (Activity_Rig) in US and the behavior of oil price (Oil_Price), based on various explanatory variables. In order to achive this objective it is necessary to first develop insight on variables that might influence the behavior of both response variables.

Exploratory data analysis

The first step in our analysis will be to upload the dataset, and explore its structure, the descriptive statistics and if weather or not observations in the dataset are complete.

'data.frame':   552 obs. of  20 variables:
 $ Date              : Date, format: "1973-01-01" "1973-02-01" ...
 $ Month             : Factor w/ 12 levels "01","02","03",..: 1 2 3 4 5 6 7 8 9 10 ...
 $ Year              : num  1973 1973 1973 1973 1973 ...
 $ Active_Rigs       : num  1219 1126 1049 993 1046 ...
 $ US_Population     : num  212 212 212 212 212 ...
 $ President_Party   : Factor w/ 2 levels "Democrat","Republican": 2 2 2 2 2 2 2 2 2 2 ...
 $ Senate            : Factor w/ 2 levels "Democrat","Republican": 1 1 1 1 1 1 1 1 1 1 ...
 $ House             : Factor w/ 2 levels "Democrat","Republican": 1 1 1 1 1 1 1 1 1 1 ...
 $ life_exp          : num  71.4 71.4 71.4 71.4 71.4 ...
 $ GDP               : num  1.43 1.43 1.43 1.43 1.43 ...
 $ Oil_Production    : num  10856 11140 11006 11042 11002 ...
 $ Oil_Stock         : num  237 235 244 248 257 248 243 248 241 246 ...
 $ Oil_Price         : num  3.89 3.89 3.89 3.89 3.89 3.89 3.89 3.89 3.89 3.89 ...
 $ HDD               : int  965 843 552 402 215 26 8 11 74 246 ...
 $ CDD               : int  5 3 19 13 62 214 297 280 142 50 ...
 $ Energy_Consumption: num  7.23 6.59 6.52 5.94 6.07 ...
 $ Net_imports_Energy: num  1.047 1.047 1.17 0.891 0.966 ...
 $ Oil_Reserves      : num  35300 35300 35300 35300 35300 35300 35300 35300 35300 35300 ...
 $ Active_Conflicts  : int  52 52 52 52 52 52 52 52 52 52 ...
 $ Drilled_Wells     : int  2166 1896 2151 1883 2298 2279 2367 2690 2333 2538 ...

      Date                Month          Year       Active_Rigs    
 Min.   :1973-01-01   01     : 46   Min.   :1973   Min.   : 407.0  
 1st Qu.:1984-06-23   02     : 46   1st Qu.:1984   1st Qu.: 856.8  
 Median :1995-12-16   03     : 46   Median :1996   Median :1184.5  
 Mean   :1995-12-16   04     : 46   Mean   :1996   Mean   :1435.0  
 3rd Qu.:2007-06-08   05     : 46   3rd Qu.:2007   3rd Qu.:1851.5  
 Max.   :2018-12-01   06     : 46   Max.   :2018   Max.   :4521.0  
                      (Other):276                                  
 US_Population     President_Party        Senate           House    
 Min.   :211.9   Democrat  :240    Democrat  :288   Democrat  :312  
 1st Qu.:235.8   Republican:312    Republican:264   Republican:240  
 Median :267.8                                                      
 Mean   :268.6                                                      
 3rd Qu.:301.2                                                      
 Max.   :327.0                                                      
                                                                    
    life_exp          GDP         Oil_Production    Oil_Stock     
 Min.   :71.36   Min.   : 1.429   Min.   : 5488   Min.   : 233.0  
 1st Qu.:74.56   1st Qu.: 4.041   1st Qu.: 7769   1st Qu.: 763.8  
 Median :75.82   Median : 7.882   Median : 9160   Median : 898.0  
 Mean   :75.98   Mean   : 9.027   Mean   : 9323   Mean   : 823.4  
 3rd Qu.:77.99   3rd Qu.:14.419   3rd Qu.:10252   3rd Qu.:1003.2  
 Max.   :78.84   Max.   :20.660   Max.   :16289   Max.   :1230.0  
                                                                  
   Oil_Price           HDD              CDD        Energy_Consumption
 Min.   :  3.89   Min.   :   3.0   Min.   :  2.0   Min.   :5.438     
 1st Qu.: 14.01   1st Qu.:  45.0   1st Qu.: 12.0   1st Qu.:6.592     
 Median : 23.51   Median : 310.0   Median : 42.0   Median :7.552     
 Mean   : 33.28   Mean   : 378.9   Mean   :103.6   Mean   :7.382     
 3rd Qu.: 45.19   3rd Qu.: 660.0   3rd Qu.:183.8   3rd Qu.:8.048     
 Max.   :128.08   Max.   :1266.0   Max.   :404.0   Max.   :9.655     
 NA's   :1        NA's   :2        NA's   :2       NA's   :2         
 Net_imports_Energy  Oil_Reserves   Active_Conflicts Drilled_Wells 
 Min.   :0.06212    Min.   :19121   Min.   : 52.0    Min.   :1268  
 1st Qu.:0.94186    1st Qu.:22351   1st Qu.: 83.5    1st Qu.:2288  
 Median :1.30195    Median :26544   Median :132.5    Median :2914  
 Mean   :1.39765    Mean   :26705   Mean   :121.8    Mean   :3482  
 3rd Qu.:1.85828    3rd Qu.:30529   3rd Qu.:151.2    3rd Qu.:4208  
 Max.   :2.74169    Max.   :39160   Max.   :205.0    Max.   :8556  
 NA's   :2          NA's   :12      NA's   :24       NA's   :96    

              Date              Month               Year 
                 0                  0                  0 
       Active_Rigs      US_Population    President_Party 
                 0                  0                  0 
            Senate              House           life_exp 
                 0                  0                  0 
               GDP     Oil_Production          Oil_Stock 
                 0                  0                  0 
         Oil_Price                HDD                CDD 
                 1                  2                  2 
Energy_Consumption Net_imports_Energy       Oil_Reserves 
                 2                  2                 12 
  Active_Conflicts      Drilled_Wells 
                24                 96 

The variable with most NA’s in the data set is Drilled wells, the dataset of the EIA did not provide this information since the end of 2010. for the exploratory analysis the complete dataset will be used altought for modeling and predicting perhaps this variable should be dropped or only use a complete.case dataset.

After the analysis of the structure of the dataset, we will first start with a series of pairs plot for different categories of variables, that for the sake of simplicity we have divided into the following groups: Environmental, Socioeconomic, Political, Oil related, Energy related

Environmental

Altogh pairwise graphs can result complicated, they migh highlight interesting patterns of our data, for instance:

• Active rigs and oil prices have a direct positive relationship, however it seems like there are two different groups and trends, two different lines
• With the years, in general oil prices have increased and the rig activity have decreased
• CDD and HDD have a clear relationship with months so it might be convenient to look for seasonality in our data set
• Altough the data is noisy, it seems to be that HDD have decreased with the pass of the years and CDD have increased (spooky global warming???), might be interesting to group the data by month or season and then look at the behavior with years
• CDD and HDD can be described as mutually exclusive, when its hot it is unlikely that is cold at the same time (HDD vs CDD scatterplot)

Social & economic variables

Allmast all the variables share an strong linear relationship with year, with the exception of Activity rigs, therefore we see a similar trend between them.

Oil Market

• Oil production had a downwards trend up to the beginning of 2000, on which there is clear change in trend (perhaps a different energy policy???)
• There is an strong linear relationship between the US oil reserves and the production, perhaps one of the strongest incentive to increase the production has been the growth of US proven reserves. Looking at both curves Production vs Year and reserves vs Year we can see how they allmost overlap
• There is an strong linear relationship between the amount of drilled wells and the number of active rigs
• Many of the variables seem to show a dual behavior, this might be related to their dependence in time and of course on political factors that determine the energy policy of the country

Energy Market

• With time, the energy consumption of US has increased, which is logical, since factors as population, life expectancy and GDP has grown as well with time
• There is sharp break point on the net energy imports curve in relationship with time, on the mid 2000’s the upwards linear trend was replaced by an strong downward trend (Shale boom), it might be interesting to divide the data on two different periods of times, since it seems there was a 180 degrees change in energetic policy for the US

Political

This plot is extremely noisy due to the fact that some of these variables are categoricals, however it still provides some interesting information

• It seems to be a relationship between Oil prices and the ruling party at the different components of the US government, this is something worth to be explored with further detail
• Active conflicts and oil prices seems to share a positive relationship (not necesarily linear but positive), this seems to be logical, US is one of the biggest arm producer and this industry has to be fueled, as per market rules, a higher demand on a product the higher the prices
• The number of active conflicts seems to share a relationship with the ruling party at the different components of the US government
• The amount of active rigs seems to be unafected by politics
• Active conflicst worldwide have been increasing in time, perhaps their nature have changed but the number has increased with years

   shared     Total 
0.6521739 0.3478261 

  Democrat Republican     shared 
 0.1739130  0.1739130  0.6521739 

   Normal Recession 
0.8605072 0.1394928 

Statistical Inference

Our Previous analysis have lead us to various questions, related to the nature of the relationships between our response variables explanatory ones. The initial EDA has uncovered or to be more precise it has awaken our curiosity regarding this questions, but the answers can only be found using more advanced techniques. One this section we intend to answer this questions.

All the code of the statistical tests performed on this section, can be found on the following Link

Does oil price varies with Months?

To answer this question we will first use and ANOVA test and if it provides significant differences then we will perform a tukeyHSD test to find out which categories are truly different.We will use and alpha of 5% or 0.05

As we can see there is no a significative difference since we obtained an ANOVA test p-value of 0.76-76%

Does Active_Rigs varies with Months?

To answer this question we will first use and ANOVA test and if it provides significant differences then we will perform a tukeyHSD test to find out which categories are truly different.We will use and alpha of 5% or 0.05

As we can see there is no a significative difference since we obtained an ANOVA test p-value of 0.29-29%

Does oil price varies with Seasons?

To answer this question we will first use and ANOVA test and if it provides significant differences then we will perform a tukeyHSD test to find out which categories are truly different.We will use and alpha of 5% or 0.05

As we can see there is no a significative difference since we obtained an ANOVA test p-value of 0.84-84%

Does Active Rigs varies with Seasons?

To answer this question we will first use and ANOVA test and if it provides significant differences then we will perform a tukeyHSD test to find out which categories are truly different.We will use and alpha of 5% or 0.05

As we can see there is no a significative difference since we obtained an ANOVA test p-value of 0.55-55%

Does Oil Price varies with the US Presidents political Party

To answer this question we will use a T test and we will evaluate the obtained p-values and the confidence interval of the test parameter.We will use and alpha of 5% or 0.05

    Welch Two Sample t-test

data:  OIL_MODIFIED$Oil_Price by OIL_MODIFIED$President_Party
t = 3.7616, df = 405.02, p-value = 0.0001937
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
  4.42473 14.11211
sample estimates:
  mean in group Democrat mean in group Republican 
                38.51321                 29.24479 

As we can see there difference is indeed a significative difference since we obtained T test p-value of 0.0001937 at a 95% confident interval, which is below our selected alpha level of 5%. Based on the values of the test we can claim that Democrat US presidents have experieced on average oil prices between 4.4 and 14.1 $ higher than republican presidents.

Does Active Rigs varies with the US Presidents political Party?

To answer this question we will use a T test and we will evaluate the obtained p-values and the confidence interval of the test parameter.We will use and alpha of 5% or 0.05

    Welch Two Sample t-test

data:  OIL_MODIFIED$Active_Rigs by OIL_MODIFIED$President_Party
t = -2.1034, df = 541.3, p-value = 0.03589
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -253.424469   -8.662711
sample estimates:
  mean in group Democrat mean in group Republican 
                1360.950                 1491.994 

As we can see there difference is indeed a significative difference since we obtained T test p-value of 0.03589-3.6 % at a 95% confident interval, which is below our selected alpha level of 5%. Based on the values of the test we can claim that Democrat US presidents on average have experienced a number of active rigs between 253 and 9 rigs lower than republican presidents.

Does Oil price varies with the majority ruling party of US Senate?

To answer this question we will use a T test and we will evaluate the obtained p-values and the confidence interval of the test parameter.We will use and alpha of 5% or 0.05

    Welch Two Sample t-test

data:  OIL_MODIFIED$Oil_Price by OIL_MODIFIED$Senate
t = 2.6457, df = 393.49, p-value = 0.008479
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
  1.55085 10.52260
sample estimates:
  mean in group Democrat mean in group Republican 
                36.16326                 30.12654 

As we can see there difference is indeed a significative difference since we obtained T test p-value of 0.008479-0.8 % at a 95% confident interval, which is below our selected alpha level of 5%. Based on the values of the T test we can claim that Democrat dominant Senate has on average have experienced Oil prices between 1.5 and 10.5 $ higher than a dominant republican senate.

Does Active Rigs varies with the majority ruling party of US Senate?

To answer this question we will use a T test and we will evaluate the obtained p-values and the confidence interval of the test parameter.We will use and alpha of 5% or 0.05

    Welch Two Sample t-test

data:  OIL_MODIFIED$Active_Rigs by OIL_MODIFIED$Senate
t = 1.9558, df = 448.03, p-value = 0.05111
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
  -0.6066393 251.1350484
sample estimates:
  mean in group Democrat mean in group Republican 
                1494.927                 1369.663 

As we can see there is no a significative difference since we obtained an T test p-value of 0.05111-5.1%

Does Oil price varies with the majority ruling party of US House?

To answer this question we will use a T test and we will evaluate the obtained p-values and the confidence interval of the test parameter.We will use and alpha of 5% or 0.05

    Welch Two Sample t-test

data:  OIL_MODIFIED$Oil_Price by OIL_MODIFIED$House
t = -8.5814, df = 447.84, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -24.24962 -15.21222
sample estimates:
  mean in group Democrat mean in group Republican 
                24.72343                 44.45435 

As we can see there difference is indeed a significative difference since we obtained T test p-value of 2.2e-16 at a 95% confident interval, which is below our selected alpha level of 5%. Based on the values of the T test we can claim that Democrat dominant House has on average experienced Oil prices between 24.24 and 15.21 $ lower than a dominant republican house.

Does Active Rigs varies with the majority ruling party of US House?

To answer this question we will use a T test and we will evaluate the obtained p-values and the confidence interval of the test parameter.We will use and alpha of 5% or 0.05

    Welch Two Sample t-test

data:  OIL_MODIFIED$Active_Rigs by OIL_MODIFIED$House
t = 9.3118, df = 493.68, p-value < 2.2e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 407.6818 625.7310
sample estimates:
  mean in group Democrat mean in group Republican 
                1659.673                 1142.967 

As we can see there difference is indeed a significative difference since we obtained T test p-value of 2.2e-16 at a 95% confident interval, which is below our selected alpha level of 5%. Based on the values of the T test we can claim that Democrat dominant House has on average experienced a Rig count between 408 and 626 rig count higher than a dominant republican house.

Does Oil prices varies if the US government is completely dominated by one sigle party at all levels?

To answer this question we will use a T test and we will evaluate the obtained p-values and the confidence interval of the test parameter.We will use and alpha of 5% or 0.05

    Welch Two Sample t-test

data:  OIL_MODIFIED$Oil_Price by OIL_MODIFIED$Gov_ctrl_type
t = -0.24384, df = 504.06, p-value = 0.8075
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -4.963467  3.867454
sample estimates:
mean in group shared  mean in group Total 
            33.09189             33.63990 

As we can see there is no a significative difference since we obtained an T test p-value of 0.8075-80.75%

Does Active rigs varies if the US government is completely dominated by one sigle party at all levels?

To answer this question we will use a T test and we will evaluate the obtained p-values and the confidence interval of the test parameter.We will use and alpha of 5% or 0.05

    Welch Two Sample t-test

data:  OIL_MODIFIED$Active_Rigs by OIL_MODIFIED$Gov_ctrl_type
t = 0.49325, df = 477.12, p-value = 0.6221
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -90.53302 151.21913
sample estimates:
mean in group shared  mean in group Total 
            1445.572             1415.229 

As we can see there is no a significative difference since we obtained an T test p-value of 0.6221-62.21%

On Scenarios of complete control by one party, does oil prices varies in relationship to the party?

To answer this question we will first use and ANOVA test and if it provides significant differences then we will perform a tukeyHSD test to find out which categories are truly different.We will use and alpha of 5% or 0.05

As we can see there difference is indeed a significative difference since we obtained an ANOVA test p-value of 0.0010171 at a 95% confident interval, which is below our selected alpha level of 5%. With such a difference we will proceed with the TukeyHSD to find out which categories present a significative difference. It is importatnt to highlight that tukeyHSD already presents the corrected or adjusted p-values based on the Bonferroni correction.

Based on the results we can claim that a completely controled republican goverment has experienced oil prices between 5.5 and 24.15 $ higher than a completely controled democrat government, It is also interesting to mention that a completely controled republican goverment shows higher oil prices in comparisson with a shared government.

On Scenarios of complete control by one party, does Active Rigs varies in relationship to the party?

To answer this question we will first use and ANOVA test and if it provides significant differences then we will perform a tukeyHSD test to find out which categories are truly different.We will use and alpha of 5% or 0.05

As we can see there difference is indeed a significative difference since we obtained an ANOVA test p-value of 1.6e-06 at a 95% confident interval, which is below our selected alpha level of 5%. With such a difference we will proceed with the TukeyHSD to find out which categories present a significative difference. It is importatnt to highlight that tukeyHSD already presents the corrected or adjusted p-values based on the Bonferroni correction.

Based on the results we can claim that a completely controled republican goverment has experienced lower levels of rig activity in comparisson with a fully democrat government or shared government. Completely Democrat governments has experienced the highest levels of active rigs.

Does Oil prices varies if the US economy is on recession?

To answer this question we will use a T test and we will evaluate the obtained p-values and the confidence interval of the test parameter.We will use and alpha of 5% or 0.05

    Welch Two Sample t-test

data:  OIL_MODIFIED$Oil_Price by OIL_MODIFIED$Eco_Rec
t = -0.045776, df = 95.274, p-value = 0.9636
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -7.707692  7.360243
sample estimates:
   mean in group Normal mean in group Recession 
               33.25757                33.43130 

As we can see there is no a significative difference since we obtained an T test p-value of 0.9636-96.4%

Does Active Rigs varies if the US economy is on recession?

    Welch Two Sample t-test

data:  OIL_MODIFIED$Active_Rigs by OIL_MODIFIED$Eco_Rec
t = -5.565, df = 85.947, p-value = 2.925e-07
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -905.1819 -428.6917
sample estimates:
   mean in group Normal mean in group Recession 
               1341.985                2008.922 

As we can see there difference is indeed a significative difference since we obtained T test p-value of 2.925e-07 at a 95% confident interval, which is below our selected alpha level of 5%. Based on the values of the T test we can claim that US economy on recession has on average experienced a Rig count between 428 and 905 rig count higher than a US economy on normal conditions.

Does oil price behavior is independent of price behavior on previous month?

This might sound like an unrelated question, but it is on the contrary fundametal for this work, what is the point of attempting to predict via regression or forecasting methods, the behavior of a variable if it responds to randomness with a certain probability? and if this is the case it is critical to now it in advance instead of attempting to develop models that are fundamentally wrong from the conception.

It is very commong to hear from time to time that trading markets like forex or stock markets, behave randomly, that the ups and downs of prices follows patterns as such the throwing of a coin; being oil a commodity it is sometimes accused of the same issue.

This sounds like a very complex question, and it is, but a chi-square GOF test can be used to study the problem. We will label each observaion as Up or Down (D) depending on whether the Oil price market was up or down that month.

DOWN   UP 
 235  316 

     DOWN        UP 
0.4264973 0.5735027 

For our dataset, composed of 552 monthly observations of Oil prices, starting on january 1973, the proportion of UP’s in the market is of 57%. The next step in our analysis will be to generate an histogram of the distance between UP’s, how many months of DOWN we have before experiencing again a market turnup. We will cal this variable streak

Since the chi-square GOF test, requires to have at least 5 observations on each category, we will group up values of streak above 6 days.

With the streak distribution, the next step is to calculate the distribution of a geometric distribution with a success probability of 57%, on which values above 6 are grouped together.

Having the geometric distribution proportions and the original table of oil price streak frequencies we can perform a chi-square goodness of fit test, the selected alpha value for this test will be 5% or 0.05.

    Chi-squared test for given probabilities

data:  price_observed
X-squared = 102.79, df = 6, p-value < 2.2e-16

As we can see the obtained p-value of the test is 2.2e-16, almost 0 which means that it is almost impossible to find the oil price streak distribution if we assume the null hypothesis of a geometric distribution with succes probability of 57%.

Since this is a complicate question we will confirm our test, by using simulation instead of the predefined statistical functions of R. We will replicate with replacement 10000 (10 thousands) observations of our dataset, and calculate the chi-square statistic assuming a null hypothesis on which our distribution is equal to the geometric distrbution with a probability of success of 57%, and then we will calculate the p-value of obtaining our observed chi-square statistic.

[1] 10000     2

With this new data frame of simulations we will calculate the simulated p-value which is just the proportion of simulated stats, more extreme or equal to our observed chi-square stat.

# A tibble: 1 x 1
  `mean(stat >= oil_observed_chi)`
                             <dbl>
1                                0

The blue curve of this graph corresponds to a chi-square distribution with 6 degrees of freedom, we can see how it almost perfecly overlap the black curve which is our simulated distribution, this give us confirmation that our simulation was correct.

As we can see both graphically and with the calculation this proportion or simulated p-value is virtually zero, which means that our oil price streak distribution doesnt follow a geometric distribution and which means as well that, there is no independence between the behavior of oil price one month after one other, making this behavior not random. This confirm our suspicious that oil prices are influenced by time and many other variables.

Why monthly oil prices and not daily? how volatiles are daily oil prices?

This is an interesting question and perhaps somehow complicated to answer, the idea of this study is to capture macroeconomic or industry related trends that on the somehow long-term cause oil prices to raise or to drop, oil being traded as a commodity on daily basis might be more volatile and more susceptible to speculation.

To test this, we are going to evaluate the effect of, US president Donald Trump, oil & energy related tweets, on the daily Brent crude price. We are going to use only tweet from 2016 up to the first quarter of 2019.

Compendium of trump tweets were downloaded in json format from the following link.

The final database and transformation code can be found on the following link.

The Trump tweets archive that is used on this workd is composed by 11363 tweets, here is an example of the first lines of the archive:

There are 1186 days between the first day of the database and the last one, using this means that president trump is tweeting on average 9.6 tweet per day since january 2016, until april 2019, which makes him very active on this social media.

for the sake of this work, we will focus only on tweets on which he used the following words, since we consider them possibly related to the oil & gas market:

• oil
• gas
• opec
• pipeline
• barrel
• oil price
• drilling
• oil production
• gas price
• Sanction
• Sanctions
• Russia

after filtering the complete database we found 421 positive tweets for oil related words. Already having this information in place, the first question that comes to my mind is weather or not there is a difference in brent prices, when trump tweets about oil and when he doesnt? to answer this question we will use a t.test with an alpha value of 5%

    Welch Two Sample t-test

data:  trump_oil2$value by trump_oil2$oil_keyword
t = -7.3975, df = 344.41, p-value = 1.068e-12
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -8.974548 -5.204563
sample estimates:
mean in group FALSE  mean in group TRUE 
           55.10396            62.19351 

Interestingly there is indeed a significative difference between brent crude prices when trump tweets about oil, in comparisson with days on which he does not. the obtained p-value is 1.068e-12 which is lower that our alpha level making the test significative. Brent crude prices tend to be higher on days on which trumo tweets about oil, between 5.2 and 8.79 dollars higher.

Now obtaining this result only means that there is indeed a correlation, however it doesnt imply causation, it is important not to be confused and think that oil related tweet of trump cause higher oil prices. In fact we can conclude that is higher oil prices, what motivates Donald trump to tweet about oil prices.

To test this hypothesis we will look and the distribution of oil tweet of trump with oil prices ranges

As we can see, the majority of oil tweets of donald trump, are condensed on high oil price ranges, but the real question is, is this distribution sufficiently different from expected values, to consider it significative? to test this we will use a chi-square independence test, using an alpha value of 0.05-5%.

    Pearson's Chi-squared test

data:  table(cut(trump_oil2$value, 3), trump_oil2$oil_keyword)
X-squared = 43.082, df = 2, p-value = 4.414e-10

The results obtained by the chi-square test, suggest that there is no independence between oil tweets and oil price ranges. which means that trump oil tweets are triggered by differences in oil prices, and this relationship is significative since the obtained p-value of 4.414e-10 is below the 5% alpha value.

If Trump oil tweets and oil prices were independent, this will be the expected distribution:

This difference can be better observed by comparing the expected and observed distributions

So it is more clear, that there a more pronounced concentration of oil tweets into the 66-87 dollars brent price binge. And we can conclude that what mostly triggers Trump oil tweets is the raise in oil prices.

It might also be interesing to study how trump oil tweets have changed over time.

Looking with more detail at the table, it is easy to notice a trend, on which the amount of oil related tweets, have increased over time, just on the first quarter of 2019, Trump has already tweeted 38 oil tweets, if we expect him to continue with this trend, we should expect 152 oil related tweets by the end of the year.

       1 
179.6667 

If we use a linear model on data of 2016,2017 and 2018 and predict 2019, tweets, we should expect at leats 179 tweets by the end of the year. So it is clear that Trump has been getting more and more active on oil tweets over time. Which move us to our next lines of questions.

Using logical thinking, as rational beings,we would only repeat and even increase a behavior if we expect it to produce the desired result, if not this might be considered to be insane or at least ilogical. The question is, does oil trump tweets, are able to decrease oil prices, at least on the shor term?

To be able to answer this question we will look at brent crude prices, the day of an oil tweet, the day after, two days after and three days after, and compare the price between these days and its behavior. We will also use linear regression inference to determine the magnitude of these effects.

After the variable creation we will perform an ANOVA test, with an alpha level of 0.05-5% to test if there is a significant level of difference on brent oil price values and the different tweet days.

The obtained p-value of 0 of the F-statistic, confirms that there is indeed a significative difference, since it is below our selected alpha level. Looking at the mean values we can observe that there is a descendent pattern on which on average the brent oil market has reacted on a downslide pattern after donal trump oil tweets

The day after an oil tweet, values are aproximately equal to the prices of the day of the tweet, altought, two days after values tend to decrease so as third days values and the forth day and so up until even a week after.

Finally we would like to confirm this trend, and to do so we will a t.test between the brent prices on Trump’s oil tweet day and the prices four days after an oil tweet, we will perform a one sided test, on which the alternative hypothesis will be that the true difference in mean is less than 0. To perform this test, we will use a significance alpha level of 5%.

    Welch Two Sample t-test

data:  trump_oil_tweet_effect$value by trump_oil_tweet_effect$tweet_pattern
t = -3.2562, df = 39.559, p-value = 0.00116
alternative hypothesis: true difference in means is less than 0
95 percent confidence interval:
      -Inf -3.537552
sample estimates:
mean in group Seven  mean in group Zero 
           54.86548            62.19351 

Based on the results obtained on the test, we can claim that there is indeed a significative difference, at a 95% confidence level, on brent price values between the tweet day and four days after a trump tweet. We can expect oil brent prices on trump’s twee day being at least 0.38 dollars higher in comparisson with four days after the tweet was made.

With such an interesting result it is important to confirm the results, due to this we will perform a similar test, but using simulation, we will simulate 10000 differences in the observed data group of brent prices on the day of tweet and four days after tweet and we will calculate a simulated p-value.

The simulated data, seems to fit very well a normal distribution, the next step is to calculate the simulated p-value, which is equivalent of calculating the proportion of values of difference more extremes or equal to the observed difference.

# A tibble: 1 x 1
   p_value
     <dbl>
1 0.000600

Equivalent to what was obtained on the formal t.test, the simulated p-value is below the 5% alpha level, which allow us to conclude that there is indeed a significative difference on brent crude prices, on the day of a tweet and four days after.

To conclude, this is the main explanation of why using daily oil price data, might not be advisible, since it might be volatile enought to be influenced by speculations created by social media interactions of powerfull entities such as the US president, and perhaps many other factors that were not evaluated during this analysis. Altought daily~weekly prices might be sensible enought to be influenced by these kind of effects, monthly prices are more related to macro economics and industry based trends and therefore are more usable to forecast on a long term basis.

Linear Regression Inference & Feature Confirmation

On the previous section of this work, we answer already plenty of the questions that the early EDA uncovered, now with the answers of this questions we can begin with the task of creating a model with the objective of predict the behavior of our response variables, but before generating definitve models it is better to first study which variables affect the most our target response variables. On this section we will explore with more depth the linear relationship of our response and explanatory variables via various methods such as:

-Spearman Correlation matrix -Full linear model -StepAIC, akaike information criterion link -BIC Bayesian information criterion link -Boruta selection criteria link

All the code related to the models built on this section can be found on the following link

Correlation matrix Oil price

We will first make a correlation matrix of Spearman correlation coefficients to determine variables that have at least a weak (0.3,-0.3) correlation with Oil prices

As we can see oil is correlated with various variables, but some of this variables are highly correlated with each others as well, one of such cases is: GDP, YEAR, US Population and life expectancy, allmost all these variables have been growing at almost the same rate with time and therefore they are perfectly correlated, this already indicate us that we should consider only using one of them for the model selection.

Correlation matrix Active Rigs

We will first make a correlation matrix of Spearman correlation coefficients to determine variables that have at least a weak (0.3,-0.3) correlation with Active Rigs.

With active rigs we see a similar trend, we have highly correlated variables in our dataset, and we would have to drop some of them to avoid multicolinearity.

Full Linear model Y = Oil Price

We will first begin by making a linear model of oil price versus all of the available exploratory variables.

Call:
lm(formula = Oil_Price ~ ., data = OIL_MODIFIED_MODEL)

Residuals:
    Min      1Q  Median      3Q     Max 
-44.275  -2.575   0.076   2.771  39.407 

Coefficients: (2 not defined because of singularities)
                            Estimate Std. Error t value Pr(>|t|)    
(Intercept)               -2.551e+04  5.085e+03  -5.016 7.71e-07 ***
Month                      2.759e-01  1.250e-01   2.207 0.027838 *  
Year                       1.390e+01  2.722e+00   5.107 4.93e-07 ***
Active_Rigs                3.909e-03  1.555e-03   2.513 0.012323 *  
US_Population             -7.068e+00  9.087e-01  -7.778 5.47e-14 ***
President_PartyRepublican -4.026e+00  1.094e+00  -3.678 0.000264 ***
SenateRepublican           2.555e+00  2.199e+00   1.162 0.245921    
HouseRepublican           -6.122e+00  2.710e+00  -2.259 0.024353 *  
life_exp                  -5.437e+00  2.236e+00  -2.432 0.015420 *  
GDP                        2.526e+01  2.025e+00  12.470  < 2e-16 ***
Oil_Production             2.847e-03  1.418e-03   2.008 0.045261 *  
Oil_Stock                 -9.132e-02  1.435e-02  -6.363 5.04e-10 ***
HDD                        2.705e-03  4.770e-03   0.567 0.570874    
CDD                        1.958e-02  1.168e-02   1.676 0.094369 .  
Energy_Consumption         2.127e-01  1.854e+00   0.115 0.908724    
Net_imports_Energy        -1.508e+01  2.566e+00  -5.875 8.46e-09 ***
Oil_Reserves              -2.038e-03  4.586e-04  -4.444 1.13e-05 ***
Active_Conflicts           4.234e-02  2.940e-02   1.440 0.150501    
Drilled_Wells              2.178e-03  8.049e-04   2.707 0.007067 ** 
Eco_RecRecession          -4.553e-01  1.039e+00  -0.438 0.661531    
SeasonSpring               1.360e+00  1.199e+00   1.134 0.257560    
SeasonSummer              -1.021e+00  1.760e+00  -0.580 0.562125    
SeasonWinter              -2.240e+00  1.615e+00  -1.387 0.166124    
Gov_ctrl_typeTotal        -1.438e+00  1.026e+00  -1.401 0.161852    
Gov_ctrl_partyRepublican          NA         NA      NA       NA    
Gov_ctrl_partyshared              NA         NA      NA       NA    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 6.597 on 432 degrees of freedom
  (96 observations deleted due to missingness)
Multiple R-squared:  0.9068,    Adjusted R-squared:  0.9018 
F-statistic: 182.7 on 23 and 432 DF,  p-value: < 2.2e-16

Analysis of Variance Table

Response: Oil_Price
                    Df Sum Sq Mean Sq   F value    Pr(>F)    
Month                1    309     309    7.1060  0.007971 ** 
Year                 1  91169   91169 2094.8581 < 2.2e-16 ***
Active_Rigs          1  50324   50324 1156.3228 < 2.2e-16 ***
US_Population        1    819     819   18.8116 1.798e-05 ***
President_Party      1   4745    4745  109.0386 < 2.2e-16 ***
Senate               1  15114   15114  347.2866 < 2.2e-16 ***
House                1   6137    6137  141.0032 < 2.2e-16 ***
life_exp             1    176     176    4.0327  0.045249 *  
GDP                  1   7642    7642  175.5931 < 2.2e-16 ***
Oil_Production       1     50      50    1.1425  0.285715    
Oil_Stock            1    419     419    9.6215  0.002049 ** 
HDD                  1    638     638   14.6708  0.000147 ***
CDD                  1      0       0    0.0104  0.918874    
Energy_Consumption   1    151     151    3.4625  0.063454 .  
Net_imports_Energy   1   2809    2809   64.5378 9.106e-15 ***
Oil_Reserves         1   1665    1665   38.2601 1.437e-09 ***
Active_Conflicts     1     53      53    1.2128  0.271390    
Drilled_Wells        1    266     266    6.1185  0.013761 *  
Eco_Rec              1      2       2    0.0538  0.816692    
Season               3    319     106    2.4455  0.063397 .  
Gov_ctrl_type        1     85      85    1.9635  0.161852    
Residuals          432  18801      44                        
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

A linear model of all variables included, give us an adjusted coefficient of determination of 0.9018, and an RMSE value of 6.597 dollars. The ANOVA test of the coefficients tell us that the variables which are non significant, at an alpha level of 5%, for this model are:

• Oil_Production
• CDD
• Energy_Consumption
• Active_Conflicts
• Eco_Rec
• Season
• Gov_ctrl_type
• Gov_ctrl_party

To evaluate with more detail the generated model we should look at the residuals to evaluate their normality and constant variability criteria

Residuals plot, show us that they are almost centered and scatered around 0, except for high oil prices levels, they also shown certain levels of “heteroscedasticity”, meaning that the residuals get larger as the prediction moves from small to large. This is something we need to keep in mind for further evaluation of variables transformation. The qqplot show us a similar story than the residuals plot.

Full Linear model Y = Active Rigs

We will first begin by making a linear model of Active rigs versus all of the available exploratory variables.

Call:
lm(formula = Active_Rigs ~ ., data = OIL_MODIFIED_MODEL)

Residuals:
    Min      1Q  Median      3Q     Max 
-579.89  -94.98    5.04   84.42 1157.95 

Coefficients: (2 not defined because of singularities)
                            Estimate Std. Error t value Pr(>|t|)    
(Intercept)               -3.177e+05  1.599e+05  -1.987 0.047572 *  
Month                      4.381e+00  3.855e+00   1.136 0.256415    
Year                       1.653e+02  8.571e+01   1.928 0.054481 .  
US_Population             -4.804e+01  2.971e+01  -1.617 0.106593    
President_PartyRepublican -1.998e+02  3.275e+01  -6.100 2.36e-09 ***
SenateRepublican          -1.155e+01  6.765e+01  -0.171 0.864463    
HouseRepublican            3.164e+00  8.371e+01   0.038 0.969867    
life_exp                   4.052e+01  6.910e+01   0.586 0.557910    
GDP                       -5.869e+01  7.249e+01  -0.810 0.418605    
Oil_Production            -1.978e-01  4.270e-02  -4.632 4.81e-06 ***
Oil_Stock                 -1.675e+00  4.538e-01  -3.692 0.000251 ***
Oil_Price                  3.687e+00  1.467e+00   2.513 0.012323 *  
HDD                        1.562e-01  1.464e-01   1.068 0.286327    
CDD                       -9.085e-02  3.598e-01  -0.252 0.800781    
Energy_Consumption        -2.245e+01  5.694e+01  -0.394 0.693599    
Net_imports_Energy        -6.835e+01  8.183e+01  -0.835 0.404064    
Oil_Reserves               5.680e-02  1.414e-02   4.017 6.95e-05 ***
Active_Conflicts           9.646e-01  9.038e-01   1.067 0.286436    
Drilled_Wells              4.099e-01  1.524e-02  26.892  < 2e-16 ***
Eco_RecRecession           1.725e+02  3.082e+01   5.597 3.89e-08 ***
SeasonSpring               6.016e+01  3.678e+01   1.636 0.102612    
SeasonSummer               1.551e+01  5.406e+01   0.287 0.774318    
SeasonWinter               5.255e+01  4.964e+01   1.059 0.290349    
Gov_ctrl_typeTotal         1.494e+01  3.157e+01   0.473 0.636218    
Gov_ctrl_partyRepublican          NA         NA      NA       NA    
Gov_ctrl_partyshared              NA         NA      NA       NA    
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 202.6 on 432 degrees of freedom
  (96 observations deleted due to missingness)
Multiple R-squared:  0.9352,    Adjusted R-squared:  0.9318 
F-statistic: 271.2 on 23 and 432 DF,  p-value: < 2.2e-16

Analysis of Variance Table

Response: Active_Rigs
                    Df   Sum Sq  Mean Sq   F value    Pr(>F)    
Month                1   738085   738085   17.9807 2.731e-05 ***
Year                 1 45785358 45785358 1115.3882 < 2.2e-16 ***
US_Population        1 13504188 13504188  328.9788 < 2.2e-16 ***
President_Party      1  4453281  4453281  108.4875 < 2.2e-16 ***
Senate               1  5932278  5932278  144.5177 < 2.2e-16 ***
House                1 81501703 81501703 1985.4827 < 2.2e-16 ***
life_exp             1 34160309 34160309  832.1875 < 2.2e-16 ***
GDP                  1  8000118  8000118  194.8928 < 2.2e-16 ***
Oil_Production       1   469909   469909   11.4476 0.0007812 ***
Oil_Stock            1  4691129  4691129  114.2817 < 2.2e-16 ***
Oil_Price            1 11911533 11911533  290.1797 < 2.2e-16 ***
HDD                  1  1046345  1046345   25.4903 6.567e-07 ***
CDD                  1    89693    89693    2.1850 0.1400869    
Energy_Consumption   1   333804   333804    8.1319 0.0045579 ** 
Net_imports_Energy   1     3174     3174    0.0773 0.7810762    
Oil_Reserves         1   480056   480056   11.6948 0.0006863 ***
Active_Conflicts     1   369798   369798    9.0087 0.0028427 ** 
Drilled_Wells        1 41125657 41125657 1001.8721 < 2.2e-16 ***
Eco_Rec              1  1328248  1328248   32.3578 2.367e-08 ***
Season               3   122802    40934    0.9972 0.3939847    
Gov_ctrl_type        1     9197     9197    0.2240 0.6362178    
Residuals          432 17733086    41049                        
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

A linear model of all variables included, give us an adjusted coefficient of determination of 0.9318, and an RMSE value of 203 Rigs. The ANOVA test of the coefficients tell us that the variables which are non significant, at an alpha level of 5%, for this model are:

• CDD
• Net_imports_Energy
• Season
• Gov_ctrl_type
• Gov_ctrl_party

To evaluate with more detail the generated model we should look at the residuals to evaluate their normality and constant variability criteria

Residuals plot, show us that they are almost centered and scatered around 0, except for high levels of active rigs, they also shown certain levels of “heteroscedasticity”, meaning that the residuals get larger as the prediction moves from small to large. This is something we need to keep in mind for further evaluation of variables transformation. The qqplot show us a similar story than the residuals plot.

Multicolinearity Evaluation

Before going any further we should evaluate multicolinearity on our model, the correlation plots already gave us a hint of which variables were highly correlated. We will first evaluate if within our datasets we have aliased variables, which means if weather or not we have perfect linear combinations of variables.

Model :
Oil_Price ~ Month + Year + Active_Rigs + US_Population + President_Party + 
    Senate + House + life_exp + GDP + Oil_Production + Oil_Stock + 
    HDD + CDD + Energy_Consumption + Net_imports_Energy + Oil_Reserves + 
    Active_Conflicts + Drilled_Wells + Eco_Rec + Season + Gov_ctrl_type + 
    Gov_ctrl_party

Complete :
                         (Intercept) Month Year Active_Rigs US_Population
Gov_ctrl_partyRepublican -1/2           0     0    0           0         
Gov_ctrl_partyshared        1           0     0    0           0         
                         President_PartyRepublican SenateRepublican
Gov_ctrl_partyRepublican  1/2                         0            
Gov_ctrl_partyshared        0                         0            
                         HouseRepublican life_exp GDP  Oil_Production
Gov_ctrl_partyRepublican  1/2               0        0    0          
Gov_ctrl_partyshared        0               0        0    0          
                         Oil_Stock HDD  CDD  Energy_Consumption
Gov_ctrl_partyRepublican    0         0    0    0              
Gov_ctrl_partyshared        0         0    0    0              
                         Net_imports_Energy Oil_Reserves Active_Conflicts
Gov_ctrl_partyRepublican    0                  0            0            
Gov_ctrl_partyshared        0                  0            0            
                         Drilled_Wells Eco_RecRecession SeasonSpring
Gov_ctrl_partyRepublican    0             0                0        
Gov_ctrl_partyshared        0             0                0        
                         SeasonSummer SeasonWinter Gov_ctrl_typeTotal
Gov_ctrl_partyRepublican    0            0          1/2              
Gov_ctrl_partyshared        0            0           -1              

Model :
Active_Rigs ~ Month + Year + US_Population + President_Party + 
    Senate + House + life_exp + GDP + Oil_Production + Oil_Stock + 
    Oil_Price + HDD + CDD + Energy_Consumption + Net_imports_Energy + 
    Oil_Reserves + Active_Conflicts + Drilled_Wells + Eco_Rec + 
    Season + Gov_ctrl_type + Gov_ctrl_party

Complete :
                         (Intercept) Month Year US_Population
Gov_ctrl_partyRepublican -1/2           0     0    0         
Gov_ctrl_partyshared        1           0     0    0         
                         President_PartyRepublican SenateRepublican
Gov_ctrl_partyRepublican  1/2                         0            
Gov_ctrl_partyshared        0                         0            
                         HouseRepublican life_exp GDP  Oil_Production
Gov_ctrl_partyRepublican  1/2               0        0    0          
Gov_ctrl_partyshared        0               0        0    0          
                         Oil_Stock Oil_Price HDD  CDD  Energy_Consumption
Gov_ctrl_partyRepublican    0         0         0    0    0              
Gov_ctrl_partyshared        0         0         0    0    0              
                         Net_imports_Energy Oil_Reserves Active_Conflicts
Gov_ctrl_partyRepublican    0                  0            0            
Gov_ctrl_partyshared        0                  0            0            
                         Drilled_Wells Eco_RecRecession SeasonSpring
Gov_ctrl_partyRepublican    0             0                0        
Gov_ctrl_partyshared        0             0                0        
                         SeasonSummer SeasonWinter Gov_ctrl_typeTotal
Gov_ctrl_partyRepublican    0            0          1/2              
Gov_ctrl_partyshared        0            0           -1              

as we can see Gov_ctrl_type and Gov_ctrl_party are aliased with other political variables so we will drop them from the dataset, and then generate a VIF matrix to look for other colinear explanatory variables

Variables that are highly correlated that high a VIF value bigger than 5 are:

• Year
• US_Population
• life_exp
• GDP
• Oil_Production
• Energy_Consumption
• Oil_Reserves

Before deciding to remove them from the models, we will evaluate the more advance feature selecion algorithms

StepAIC akaike information criterion Oil Price model

We will run the AIC algorithm backwards and see which variables the algorithm decides to drop

Stepwise Model Path 
Analysis of Deviance Table

Initial Model:
Oil_Price ~ Month + Year + Active_Rigs + US_Population + President_Party + 
    Senate + House + life_exp + GDP + Oil_Production + Oil_Stock + 
    HDD + CDD + Energy_Consumption + Net_imports_Energy + Oil_Reserves + 
    Active_Conflicts + Drilled_Wells + Eco_Rec + Season

Final Model:
Oil_Price ~ Month + Year + Active_Rigs + US_Population + President_Party + 
    Senate + House + life_exp + GDP + Oil_Production + Oil_Stock + 
    CDD + Energy_Consumption + Net_imports_Energy + Oil_Reserves + 
    Active_Conflicts + Drilled_Wells + Season

       Step Df Deviance Resid. Df Resid. Dev      AIC
1                             433   18886.24 1744.005
2     - HDD  1 1.773351       434   18888.01 1742.048
3 - Eco_Rec  1 5.320678       435   18893.33 1740.177

The AIC algorithm decides to drop the following variables:

• HDD
• Eco_Rec (Economical Recession)

StepAIC akaike information criterion Active Rigs model

We will run the AIC algorithm backwards and see which variables the algorithm decides to drop

Stepwise Model Path 
Analysis of Deviance Table

Initial Model:
Active_Rigs ~ Month + Year + US_Population + President_Party + 
    Senate + House + life_exp + GDP + Oil_Production + Oil_Stock + 
    Oil_Price + HDD + CDD + Energy_Consumption + Net_imports_Energy + 
    Oil_Reserves + Active_Conflicts + Drilled_Wells + Eco_Rec + 
    Season

Final Model:
Active_Rigs ~ Year + US_Population + President_Party + Oil_Production + 
    Oil_Stock + Oil_Price + HDD + CDD + Net_imports_Energy + 
    Oil_Reserves + Active_Conflicts + Drilled_Wells + Eco_Rec

                  Step Df    Deviance Resid. Df Resid. Dev      AIC
1                                           433   17742283 4865.450
2             - Season  3 122801.6609       436   17865085 4862.595
3              - House  1    262.9605       437   17865348 4860.602
4              - Month  1    638.7724       438   17865986 4858.618
5 - Energy_Consumption  1   2877.4922       439   17868864 4856.691
6             - Senate  1   9271.8029       440   17878136 4854.928
7           - life_exp  1  32495.7768       441   17910631 4853.756
8                - GDP  1  38134.0117       442   17948765 4852.726

The AIC algorithm decides to drop the following variables:

• Season
• House
• Month
• Energy_Consumption
• Senate
• life_exp
• GDP

BIC Bayesian information criterion Oil Price model

We will run the BIC algorithm and see which variables has the highest likehood for being included

BIC is considerably more strict on the penalization of the number of terms in comparisson with AIC, the variables that are not included in the most likely model are:

• Month
• Active_Rigs
• Senate
• Oil_Production
• HDD
• CDD
• Energy_Consumption
• Active_Conflicts
• Eco_Rec
• Season

BIC Bayesian information criterion Active Rigs model

We will run the BIC algorithm and see which variables has the highest likehood for being included

BIC is considerably more strict on the penalization of the number of terms in comparisson with AIC, the variables that are not included in the most likely model are:

• Month
• Senate
• House
• life_exp
• GDP
• CDD
• Energy consumption
• Net imports energy
• Active conflicts
• Season

Boruta Feature Selection Oil Price

We will run the Boruta feature selection algorithm and see which variables are included

Boruta performed 99 iterations in 21.89947 secs.
 17 attributes confirmed important: Active_Conflicts, Active_Rigs,
CDD, Eco_Rec, Energy_Consumption and 12 more;
 1 attributes confirmed unimportant: Season;
 1 tentative attributes left: HDD;

             Month               Year        Active_Rigs 
         Confirmed          Confirmed          Confirmed 
     US_Population    President_Party             Senate 
         Confirmed          Confirmed          Confirmed 
             House           life_exp                GDP 
         Confirmed          Confirmed          Confirmed 
    Oil_Production          Oil_Stock                HDD 
         Confirmed          Confirmed          Tentative 
               CDD Energy_Consumption Net_imports_Energy 
         Confirmed          Confirmed          Confirmed 
      Oil_Reserves   Active_Conflicts            Eco_Rec 
         Confirmed          Confirmed          Confirmed 
            Season 
          Rejected 
Levels: Tentative Confirmed Rejected

Only one variable was dropped by borutas final decission:

• Season

interestigly the variables which are given the highest importance are economic variables like GDP and population

Boruta Feature Selection Active Rigs

Boruta performed 99 iterations in 21.22427 secs.
 16 attributes confirmed important: Active_Conflicts, Eco_Rec,
Energy_Consumption, GDP, House and 11 more;
 No attributes deemed unimportant.
 3 tentative attributes left: CDD, HDD, Season;

             Month               Year      US_Population 
         Confirmed          Confirmed          Confirmed 
   President_Party             Senate              House 
         Confirmed          Confirmed          Confirmed 
          life_exp                GDP     Oil_Production 
         Confirmed          Confirmed          Confirmed 
         Oil_Stock          Oil_Price                HDD 
         Confirmed          Confirmed          Tentative 
               CDD Energy_Consumption Net_imports_Energy 
         Tentative          Confirmed          Confirmed 
      Oil_Reserves   Active_Conflicts            Eco_Rec 
         Confirmed          Confirmed          Confirmed 
            Season 
         Tentative 
Levels: Tentative Confirmed Rejected

Threee variables were classified as tentative by borutas final decision:

• CDD
• HDD
• Season