Evaluate Multilinear Regression, Artificial Neural Network and Deep Learning Model to Predicting Crude Protein and Fibre From Physical Properties of Feedstuffs

Aims

The aim of this research was to build and compare multilinear regression, artificial neural network and deep learning model to predict crude protein and fibre content from physical properties of feedstuffs. The 91 data were obtained from https://repository.ipb.ac.id using keywords (e.g. sifat fisik and pakan). To reduce the dimensional of the data had been transformed. The independent variables consist of X1: specific gravity, X2: bulk density, X3: compacted bulk density and X4: angle of repose. The dependent variable was Y1: crude protein and Y2: crude fibre. Multilinear regression (MLR), artificial neural network (ANN) and deep learning (DLr) model built by R programing language 3.5.3 using library R-base, neuralnet and keras. Mean square error (MSE) used to evaluate the model, lower MSE within two models means better than others. Mean square error of crude protein MLR model was higher than ANN model, that were 0.0143 and 0.0093. Besides that, MSE of crude fibre MLR model was lower than ANN model, 0.0605 and 0.0722. Furthermore, the model built using DLr can improve ANN model, that has MSE 0.0249 and 0.0449, respectively, for crude protein and crude fibre deep learning model. The deep learning model generally can perform better to predict crude protein and fibre from physical properties than multilinear regression and artificial neural network.

Data

BHN	BJ	KT	KPT	ST	PK	SK
Bungkil_inti_sawit	1.31300	0.53864	0.72817	34.586	12.862160	14.387621
Bungkil_inti_sawit	1.12300	0.27824	0.49290	46.154	25.798583	11.244047
Bungkil_inti_sawit	1.15500	0.32768	0.52363	42.410	18.995127	18.971919
Bungkil_inti_sawit	1.24000	0.43106	0.66184	36.273	17.981610	23.930072
Bungkil_kelapa	1.27700	0.51022	0.67802	35.473	14.047751	9.627985
Bungkil_kelapa	1.07100	0.27274	0.45641	48.175	19.204623	13.539542
Bungkil_kelapa	1.14200	0.30943	0.49751	43.268	20.639249	18.752045
Bungkil_kelapa	1.23200	0.39648	0.61905	36.551	17.675439	27.532895
Pollard	1.29000	0.32086	0.43351	42.320	22.321942	9.780232
Pollard	1.28000	0.35067	0.52001	44.350	18.058862	7.931184
Pollard	1.12000	0.33839	0.49383	41.810	20.683434	9.222116
Pollard	1.42000	0.38647	0.53728	46.980	22.664080	6.476991
Dedak_padi	1.29000	0.35295	0.50935	40.530	13.008494	32.565936
Dedak_padi	1.50000	0.39409	0.52633	46.010	13.584906	23.234501
Dedak_padi	1.25000	0.28103	0.48415	39.800	8.257373	36.439678
Dedak_padi	1.36000	0.35552	0.50341	41.360	8.794509	36.304161
Jagung	1.34000	0.71732	0.74388	26.110	9.517632	4.693950
Onggok:Indigofera	0.96000	0.25616	0.32640	33.520	9.432169	9.895462
Onggok:Singkong	0.90000	0.24376	0.31524	32.780	8.911597	11.299905
Onggok:Indigofera:Singkong	0.92000	0.25385	0.32135	33.360	9.164388	10.590752
Dedak_padi	1.39000	0.34087	0.51680	35.580	10.966584	29.423638
Dedak_padi	1.27000	0.32301	0.48933	34.300	12.267533	29.621331
Dedak_padi	1.37000	0.33888	0.51573	35.430	9.047183	37.803481
Dedak_padi	1.33000	0.33724	0.51371	35.040	9.229544	34.186867
Dedak_padi	1.26000	0.32205	0.48901	34.290	10.223027	23.921657
Dedak_padi	1.30000	0.32844	0.49804	34.370	10.793076	29.992081
Dedak_padi	1.28000	0.32521	0.49520	34.360	11.509155	27.214830
Dedak_padi	1.31000	0.32905	0.49908	34.890	11.734924	29.129163
Dedak_padi	1.35000	0.30292	0.46300	36.960	13.371819	19.804430
Dedak_padi	1.35000	0.31843	0.46956	37.730	11.853069	19.575167
Dedak_padi	1.36000	0.32050	0.47244	38.120	12.943753	22.637160
Dedak_padi	1.35000	0.31457	0.46684	37.010	11.900595	20.884378
Dedak_padi	1.28000	0.28252	0.41348	36.340	12.080378	12.695035
Dedak_padi	1.36000	0.32633	0.47748	38.490	13.491405	16.787196
Dedak_padi	1.28000	0.29847	0.42710	36.400	12.925328	25.084195
Dedak_padi	1.29000	0.29245	0.41533	36.830	13.076923	18.601399
Dedak_padi	1.36000	0.35310	0.52130	40.630	12.781929	20.588768
Dedak_padi	1.41000	0.36177	0.53875	41.520	13.043627	20.712654
Dedak_padi	1.36000	0.35127	0.52038	41.900	13.028046	10.089608
Dedak_padi	1.45000	0.36472	0.54077	42.860	13.317145	12.297267
Dedak_padi	1.21000	0.27814	0.44089	44.840	14.717476	13.302575
Dedak_padi	1.23000	0.30783	0.45134	42.180	14.110919	10.239168
Dedak_padi	1.20000	0.26108	0.43532	44.380	13.980330	9.508252
Dedak_padi	1.35000	0.35050	0.51699	40.900	14.140224	21.257583
Dedak_padi	1.35000	0.33005	0.50122	43.090	13.244824	13.847234
Dedak_padi	1.25000	0.27543	0.43256	44.510	10.170000	13.800000
Dedak_padi	1.23000	0.23912	0.39502	43.490	9.150000	16.150000
Dedak_padi	1.25000	0.27412	0.42913	44.410	10.070000	13.870000
Dedak_padi	1.25000	0.27521	0.43220	44.440	10.120000	13.890000
Dedak_padi	1.22000	0.24206	0.39744	42.370	9.320000	15.970000
Dedak_padi	1.23000	0.28317	0.42868	43.380	12.460000	10.670000
Dedak_padi	1.22000	0.26018	0.40356	43.090	11.160000	13.060000
Dedak_padi	1.23000	0.28355	0.42580	44.120	12.050000	10.480000
Dedak_padi	1.24000	0.31884	0.46528	45.240	13.900000	8.690000
Dedak_padi	1.24000	0.30648	0.45228	44.960	13.010000	9.250000
Bungkil_kedelai	1.66000	0.61000	0.80000	26.330	50.417042	3.866096
Bungkil_kelapa	1.46000	0.48000	0.50000	33.220	24.188504	20.133722
Bungkil_inti_sawit	1.67000	0.33800	0.46300	35.080	19.004669	29.870543
Pollard	1.25000	0.30300	0.40300	35.840	15.153943	12.853382
Corn_gluten_meal	1.25000	0.28300	0.78000	24.330	68.026506	0.652309
Bungkil_kapuk	1.67000	0.28000	0.36300	34.190	39.058824	14.095238
Dedak_padi	1.36000	0.28653	0.47544	44.400	10.530000	15.420000
Tongkol_jagung	0.57000	0.20619	0.26667	36.540	4.370000	40.070000
Gamal	0.69000	0.33200	0.44800	35.180	20.040013	12.337446
Kaliandra	0.83000	0.37300	0.50000	33.220	23.115079	16.644621
Lamtoro	0.88000	0.33800	0.46300	35.000	19.578279	13.943997
Indigofera	0.72000	0.30300	0.40300	35.840	28.322465	10.310322
Trikantera	0.80000	0.28300	0.38500	35.040	24.120830	13.863841
Daun_pepaya	0.92000	0.36800	0.47700	36.680	24.302023	12.433593
Daun_singkong	0.61000	0.28000	0.36300	34.190	32.929136	12.682167
Bungkil_kelapa	0.96603	0.61750	0.74250	31.510	19.309161	15.018236
Bungkil_kelapa	1.06770	0.46000	0.60250	37.600	17.588216	17.588216
Bungkil_kedelai	1.13023	0.62250	0.74750	24.100	49.917718	6.856829
Bungkil_kedelai	1.29765	0.54250	0.69750	35.180	49.700666	6.777364
Bungkil_inti_sawit	1.20455	0.66500	0.82250	36.610	17.789072	23.178738
Bungkil_inti_sawit	1.15078	0.59000	0.79750	42.190	16.800760	25.910675
Ransum_Ayam	1.60000	0.49000	0.55000	33.900	24.167822	5.547850
Ransum_Ayam	1.47000	0.51000	0.59000	34.530	24.033402	5.376344
Ransum_Ayam	1.47000	0.51000	0.56000	35.070	24.694630	5.565799
Ransum_Ayam	1.45000	0.56000	0.63000	31.440	23.643065	5.092696
Ransum_Ayam	1.32000	0.56000	0.64000	31.460	23.502825	5.401130
Ransum_Ayam	1.38000	0.55000	0.64000	31.300	23.129404	5.223129
Ransum_Ayam	1.37000	0.72000	0.76000	27.770	22.202604	4.943721
Ransum_Ayam	1.35000	0.71000	0.76000	27.690	22.470172	4.396818
Ransum_Ayam	1.37000	0.68000	0.74000	27.700	22.229640	4.617267
Ransum_Ayam	1.30000	0.73000	0.78000	26.100	21.767666	4.630278
Ransum_Ayam	1.36000	0.72000	0.87000	25.510	21.470491	4.872107
Ransum_Ayam	1.36000	0.73000	0.79000	25.810	20.854908	4.285557
Ransum_Ayam_M	1.16000	0.49300	0.69900	33.200	24.333522	4.549064
Ransum_Ayam_C	1.19400	0.66000	0.76200	20.850	22.790411	4.357567
Ransum_Ayam_P	1.18000	0.62900	0.68800	14.940	24.627557	4.781282

Matrix correlation

Transformation of the data

# transformasi data
scale01 <- function(x){
  (x-min(x))/(max(x)-min(x))
}

dt <- dt %>%
  mutate_all(scale01)

Splitting data

## Joining, by = c("BJ", "KT", "KPT", "ST", "PK", "SK")

BJ	KT	KPT	ST	PK	SK
0.8454545	0.3587179	0.4303781	0.9348578	0.1447598	0.5728949
0.6000000	0.1469617	0.2685263	0.8557244	0.1270883	0.2541420
0.8000000	0.6754548	0.6022077	0.4964646	0.3027666	0.1126496
0.6909091	0.2501861	0.4094608	0.6047841	0.0763401	0.8507489
1.0000000	0.2516370	0.3254106	0.6059877	0.2299006	0.7412467
0.5768636	0.8759092	0.9212703	0.6520235	0.2108044	0.5714802
0.5027273	0.1375499	0.3749689	0.9391906	0.3366283	0.2687052
0.6636364	1.0000000	0.8508279	0.3357906	0.2733054	0.1009184
0.7636364	0.2970161	0.4509638	0.7997593	0.1362567	0.5089173
0.6454545	0.2758252	0.4199029	0.8849105	0.2150426	0.1846601
0.6818182	0.6754548	0.6187824	0.4970663	0.3005635	0.1204744
0.7090909	0.2069071	0.3317753	0.6640590	0.1183005	0.5132738
0.8000000	0.3026479	0.4543119	0.8400782	0.1405535	0.2954247
0.7181818	0.2769707	0.4205161	0.8111930	0.1360120	0.2394179
0.7272727	0.9809091	0.8176786	0.3860388	0.2801380	0.1088702
0.7090909	0.2142762	0.3362836	0.6857229	0.1175539	0.4800600
0.6754545	0.6346767	0.7649214	0.5911238	0.1334060	0.3484555
0.4524545	0.4845459	0.5566274	0.6818113	0.2076491	0.4296524
0.7727273	0.3441706	0.4485273	0.9640439	0.2873874	0.1477682
0.5318182	0.2319352	0.4259029	0.8265383	0.2297507	0.4647560
0.6545455	0.2189153	0.2765319	0.8238303	0.2820127	0.2315692
0.5545455	0.8071820	0.6983409	0.0000000	0.3182323	0.1047492
0.6545455	0.2801779	0.4022343	0.7699714	0.1357048	0.8096270
0.1090909	0.2401825	0.3005486	0.6089965	0.2461651	0.2964440
0.3181818	0.0909872	0.0906303	0.5542350	0.0753165	0.2521315
0.7181818	0.2850843	0.3923889	0.7949451	0.0695060	0.9044632
0.7181818	0.1533762	0.3460295	0.8864149	0.0967694	0.3746463
0.5363636	0.5475459	0.7165730	0.5494208	0.3136132	0.0988580
0.7181818	0.2293580	0.3494108	0.7085903	0.1432910	0.4093311
0.5909091	0.1030717	0.2268908	0.8469986	0.1066662	0.3147747
0.6181818	0.1848189	0.2259626	0.6288551	0.1694083	0.3095329
0.6181818	0.1466371	0.8508279	0.2825335	1.0000000	0.0000000
0.5818182	0.1373590	0.2887640	0.8996540	0.1625517	0.3209286
0.2090909	0.1466371	0.1961282	0.6047841	0.3102720	0.3351676
0.6272727	0.2211871	0.3685214	0.5822175	0.0919470	0.5903275
0.6545455	0.1646780	0.2463992	0.6586430	0.1367798	0.4553562
0.8181818	0.5800004	0.4861850	0.6056868	0.3192860	0.1246519
0.5000000	0.2523816	0.3765104	0.8084850	0.2562728	0.2174102
0.9363636	0.5418186	0.4696103	0.5704829	0.3110102	0.1241966
0.3600273	0.7852275	0.7886729	0.4985708	0.2346840	0.3644538
0.6427273	0.5804204	0.6817993	0.6178125	0.1520308	0.2277068
0.6181818	0.1428762	0.3604661	0.7480066	0.0610680	0.9079012
0.3000000	0.0717245	0.0805032	0.5367835	0.0713454	0.2701223
0.7000000	0.9757927	0.7909602	0.3360915	0.0808658	0.1025337
0.6181818	0.1296844	0.2692722	0.8867158	0.0895431	0.3353238
0.7181818	0.2804643	0.4220410	0.7729803	0.1321456	0.5057744
0.6000000	0.1940398	0.3060846	0.8196179	0.1530231	0.2432121
0.6615000	0.6420458	0.7140868	0.6089965	0.7121136	0.1553885
0.6181818	0.1317653	0.2743606	0.8876185	0.0903286	0.3358312
0.3181818	0.3089097	0.3486152	0.6541297	0.3131184	0.2988832
0.7181818	0.2182280	0.3410571	0.6974575	0.1346878	0.5577407
0.7090909	0.2755007	0.4148973	0.7811043	0.1534835	0.5227418
0.7363636	0.6563639	0.6187824	0.4922521	0.2946974	0.1159586
0.6454545	0.2272198	0.3787811	0.5843238	0.1121512	0.6738731
0.0000000	0.0000000	0.0000000	0.6499173	0.0000000	1.0000000
0.6454545	0.1761707	0.2659075	0.6457048	0.1343983	0.6198203
0.6363636	0.2230198	0.3690518	0.5825184	0.1240648	0.7349244
0.7272727	0.2533170	0.4128089	0.6165187	0.0734753	0.9425000
1.0000000	0.1409099	0.1596639	0.5792087	0.5449376	0.3410380
0.6636364	0.2333862	0.3834883	0.5846246	0.1009021	0.7443300
0.6181818	0.1321853	0.2749573	0.8897247	0.0911140	0.3335480
0.5200000	0.1970944	0.3826098	0.8523544	0.2555787	0.4591780
0.5909091	0.0684790	0.2167471	0.8253347	0.0777611	0.3885994
0.5672727	0.8663638	0.8209935	0.1778246	0.2893720	0.0939999
0.0363636	0.1409099	0.1596639	0.5792087	0.4486444	0.3051893
0.6454545	0.1457208	0.2433328	0.6438995	0.1211247	0.3055158
0.5093000	0.7947729	0.7969602	0.2756131	0.7155234	0.1574045
0.7181818	1.0000000	0.8674026	0.3270648	0.2589666	0.0921730
0.6090909	0.1914626	0.3076426	0.9032646	0.1357285	0.2181176
0.6727273	0.2345507	0.3852121	0.6002708	0.1156979	0.7224384
0.6090909	0.2150589	0.3291897	0.9116895	0.1497098	0.2039108
0.6090909	0.4292969	0.6549815	0.6418836	0.2138290	0.5905410
0.7454545	0.2571161	0.4145824	0.6210320	0.1036278	0.7299090

BJ	KT	KPT	ST	PK	SK
0.4554545	0.1270499	0.3144879	1.0000000	0.2330417	0.3269403
0.6018182	0.3632806	0.5840585	0.6502482	0.2090193	0.6819422
0.3545455	0.0953972	0.0990005	0.5590492	0.0795232	0.2344925
0.7090909	0.1846662	0.3254106	0.6625545	0.1414124	0.4858763
0.5727273	0.1047899	0.2795319	0.8858131	0.1509717	0.2246693
0.7090909	0.2364598	0.3887591	0.8469986	0.1394174	0.3347463
0.6000000	0.0628663	0.2127360	0.8590342	0.0750905	0.3931659
0.6000000	0.1476871	0.2637528	0.8779901	0.1206475	0.2493218
0.9909091	0.7709093	0.8839773	0.3427110	0.7233674	0.0815316
0.8090909	0.5227277	0.3867369	0.5500226	0.3113351	0.4942302
0.2363636	0.3184552	0.3867369	0.5500226	0.2944723	0.4057141
0.2818182	0.2516370	0.3254106	0.6035806	0.2389116	0.3372011
0.1363636	0.1848189	0.2259626	0.6288551	0.3762768	0.2450172
0.5279818	0.7327275	0.8798336	0.8199188	0.1952787	0.6407876
0.8181818	0.5800004	0.5359090	0.5894388	0.3088985	0.1198456
0.7090909	0.9618182	0.8176786	0.3836317	0.2843413	0.0949957
0.7272727	0.9045455	0.7845292	0.3839326	0.2805627	0.1005883
0.7181818	0.9809091	1.0000000	0.3180382	0.2686370	0.1070534

Build ANN and MLR model

Crude Protein (PK)

## 
## Call:
## lm(formula = PK ~ BJ + KT + KPT + ST, data = dt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.20222 -0.07987 -0.03389  0.04428  0.43238 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  0.34046    0.08743   3.894 0.000194 ***
## BJ          -0.03132    0.07990  -0.392 0.696049    
## KT          -0.41166    0.13341  -3.086 0.002731 ** 
## KPT          0.61093    0.14971   4.081 0.000100 ***
## ST          -0.34804    0.09981  -3.487 0.000771 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.1327 on 86 degrees of freedom
## Multiple R-squared:  0.331,  Adjusted R-squared:  0.2998 
## F-statistic: 10.64 on 4 and 86 DF,  p-value: 4.757e-07

## [1] 0.02381218

## 
## Call:
## lm(formula = SK ~ BJ + KT + KPT + ST, data = dt)
## 
## Residuals:
##      Min       1Q   Median       3Q      Max 
## -0.56274 -0.14468 -0.06268  0.11251  0.58337 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)   
## (Intercept)  0.42969    0.14176   3.031  0.00322 **
## BJ           0.09736    0.12955   0.752  0.45437   
## KT          -0.51533    0.21629  -2.383  0.01940 * 
## KPT          0.18113    0.24273   0.746  0.45757   
## ST          -0.02010    0.16181  -0.124  0.90144   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 0.2152 on 86 degrees of freedom
## Multiple R-squared:  0.1851, Adjusted R-squared:  0.1472 
## F-statistic: 4.884 on 4 and 86 DF,  p-value: 0.001347

## [1] 0.0370491

Crude Protein (SK)

## [1] 0.03611256

## [1] 0.03718827

Mean Square Error

Model	Mean_Square_Error
mlr_PK	0.0238122
ann_PK	0.0361126
mlr_SK	0.0370491
ann_SK	0.0371883

Improve ANN Models

## ___________________________________________________________________________
## Layer (type)                     Output Shape                  Param #     
## ===========================================================================
## dense (Dense)                    (None, 100)                   500         
## ___________________________________________________________________________
## dense_1 (Dense)                  (None, 10000)                 1010000     
## ___________________________________________________________________________
## dense_2 (Dense)                  (None, 1)                     10001       
## ===========================================================================
## Total params: 1,020,501
## Trainable params: 1,020,501
## Non-trainable params: 0
## ___________________________________________________________________________

## ___________________________________________________________________________
## Layer (type)                     Output Shape                  Param #     
## ===========================================================================
## dense_3 (Dense)                  (None, 100)                   500         
## ___________________________________________________________________________
## dense_4 (Dense)                  (None, 10000)                 1010000     
## ___________________________________________________________________________
## dense_5 (Dense)                  (None, 1)                     10001       
## ===========================================================================
## Total params: 1,020,501
## Trainable params: 1,020,501
## Non-trainable params: 0
## ___________________________________________________________________________