MXNet LogisticRegressionOutputが上手く行かなかったtest（20160601追記で成功）

https://twitter.com/TJO_datasci/status/737588070225043457
この話があったので試してみました。

データ読み込み

library(mxnet)
data(Sonar, package="mlbench")
Sonar[,61] = as.numeric(Sonar[,61])-1
train.ind = c(1:50, 100:150)
train.x = data.matrix(Sonar[train.ind, 1:60])
train.y = Sonar[train.ind, 61]
test.x = data.matrix(Sonar[-train.ind, 1:60])
test.y = Sonar[-train.ind, 61]

まずは1層で動かしてみる

data <- mx.symbol.Variable("data")
fc1 <- mx.symbol.FullyConnected(data, num_hidden=1)
lro <- mx.symbol.LogisticRegressionOutput(fc1,name='lr')
mx.set.seed(0)
model <- mx.model.FeedForward.create(lro, X=train.x, y=train.y,
                                     ctx=mx.cpu(), num.round=20, array.batch.size=20,
                                     learning.rate=0.07, momentum=0.9, eval.metric=mx.metric.accuracy)

 mx.model.select.layout.train(X, y) で警告がありました: 
  Auto detect layout of input matrix, use rowmajor..

Start training with 1 devices
[1] Train-accuracy=0.52
[2] Train-accuracy=0.508333333333333
[3] Train-accuracy=0.508333333333333
[4] Train-accuracy=0.508333333333333
[5] Train-accuracy=0.508333333333333
[6] Train-accuracy=0.508333333333333
[7] Train-accuracy=0.508333333333333
[8] Train-accuracy=0.508333333333333
[9] Train-accuracy=0.508333333333333
[10] Train-accuracy=0.508333333333333
[11] Train-accuracy=0.508333333333333
[12] Train-accuracy=0.508333333333333
[13] Train-accuracy=0.508333333333333
[14] Train-accuracy=0.508333333333333
[15] Train-accuracy=0.508333333333333
[16] Train-accuracy=0.508333333333333
[17] Train-accuracy=0.508333333333333
[18] Train-accuracy=0.508333333333333
[19] Train-accuracy=0.508333333333333
[20] Train-accuracy=0.508333333333333

動いたがaccuracy上がらず。

多層なら？

data <- mx.symbol.Variable("data")
fc1 <-   mx.symbol.FullyConnected(data,  name="fc1", num_hidden=128) 
act1    <- mx.symbol.Activation(fc1, name="tanh1",   act_type="tanh") 
drop1   <-   mx.symbol.Dropout(act1, p = 0.1,    name="drop1") 
fc2 <-   mx.symbol.FullyConnected(drop1, name="fc2", num_hidden=64) 
act2    <-   mx.symbol.Activation(fc2,   name="tanh2",   act_type="tanh") 
drop2   <- mx.symbol.Dropout(act2,   p   =   0.1,    name="drop2") 
fc3 <-   mx.symbol.FullyConnected(drop2, name="fc3", num_hidden=1)
lro2<- mx.symbol.LogisticRegressionOutput(fc3)
mx.set.seed(0)
model <- mx.model.FeedForward.create(lro2, X=train.x, y=train.y,
                                     ctx=mx.cpu(), num.round=20, array.batch.size=20,
                                     learning.rate=0.07, momentum=0.9, eval.metric=mx.metric.accuracy)

 mx.model.select.layout.train(X, y) で警告がありました: 
  Auto detect layout of input matrix, use rowmajor..

Start training with 1 devices
[1] Train-accuracy=0.52
[2] Train-accuracy=0.508333333333333
[3] Train-accuracy=0.508333333333333
[4] Train-accuracy=0.508333333333333
[5] Train-accuracy=0.508333333333333
[6] Train-accuracy=0.508333333333333
[7] Train-accuracy=0.508333333333333
[8] Train-accuracy=0.508333333333333
[9] Train-accuracy=0.508333333333333
[10] Train-accuracy=0.508333333333333
[11] Train-accuracy=0.508333333333333
[12] Train-accuracy=0.508333333333333
[13] Train-accuracy=0.508333333333333
[14] Train-accuracy=0.508333333333333
[15] Train-accuracy=0.508333333333333
[16] Train-accuracy=0.508333333333333
[17] Train-accuracy=0.508333333333333
[18] Train-accuracy=0.508333333333333
[19] Train-accuracy=0.508333333333333
[20] Train-accuracy=0.508333333333333

上手く回らない。

mx.mlpからout_activation=“logistic”を呼んでみる

mx.set.seed(0)
model <- mx.mlp(train.x, train.y, hidden_node=10, out_node=1, out_activation="logistic",
                num.round=20, array.batch.size=15, learning.rate=0.07, momentum=0.9, 
                eval.metric=mx.metric.accuracy)

 mx.model.select.layout.train(X, y) で警告がありました: 
  Auto detect layout of input matrix, use rowmajor..

Start training with 1 devices
[1] Train-accuracy=0.533333333333333
[2] Train-accuracy=0.514285714285714
[3] Train-accuracy=0.514285714285714
[4] Train-accuracy=0.514285714285714
[5] Train-accuracy=0.514285714285714
[6] Train-accuracy=0.514285714285714
[7] Train-accuracy=0.514285714285714
[8] Train-accuracy=0.514285714285714
[9] Train-accuracy=0.514285714285714
[10] Train-accuracy=0.514285714285714
[11] Train-accuracy=0.514285714285714
[12] Train-accuracy=0.514285714285714
[13] Train-accuracy=0.514285714285714
[14] Train-accuracy=0.514285714285714
[15] Train-accuracy=0.514285714285714
[16] Train-accuracy=0.514285714285714
[17] Train-accuracy=0.514285714285714
[18] Train-accuracy=0.514285714285714
[19] Train-accuracy=0.514285714285714
[20] Train-accuracy=0.514285714285714

この方法でも全然だめ。

mx.mlpからout_activation=“softmax”を呼んでみる

mx.set.seed(0)
model <- mx.mlp(train.x, train.y, hidden_node=10, out_node=2, out_activation="softmax",
                num.round=20, array.batch.size=15, learning.rate=0.07, momentum=0.9, 
                eval.metric=mx.metric.accuracy)

 mx.model.select.layout.train(X, y) で警告がありました: 
  Auto detect layout of input matrix, use rowmajor..

Start training with 1 devices
[1] Train-accuracy=0.488888888888889
[2] Train-accuracy=0.514285714285714
[3] Train-accuracy=0.514285714285714
[4] Train-accuracy=0.514285714285714
[5] Train-accuracy=0.514285714285714
[6] Train-accuracy=0.523809523809524
[7] Train-accuracy=0.619047619047619
[8] Train-accuracy=0.695238095238095
[9] Train-accuracy=0.695238095238095
[10] Train-accuracy=0.761904761904762
[11] Train-accuracy=0.828571428571429
[12] Train-accuracy=0.771428571428571
[13] Train-accuracy=0.742857142857143
[14] Train-accuracy=0.733333333333333
[15] Train-accuracy=0.771428571428571
[16] Train-accuracy=0.847619047619048
[17] Train-accuracy=0.857142857142857
[18] Train-accuracy=0.838095238095238
[19] Train-accuracy=0.838095238095238
[20] Train-accuracy=0.838095238095238

softmaxならうまくいく、謎。

色々やってみたがだめ。
“out_node”とか“num_hidden”とか1以外にしたら動かないし全然わからん。
Weird LogisticRegressionOutput Bugにもnum_hiddenは1にしろとあったので間違いではないと思うが…。
trainのところをfactorにしてみたりもしたがaccuraryが0になって草が生えた。
結局あんまり役に立たないまとめになってしまいました。すいません。

20160601追記

https://twitter.com/TJO_datasci/status/737695141645803520
eval_metricに明示的にmx.MSE()を与えてやると解決するという話をTJOさんが見つけてくださったので試してみます。

data <- mx.symbol.Variable("data")
fc1 <- mx.symbol.FullyConnected(data, num_hidden=1)
lro <- mx.symbol.LogisticRegressionOutput(fc1,name='lr')
mx.set.seed(0)
model <- mx.model.FeedForward.create(lro, X=train.x, y=train.y,
                                     ctx=mx.cpu(), num.round=20, array.batch.size=20,
                                     learning.rate=0.07, momentum=0.9, eval.metric = mx.metric.rmse)

 mx.model.select.layout.train(X, y) で警告がありました: 
  Auto detect layout of input matrix, use rowmajor..

Start training with 1 devices
[1] Train-rmse=0.504584959621227
[2] Train-rmse=0.486834490415004
[3] Train-rmse=0.46832481273427
[4] Train-rmse=0.450686512770706
[5] Train-rmse=0.437911443769791
[6] Train-rmse=0.428250970255103
[7] Train-rmse=0.418913606529651
[8] Train-rmse=0.411397469249855
[9] Train-rmse=0.404743247435068
[10] Train-rmse=0.39870206817925
[11] Train-rmse=0.393734536261962
[12] Train-rmse=0.389228543948827
[13] Train-rmse=0.385115866876684
[14] Train-rmse=0.381506997094149
[15] Train-rmse=0.378147166001399
[16] Train-rmse=0.375042510549718
[17] Train-rmse=0.372218810990555
[18] Train-rmse=0.36957206621295
[19] Train-rmse=0.367103434140826
[20] Train-rmse=0.36481225805133

Rの場合だとeval.metric = mx.metric.rmseで行けるみたいですね。
勉強になりました。ありがとうございます。

参考
Neural Network with MXNet in Five Minutes
Weird LogisticRegressionOutput Bug
No learning using LogisticRegressionOutput. Mistaken net definition?