ASAI 2020 Results
- Results for CTU13:
- LSTM Woodbridge + 256 batchs + Masking after 10 executions + maxlen from 10 to 1000 + first 4 symbols removed
- RF + maxlen from 10 to 1000 + first 4 symbols removed
- LSTM vs RF
- RF trained with random sequences of maxlen=10
- LSTM trained with random sequences of maxlen=10
- LSTM vs RF random subsquence maxlen=10 using dataset with sequences >75
- LSTM vs RF random subsquence maxlen=10,25,75 using full dataset
- Results for CTU19:
Results for CTU13:
LSTM Woodbridge + 256 batchs + Masking after 10 executions + maxlen from 10 to 1000 + first 4 symbols removed
[1] "80"
RF + maxlen from 10 to 1000 + first 4 symbols removed
LSTM vs RF
RF trained with random sequences of maxlen=10
Dataset ctu13subs.csv: ctu13.csv filtering sequences >75 and extracting subsquences of lenght=10 with random start
LSTM trained with random sequences of maxlen=10
LSTM Woodbridge + 256 batchs + Masking after 10 executions + maxlen from 10 to 1000 + first 4 symbols removed Dataset ctu13subs.csv: ctu13.csv ~625 sequences. Filtering sequences >75 and extracting subsquences of lenght=10 with random start
[1] "80"LSTM vs RF random subsquence maxlen=10 using dataset with sequences >75
LSTM vs RF random subsquence maxlen=10,25,75 using full dataset
Results for CTU19:
LSTM Woodbridge + 256 batchs + Masking after 10 executions + maxlen from 10 to 1000 + first 4 symbols removed
[1] "80"
RF + maxlen from 10 to 1000 + first 4 symbols removed
LSTM vs RF
LS0tCnRpdGxlOiAiQVNBSSAyMDIwIFJlc3VsdHMiCm91dHB1dDogCiAgaHRtbF9ub3RlYm9vazogCiAgICBjb2RlX2ZvbGRpbmc6IG5vbmUKICAgIGZpZ19oZWlnaHQ6IDYKICAgIGZpZ193aWR0aDogMTQKICAgIHRvYzogeWVzCi0tLQoKYGBge3IgbWVzc2FnZT1GQUxTRSwgd2FybmluZz1GQUxTRSwgaW5jbHVkZT1GQUxTRX0KbGlicmFyeShzdHJpbmdyKQpsaWJyYXJ5KGRwbHlyKQpsaWJyYXJ5KHRpZHlyKQpsaWJyYXJ5KHRpYmJsZSkKbGlicmFyeShnZ3Bsb3QyKQpsaWJyYXJ5KHJlYWRyKQpsaWJyYXJ5KGdnZGFyaykKYGBgCgoKYGBge3IgZXZhbD1GQUxTRSwgaW5jbHVkZT1GQUxTRX0KbGlicmFyeShnZ3Bsb3QyKQpsaWJyYXJ5KGRwbHlyKQpyZXN1bHRwZXJjbGFzczwtcmVhZF9jc3YoInJlc3VsdHMvcmVzdWx0c19wZXJfc3ViY2xhc3NfdGVzdF9ray5jc3YiKQoKZGdhX3Jlc3VsdHNfcGxvdDwtZ2dwbG90KHJlc3VsdHBlcmNsYXNzICAlPiUgZmlsdGVyKGdyZXBsKCdOb3JtYWwnLGxhYmVsKSkgJT4lIG11dGF0ZShsYWJlbD1nc3ViKCJCb3RuZXRcXC4iLCIiLGxhYmVsKSkKICAgICAgICAgICAgICAgICAgICAgICAgICxhZXMoeD1sYWJlbCx5PXJlY2FsbCkpKwogIGdlb21fcG9pbnQoYWVzKHNpemU9c3VwcG9ydCxmaWxsPSdza3libHVlJyksY29sb3I9J2JsYWNrJyxzaGFwZSA9IDIxLGFscGhhPTAuNSkrCiAgZ2VvbV9wb2ludChzaXplPTEsY29sb3I9J2JsdWUnLGZpbGw9J2JsdWUnLHNoYXBlID0gMjEpKwogIHlsYWIoIlRydWUgUG9zaXRpdmUgUmF0ZSBbVFBSXSIpKwogIGdndGl0bGUoIiIpKwogIHRoZW1lX2J3KCkrCiAgdGhlbWUoYXhpcy50ZXh0LnggPSBlbGVtZW50X3RleHQoYW5nbGUgPSA0NSwgaGp1c3QgPSAxKSkrCiAgc2NhbGVfY29sb3JfZ3JhZGllbnQyKGxvdyA9ICJjeWFuIiwgbWlkPSdibHVlJyxoaWdoID0gInJlZCIpKwogICB4bGFiKCcgTWFsd2FyZSBmYW1pbGllcycgKSsKICAgZ3VpZGVzKGNvbG91cj1GQUxTRSxzaXplPUZBTFNFKSsKICAgIHNjYWxlX3NpemVfY29udGludW91cyhyYW5nZSA9IGMoNSwxNSkpKwogIHRoZW1lKGxlZ2VuZC5wb3NpdGlvbj0iYm90dG9tIikKZGdhX3Jlc3VsdHNfcGxvdAoKYGBgCgoKCiMgUmVzdWx0cyBmb3IgQ1RVMTM6CiMjIExTVE0gV29vZGJyaWRnZSArIDI1NiBiYXRjaHMgKyBNYXNraW5nIGFmdGVyIDEwIGV4ZWN1dGlvbnMgKyBtYXhsZW4gZnJvbSAxMCB0byAxMDAwICsgZmlyc3QgNCBzeW1ib2xzIHJlbW92ZWQKCmBgYHtyIGVjaG89RkFMU0UsIG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0V9CiNzZXR3ZCgicmVzdWx0cy8iKQpmaWxlcyA8LSBsaXN0LmZpbGVzKHBhdGggPSAicmVzdWx0cy8iLHBhdHRlcm49InJlc3VsdHNfdGVzdF9jdHUxMy1sc3RtX2VuZGdhbWUtXFxkKyIpCgpzdHJfcmVwbGFjZShzdHJpbmcgPSAicmVzdWx0c190ZXN0X2N0dTEzLWxzdG1fZW5kZ2FtZS04MC0xLmNzdiIscGF0dGVybiA9ICIuKi0oWzAtOV0rKS1bMC05XSsuY3N2IiwiXFwxIikKCnJlc3VsdHNfbHN0bSA8LSBsYXBwbHkoZmlsZXMsIGZ1bmN0aW9uKHgpIAogICAgICAgICAgICAgICAgcmVhZF9jc3YocGFzdGUoInJlc3VsdHMvIix4LHNlcD0iIikpICAgCiAgICAgICAgICAgICAgICAlPiUgYWRkX2NvbHVtbihtYXhsZW49YXMuaW50ZWdlcihzdHJfcmVwbGFjZShzdHJpbmcgPSB4ICxwYXR0ZXJuID0gIi4qLShbMC05XSspLVswLTldKy5jc3YiLCJcXDEiKSApKQogICAgICAgICAgICAgICAgKQoKcmVzdWx0c19sc3RtIDwtZG8uY2FsbChyYmluZCxyZXN1bHRzX2xzdG0pCgpyZXN1bHRzX2xzdG0gICU+JSBhcnJhbmdlKG1heGxlbikgJT4lIGZpbHRlcihtZXRyaWMgJWluJSAgYygiQmFsYW5jZWQgQWNjdXJhY3kiLCJGMSIsIlByZWNpc2lvbiIsIlNlbnNpdGl2aXR5IiwiU3BlY2lmaWNpdHkiKSkgJT4lCiAgZ2dwbG90KCkgKwogICBsYWJzKHRpdGxlPSJDVFUxMyBbVHJhaW4sVGVzdF0gTFNUTSBXb29kYnJpZGdlIFsxMCBleGVjdXRpb25zXSIsCiAgICAgICAgc3VidGl0bGU9Ik1hc2tpbmcsIDI1NiBiYXRjaCwgZmlyc3QgNCBzeW1ib2xzIHJlbW92ZWRcbk1heGxlbiBmcm9tIDEwIHRvIDEwMDAgIikrCiAgZ2VvbV9ib3hwbG90KGFlcyh4PWFzLmZhY3RvcihtYXhsZW4pLHk9dmFsdWUsZmlsbD1hcy5mYWN0b3IobWF4bGVuKSksY29sb3I9J2JsYWNrJykrCiAgeGxhYigibWF4bGVuIikrCiAgdGhlbWVfYncoKSsKICBkYXJrX3RoZW1lX2dyYXkoKSsKICB0aGVtZShheGlzLnRleHQueCA9IGVsZW1lbnRfdGV4dChhbmdsZSA9IDQ1LCBoanVzdCA9IDEpKSsKICB0aGVtZShsZWdlbmQucG9zaXRpb249Im5vbmUiKSsKICBmYWNldF93cmFwKH5tZXRyaWMsIHNjYWxlcyA9ICJmcmVlIikKYGBgCgojIyBSRiArIG1heGxlbiBmcm9tIDEwIHRvIDEwMDAgKyBmaXJzdCA0IHN5bWJvbHMgcmVtb3ZlZAoKYGBge3IgZWNobz1GQUxTRSwgbWVzc2FnZT1GQUxTRSwgd2FybmluZz1GQUxTRX0KI3NldHdkKCJyZXN1bHRzLyIpCmZpbGVzIDwtIGxpc3QuZmlsZXMocGF0aCA9ICJyZXN1bHRzLyIscGF0dGVybj0icmVzdWx0c190ZXN0X2N0dTEzLXJmLSoiKQoKI3N0cl9yZXBsYWNlKHN0cmluZyA9ICJyZXN1bHRzX3Rlc3RfY3R1MTMtbHN0bV9lbmRnYW1lLTgwLTEuY3N2IixwYXR0ZXJuID0gIi4qLShbMC05XSspLVswLTldKy5jc3YiLCJcXDEiKQoKcmVzdWx0c19yZiA8LSBsYXBwbHkoZmlsZXMsIGZ1bmN0aW9uKHgpIAogICAgICAgICAgICAgICAgcmVhZF9jc3YocGFzdGUoInJlc3VsdHMvIix4LHNlcD0iIikpICAgCiAgICAgICAgICAgICAgICAlPiUgYWRkX2NvbHVtbihtYXhsZW49YXMuaW50ZWdlcihzdHJfcmVwbGFjZShzdHJpbmcgPSB4ICxwYXR0ZXJuID0gIi4qLShbMC05XSspLVswLTldKy5jc3YiLCJcXDEiKSApKQogICAgICAgICAgICAgICAgKQoKcmVzdWx0c19yZiA8LWRvLmNhbGwocmJpbmQscmVzdWx0c19yZikKCnJlc3VsdHNfcmYgICU+JSBhcnJhbmdlKG1heGxlbikgJT4lIGZpbHRlcihtZXRyaWMgJWluJSAgYygiQmFsYW5jZWQgQWNjdXJhY3kiLCJGMSIsIlByZWNpc2lvbiIsIlNlbnNpdGl2aXR5IiwiU3BlY2lmaWNpdHkiKSkgJT4lCiAgZ2dwbG90KCkgKwogIGdndGl0bGUoIlJGIikrCiAgZ2VvbV9ib3hwbG90KGFlcyh4PWFzLmZhY3RvcihtYXhsZW4pLHk9dmFsdWUsZmlsbD1hcy5mYWN0b3IobWF4bGVuKSksY29sb3I9J2JsYWNrJykrCiAgeGxhYigibWF4bGVuIikrCiAgdGhlbWVfYncoKSsKICBkYXJrX3RoZW1lX2dyYXkoKSsKICB0aGVtZShheGlzLnRleHQueCA9IGVsZW1lbnRfdGV4dChhbmdsZSA9IDQ1LCBoanVzdCA9IDEpKSsKICB0aGVtZShsZWdlbmQucG9zaXRpb249Im5vbmUiKSsKICBmYWNldF93cmFwKH5tZXRyaWMsIHNjYWxlcyA9ICJmcmVlIikKYGBgCgojIyBMU1RNIHZzIFJGCgpgYGB7ciBlY2hvPUZBTFNFLCBmaWcud2lkdGg9MTIsIG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0UsIHBhZ2VkLnByaW50PVRSVUV9CnJlc3VsdHM8LXJiaW5kKHJlc3VsdHNfbHN0bSAlPiUgYWRkX2NvbHVtbihtb2RlbD0ibHN0bSIpLHJlc3VsdHNfcmYgJT4lIGFkZF9jb2x1bW4obW9kZWw9InJmIikpCiNyZXN1bHRzCgpyZXN1bHRzICU+JSBmaWx0ZXIobWV0cmljICVpbiUgIGMoIkJhbGFuY2VkIEFjY3VyYWN5IiwiRjEiLCJQcmVjaXNpb24iLCJTZW5zaXRpdml0eSIsIlNwZWNpZmljaXR5IikpICU+JQogIGdncGxvdCgpICsKICBnZ3RpdGxlKCJSRiB2cyBMU1RNIikrCiAgZ2VvbV9ib3hwbG90KGFlcyh4PWFzLmZhY3Rvcihtb2RlbCkseT12YWx1ZSxmaWxsPWFzLmZhY3Rvcihtb2RlbCkpLGNvbG9yPSdibGFjaycpKwogIHhsYWIoIm1heGxlbiIpKwogIHRoZW1lX2J3KCkrCiAgZGFya190aGVtZV9ncmF5KCkrCiAgdGhlbWUoYXhpcy50ZXh0LnggPSBlbGVtZW50X3RleHQoYW5nbGUgPSA0NSwgaGp1c3QgPSAxKSkrCiAgdGhlbWUobGVnZW5kLnBvc2l0aW9uPSJub25lIikrCiAgZmFjZXRfZ3JpZChtZXRyaWN+bWF4bGVuLCBzY2FsZXMgPSAiZnJlZSIpCgpgYGAKCiMjIFJGIHRyYWluZWQgd2l0aCByYW5kb20gc2VxdWVuY2VzIG9mIG1heGxlbj0xMCAKCkRhdGFzZXQgY3R1MTNzdWJzLmNzdjogY3R1MTMuY3N2IGZpbHRlcmluZyBzZXF1ZW5jZXMgPjc1IGFuZCBleHRyYWN0aW5nIHN1YnNxdWVuY2VzIG9mIGxlbmdodD0xMCB3aXRoIHJhbmRvbSBzdGFydApgYGB7ciBlY2hvPUZBTFNFLCBtZXNzYWdlPUZBTFNFLCB3YXJuaW5nPUZBTFNFfQpmaWxlcyA8LSBsaXN0LmZpbGVzKHBhdGggPSAicmVzdWx0cy8iLHBhdHRlcm49InJlc3VsdHNfdGVzdF9jdHUxMy1yZi1zdWIqIikKCgpyZXN1bHRzX3JmIDwtIGxhcHBseShmaWxlcywgZnVuY3Rpb24oeCkgCiAgICAgICAgICAgICAgICByZWFkX2NzdihwYXN0ZSgicmVzdWx0cy8iLHgsc2VwPSIiKSkgICAKICAgICAgICAgICAgICAgICU+JSBhZGRfY29sdW1uKG1heGxlbj1hcy5pbnRlZ2VyKHN0cl9yZXBsYWNlKHN0cmluZyA9IHggLHBhdHRlcm4gPSAiLiotKFswLTldKyktWzAtOV0rLmNzdiIsIlxcMSIpICkpCiAgICAgICAgICAgICAgICApCgpyZXN1bHRzX3JmIDwtZG8uY2FsbChyYmluZCxyZXN1bHRzX3JmKQojcmVzdWx0c19yZgpgYGAKCiMjIExTVE0gdHJhaW5lZCB3aXRoIHJhbmRvbSBzZXF1ZW5jZXMgb2YgbWF4bGVuPTEwIAoKTFNUTSBXb29kYnJpZGdlICsgMjU2IGJhdGNocyArIE1hc2tpbmcgYWZ0ZXIgMTAgZXhlY3V0aW9ucyArIG1heGxlbiBmcm9tIDEwIHRvIDEwMDAgKyBmaXJzdCA0IHN5bWJvbHMgcmVtb3ZlZApEYXRhc2V0IGN0dTEzc3Vicy5jc3Y6IGN0dTEzLmNzdiB+NjI1IHNlcXVlbmNlcy4gRmlsdGVyaW5nIHNlcXVlbmNlcyA+NzUgYW5kIGV4dHJhY3Rpbmcgc3Vic3F1ZW5jZXMgb2YgbGVuZ2h0PTEwIHdpdGggcmFuZG9tIHN0YXJ0CgpgYGB7ciBlY2hvPUZBTFNFLCBtZXNzYWdlPUZBTFNFLCB3YXJuaW5nPUZBTFNFfQpmaWxlcyA8LSBsaXN0LmZpbGVzKHBhdGggPSAicmVzdWx0cy8iLHBhdHRlcm49InJlc3VsdHNfdGVzdF9jdHUxMy1sc3RtX2VuZGdhbWUtc3ViKiIpCgpzdHJfcmVwbGFjZShzdHJpbmcgPSAicmVzdWx0c190ZXN0X2N0dTEzLWxzdG1fZW5kZ2FtZS04MC0xLmNzdiIscGF0dGVybiA9ICIuKi0oWzAtOV0rKS1bMC05XSsuY3N2IiwiXFwxIikKCnJlc3VsdHNfbHN0bSA8LSBsYXBwbHkoZmlsZXMsIGZ1bmN0aW9uKHgpIAogICAgICAgICAgICAgICAgcmVhZF9jc3YocGFzdGUoInJlc3VsdHMvIix4LHNlcD0iIikpICAgCiAgICAgICAgICAgICAgICAlPiUgYWRkX2NvbHVtbihtYXhsZW49YXMuaW50ZWdlcihzdHJfcmVwbGFjZShzdHJpbmcgPSB4ICxwYXR0ZXJuID0gIi4qLShbMC05XSspLVswLTldKy5jc3YiLCJcXDEiKSApKQogICAgICAgICAgICAgICAgKQoKcmVzdWx0c19sc3RtIDwtZG8uY2FsbChyYmluZCxyZXN1bHRzX2xzdG0pCiNyZXN1bHRzX2xzdG0KYGBgCgojIyBMU1RNIHZzIFJGIHJhbmRvbSBzdWJzcXVlbmNlIG1heGxlbj0xMCB1c2luZyBkYXRhc2V0IHdpdGggc2VxdWVuY2VzID43NQoKYGBge3IgZWNobz1GQUxTRSwgbWVzc2FnZT1GQUxTRSwgd2FybmluZz1GQUxTRX0KcmVzdWx0czwtcmJpbmQocmVzdWx0c19sc3RtICU+JSBhZGRfY29sdW1uKG1vZGVsPSJsc3RtIikscmVzdWx0c19yZiAlPiUgYWRkX2NvbHVtbihtb2RlbD0icmYiKSkKCgpyZXN1bHRzICU+JSBmaWx0ZXIobWV0cmljICVpbiUgIGMoIkJhbGFuY2VkIEFjY3VyYWN5IiwiRjEiLCJQcmVjaXNpb24iLCJTZW5zaXRpdml0eSIsIlNwZWNpZmljaXR5IikpICU+JQogIGdncGxvdCgpICsKICBnZ3RpdGxlKCJSRiB2cyBMU1RNIikrCiAgZ2VvbV9ib3hwbG90KGFlcyh4PWFzLmZhY3Rvcihtb2RlbCkseT12YWx1ZSxmaWxsPWFzLmZhY3Rvcihtb2RlbCkpLGNvbG9yPSdibGFjaycpKwogIHhsYWIoIm1heGxlbiIpKwogIHRoZW1lX2J3KCkrCiAgZGFya190aGVtZV9ncmF5KCkrCiAgdGhlbWUoYXhpcy50ZXh0LnggPSBlbGVtZW50X3RleHQoYW5nbGUgPSA0NSwgaGp1c3QgPSAxKSkrCiAgdGhlbWUobGVnZW5kLnBvc2l0aW9uPSJub25lIikrCiAgZmFjZXRfd3JhcCh+bWV0cmljLCBzY2FsZXMgPSAiZnJlZSIpCmBgYAoKIyMgTFNUTSB2cyBSRiByYW5kb20gc3Vic3F1ZW5jZSBtYXhsZW49MTAsMjUsNzUgdXNpbmcgZnVsbCBkYXRhc2V0IAoKYGBge3IgZWNobz1GQUxTRSwgbWVzc2FnZT1GQUxTRSwgd2FybmluZz1GQUxTRX0KZmlsZXMgPC0gbGlzdC5maWxlcyhwYXRoID0gInJlc3VsdHMvIixwYXR0ZXJuPSJyZXN1bHRzX3Rlc3RfY3R1MTMtbHN0bV9lbmRnYW1lLXN1YnMtZnVsbGRhdGFzZXQtY29ycmVjdCoiKQoKCnJlc3VsdHNfbHN0bSA8LSBsYXBwbHkoZmlsZXMsIGZ1bmN0aW9uKHgpIAogICAgICAgICAgICAgICAgcmVhZF9jc3YocGFzdGUoInJlc3VsdHMvIix4LHNlcD0iIikpICAgCiAgICAgICAgICAgICAgICAlPiUgYWRkX2NvbHVtbihtYXhsZW49YXMuaW50ZWdlcihzdHJfcmVwbGFjZShzdHJpbmcgPSB4ICxwYXR0ZXJuID0gIi4qLShbMC05XSspLVswLTldKy5jc3YiLCJcXDEiKSApKQogICAgICAgICAgICAgICAgKQoKZmlsZXMgPC0gbGlzdC5maWxlcyhwYXRoID0gInJlc3VsdHMvIixwYXR0ZXJuPSJyZXN1bHRzX3Rlc3RfY3R1MTMtcmYtc3Vicy1mdWxsZGF0YXNldC1jb3JyZWN0KiIpCgoKcmVzdWx0c19yZiA8LSBsYXBwbHkoZmlsZXMsIGZ1bmN0aW9uKHgpIAogICAgICAgICAgICAgICAgcmVhZF9jc3YocGFzdGUoInJlc3VsdHMvIix4LHNlcD0iIikpICAgCiAgICAgICAgICAgICAgICAlPiUgYWRkX2NvbHVtbihtYXhsZW49YXMuaW50ZWdlcihzdHJfcmVwbGFjZShzdHJpbmcgPSB4ICxwYXR0ZXJuID0gIi4qLShbMC05XSspLVswLTldKy5jc3YiLCJcXDEiKSApKQogICAgICAgICAgICAgICAgKQoKcmVzdWx0c19yZiA8LWRvLmNhbGwocmJpbmQscmVzdWx0c19yZikKcmVzdWx0c19sc3RtIDwtZG8uY2FsbChyYmluZCxyZXN1bHRzX2xzdG0pCgpyZXN1bHRzPC1yYmluZChyZXN1bHRzX2xzdG0gJT4lIGFkZF9jb2x1bW4obW9kZWw9ImxzdG0iKSxyZXN1bHRzX3JmICU+JSBhZGRfY29sdW1uKG1vZGVsPSJyZiIpKQoKCnJlc3VsdHMgJT4lIGZpbHRlcihtZXRyaWMgJWluJSAgYygiQmFsYW5jZWQgQWNjdXJhY3kiLCJGMSIsIlByZWNpc2lvbiIsIlNlbnNpdGl2aXR5IiwiU3BlY2lmaWNpdHkiKSkgJT4lCiAgZ2dwbG90KCkgKwogIGdndGl0bGUoIlJGIHZzIExTVE0iKSsKICBnZW9tX2JveHBsb3QoYWVzKHg9YXMuZmFjdG9yKG1vZGVsKSx5PXZhbHVlLGZpbGw9YXMuZmFjdG9yKG1vZGVsKSksY29sb3I9J2JsYWNrJykrCiAgeGxhYigibWF4bGVuIikrCiAgdGhlbWVfYncoKSsKICBkYXJrX3RoZW1lX2dyYXkoKSsKICB0aGVtZShheGlzLnRleHQueCA9IGVsZW1lbnRfdGV4dChhbmdsZSA9IDQ1LCBoanVzdCA9IDEpKSsKICB0aGVtZShsZWdlbmQucG9zaXRpb249Im5vbmUiKSsKICBmYWNldF9ncmlkKG1heGxlbn5tZXRyaWMpCmBgYAoKIyBSZXN1bHRzIGZvciBDVFUxOToKCiMjIExTVE0gV29vZGJyaWRnZSArIDI1NiBiYXRjaHMgKyBNYXNraW5nIGFmdGVyIDEwIGV4ZWN1dGlvbnMgKyBtYXhsZW4gZnJvbSAxMCB0byAxMDAwICsgZmlyc3QgNCBzeW1ib2xzIHJlbW92ZWQKCmBgYHtyIGVjaG89RkFMU0UsIG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0V9Cgojc2V0d2QoInJlc3VsdHMvIikKZmlsZXMgPC0gbGlzdC5maWxlcyhwYXRoID0gInJlc3VsdHMvIixwYXR0ZXJuPSJyZXN1bHRzX3Rlc3RfY3R1MTktbHN0bV9lbmRnYW1lLVxcZCsiKQoKc3RyX3JlcGxhY2Uoc3RyaW5nID0gInJlc3VsdHNfdGVzdF9jdHUxMy1sc3RtX2VuZGdhbWUtODAtMS5jc3YiLHBhdHRlcm4gPSAiLiotKFswLTldKyktWzAtOV0rLmNzdiIsIlxcMSIpCgpyZXN1bHRzX2xzdG0gPC0gbGFwcGx5KGZpbGVzLCBmdW5jdGlvbih4KSAKICAgICAgICAgICAgICAgIHJlYWRfY3N2KHBhc3RlKCJyZXN1bHRzLyIseCxzZXA9IiIpKSAgIAogICAgICAgICAgICAgICAgJT4lIGFkZF9jb2x1bW4obWF4bGVuPWFzLmludGVnZXIoc3RyX3JlcGxhY2Uoc3RyaW5nID0geCAscGF0dGVybiA9ICIuKi0oWzAtOV0rKS1bMC05XSsuY3N2IiwiXFwxIikgKSkKICAgICAgICAgICAgICAgICkKCnJlc3VsdHNfbHN0bSA8LWRvLmNhbGwocmJpbmQscmVzdWx0c19sc3RtKQoKcmVzdWx0c19sc3RtICAlPiUgYXJyYW5nZShtYXhsZW4pICU+JSBmaWx0ZXIobWV0cmljICVpbiUgIGMoIkJhbGFuY2VkIEFjY3VyYWN5IiwiRjEiLCJQcmVjaXNpb24iLCJTZW5zaXRpdml0eSIsIlNwZWNpZmljaXR5IikpICU+JQogIGdncGxvdCgpICsKICBsYWJzKHRpdGxlPSJDVFUxOSBbVHJhaW4sVGVzdF0gTFNUTSBXb29kYnJpZGdlIFsxMCBleGVjdXRpb25zXSIsCiAgICAgICAgc3VidGl0bGU9Ik1hc2tpbmcsIDI1NiBiYXRjaCwgZmlyc3QgNCBzeW1ib2xzIHJlbW92ZWRcbk1heGxlbiBmcm9tIDEwIHRvIDEwMDAgIikrCiAgZ2VvbV9ib3hwbG90KGFlcyh4PWFzLmZhY3RvcihtYXhsZW4pLHk9dmFsdWUsZmlsbD1hcy5mYWN0b3IobWF4bGVuKSksY29sb3I9J2JsYWNrJykrCiAgeGxhYigibWF4bGVuIikrCiAgdGhlbWVfYncoKSsKICBkYXJrX3RoZW1lX2dyYXkoKSsKICB0aGVtZShheGlzLnRleHQueCA9IGVsZW1lbnRfdGV4dChhbmdsZSA9IDQ1LCBoanVzdCA9IDEpKSsKICB0aGVtZShsZWdlbmQucG9zaXRpb249Im5vbmUiKSsKICBmYWNldF93cmFwKH5tZXRyaWMsIHNjYWxlcyA9ICJmcmVlIikKYGBgCgojIyBSRiArIG1heGxlbiBmcm9tIDEwIHRvIDEwMDAgKyBmaXJzdCA0IHN5bWJvbHMgcmVtb3ZlZApgYGB7ciBlY2hvPUZBTFNFLCBtZXNzYWdlPUZBTFNFLCB3YXJuaW5nPUZBTFNFfQojc2V0d2QoInJlc3VsdHMvIikKZmlsZXMgPC0gbGlzdC5maWxlcyhwYXRoID0gInJlc3VsdHMvIixwYXR0ZXJuPSJyZXN1bHRzX3Rlc3RfY3R1MTktcmYtXFxkKyIpCgojc3RyX3JlcGxhY2Uoc3RyaW5nID0gInJlc3VsdHNfdGVzdF9jdHUxMy1sc3RtX2VuZGdhbWUtODAtMS5jc3YiLHBhdHRlcm4gPSAiLiotKFswLTldKyktWzAtOV0rLmNzdiIsIlxcMSIpCgpyZXN1bHRzX3JmIDwtIGxhcHBseShmaWxlcywgZnVuY3Rpb24oeCkgCiAgICAgICAgICAgICAgICByZWFkX2NzdihwYXN0ZSgicmVzdWx0cy8iLHgsc2VwPSIiKSkgICAKICAgICAgICAgICAgICAgICU+JSBhZGRfY29sdW1uKG1heGxlbj1hcy5pbnRlZ2VyKHN0cl9yZXBsYWNlKHN0cmluZyA9IHggLHBhdHRlcm4gPSAiLiotKFswLTldKyktWzAtOV0rLmNzdiIsIlxcMSIpICkpCiAgICAgICAgICAgICAgICApCgpyZXN1bHRzX3JmIDwtZG8uY2FsbChyYmluZCxyZXN1bHRzX3JmKQoKcmVzdWx0c19yZiAgJT4lIGFycmFuZ2UobWF4bGVuKSAlPiUgZmlsdGVyKG1ldHJpYyAlaW4lICBjKCJCYWxhbmNlZCBBY2N1cmFjeSIsIkYxIiwiUHJlY2lzaW9uIiwiU2Vuc2l0aXZpdHkiLCJTcGVjaWZpY2l0eSIpKSAlPiUKICBnZ3Bsb3QoKSArCiAgIGxhYnModGl0bGU9IkNUVTE5IFtUcmFpbixUZXN0XSBSRiBbMTAgZXhlY3V0aW9uc10iLAogICAgICAgIHN1YnRpdGxlPSJGaXJzdCA0IHN5bWJvbHMgcmVtb3ZlZFxuTWF4bGVuIGZyb20gMTAgdG8gMTAwMCAiKSsKICBnZW9tX2JveHBsb3QoYWVzKHg9YXMuZmFjdG9yKG1heGxlbikseT12YWx1ZSxmaWxsPWFzLmZhY3RvcihtYXhsZW4pKSxjb2xvcj0nYmxhY2snKSsKICB4bGFiKCJtYXhsZW4iKSsKICBkYXJrX3RoZW1lX2dyYXkoKSsKICAjdGhlbWVfYncoKSsKICB0aGVtZShheGlzLnRleHQueCA9IGVsZW1lbnRfdGV4dChhbmdsZSA9IDQ1LCBoanVzdCA9IDEpKSsKICB0aGVtZShsZWdlbmQucG9zaXRpb249Im5vbmUiKSsKICBmYWNldF93cmFwKH5tZXRyaWMsIHNjYWxlcyA9ICJmcmVlIikKYGBgCgojIyBMU1RNIHZzIFJGCgpgYGB7ciBlY2hvPUZBTFNFLCBmaWcud2lkdGg9MTIsIG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0UsIHBhZ2VkLnByaW50PVRSVUV9CnJlc3VsdHM8LXJiaW5kKHJlc3VsdHNfbHN0bSAlPiUgYWRkX2NvbHVtbihtb2RlbD0ibHN0bSIpLHJlc3VsdHNfcmYgJT4lIGFkZF9jb2x1bW4obW9kZWw9InJmIikpCiNyZXN1bHRzCgpyZXN1bHRzICU+JSBmaWx0ZXIobWV0cmljICVpbiUgIGMoIkJhbGFuY2VkIEFjY3VyYWN5IiwiRjEiLCJQcmVjaXNpb24iLCJTZW5zaXRpdml0eSIsIlNwZWNpZmljaXR5IikpICU+JQogIGdncGxvdCgpICsKICBsYWJzKHRpdGxlPSJDVFUxOSBbVHJhaW4sVGVzdF0gUkYgdnMgTFNUTSBbMTAgZXhlY3V0aW9uc10iLAogICAgICAgIHN1YnRpdGxlPSJGaXJzdCA0IHN5bWJvbHMgcmVtb3ZlZFxuTWF4bGVuIGZyb20gMTAgdG8gMTAwMCAiKSsKICBnZW9tX2JveHBsb3QoYWVzKHg9YXMuZmFjdG9yKG1vZGVsKSx5PXZhbHVlLGZpbGw9YXMuZmFjdG9yKG1vZGVsKSksY29sb3I9J2JsYWNrJykrCiAgeGxhYigibWF4bGVuIikrCiAgdGhlbWVfYncoKSsKICBkYXJrX3RoZW1lX2dyYXkoKSsKICB0aGVtZShheGlzLnRleHQueCA9IGVsZW1lbnRfdGV4dChhbmdsZSA9IDQ1LCBoanVzdCA9IDEpKSsKICB0aGVtZShsZWdlbmQucG9zaXRpb249Im5vbmUiKSsKICBmYWNldF9ncmlkKG1ldHJpY35tYXhsZW4sIHNjYWxlcyA9ICJmcmVlIikKCmBgYAo=