{"id":580,"date":"2020-08-12T22:27:03","date_gmt":"2020-08-12T13:27:03","guid":{"rendered":"http:\/\/www.mukgee.com\/?p=580"},"modified":"2020-12-27T01:24:16","modified_gmt":"2020-12-26T16:24:16","slug":"m-llol-%eb%8d%b0%ec%9d%b4%ed%84%b0%eb%a5%bc-%ed%99%9c%ec%9a%a9%ed%95%9c-skill-%ed%8a%b8%eb%a6%ac-%ed%95%99%ec%8a%b55","status":"publish","type":"post","link":"http:\/\/mukgee.com\/?p=580","title":{"rendered":"[M\/L]LOL \ub370\uc774\ud130\ub97c \ud65c\uc6a9\ud55c Skill \ud2b8\ub9ac \ud559\uc2b5(5)"},"content":{"rendered":"<p>LOL \uac8c\uc784 \ub370\uc774\ud130\ub97c \ubc14\ud0d5\uc73c\ub85c RNN\uc744 \uc774\uc6a9\ud574 18\ub808\ubca8\uae4c\uc9c0\uc758 skill \uc2dc\ud000\uc2a4\ub97c \ub9cc\ub4e4\uc5b4\ubcf4\ub294 \uc791\uc5c5\uc744 \ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n<p>\uc774 toy \uacfc\uc81c\uc758 \ubaa9\uc801\uc740 \ud559\uc2b5\ub41c \ubaa8\ub378\uc774 \uc0dd\uc131\ud558\ub294 \uc2dc\ud000\uc2a4\uac00 \uac8c\uc784\uc758 \uaddc\uce59\uc744 \uc774\ud574\ud55c skill \uc2dc\ud000\uc2a4 \uc0dd\uc131 \uc785\ub2c8\ub2e4. \ub370\uc774\ud130\ub9cc\uc73c\ub85c \uc0dd\uc131\ub41c skill \uc2dc\ud000\uc2a4\ub294 6\ubc88\uc9f8\uc5d0 R\uc774 \ub098\uc640\uc57c\ud558\uace0 QWE \uc2a4\ud0ac\uc740 5\ubc88\uae4c\uc9c0\ub9cc \ubc18\ubcf5\ub418\uc5b4\uc57c\ud558\ub294 \uaddc\uce59\uc744 \ucc3e\uc544\uc57c\ud569\ub2c8\ub2e4.<\/p>\n<p>&nbsp;<\/p>\n<p>RNN(LSTM)\uc73c\ub85c \ud559\uc2b5\uc744 \uc9c4\ud589\ud588\uc9c0\ub9cc window size, \ub370\uc774\ud130 \uad6c\uc131 \ub4f1\uc758 \uc774\uc720\uc5d0\uc11c \uc6d0\ud558\ub294 \uacb0\uacfc\ub97c \uc5bb\uc9c0 \ubabb\ud588\uc2b5\ub2c8\ub2e4.<\/p>\n<p>seq2seq \ud559\uc2b5\uc5d0\uc11c \ucd5c\uadfc \ud2b8\ub79c\ub4dc\ub294 RNN\ubcf4\ub2e4\ub294 \ub370\uc774\ud130\uac00 time \ubcc4\ub85c \uc5bc\ub9c8\ub098 \uc601\ud5a5\uc744 \ubc1b\uc558\ub294\uc9c0\ub97c \ub098\ud0c0\ub0b4\ub294 Attention \uc744 \ud65c\uc6a9\ud55c \ubaa8\ub378\ub4e4\uc774 \uc0ac\uc6a9\ub418\uace0 \uc788\uc2b5\ub2c8\ub2e4. 2017\ub144 \ubc1c\ud45c\ub41c\u00a0Attention Is All You Need \ub77c\ub294 \ub17c\ubb38 \uc774\ud6c4 attension \uc744 \uc774\uc6a9\ud55c seq2seq \ubaa8\ub378\ub4e4\uc774 \ub9ce\uc774 \ub098\uc624\uace0 \uc788\ub294\ub370\uc694.<\/p>\n<p>\uadf8 \uc911 Transformer \ubaa8\ub378\uc744 \uc774\uc6a9\ud558\uc5ec \ud559\uc2b5\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.<\/p>\n<p>( \uae30\uacc4 \ubc88\uc5ed\uc744 \uc704\ud55c Transformer \ubaa8\ub378\uc740 <a href=\"https:\/\/blog.tensorflow.org\/2019\/05\/transformer-chatbot-tutorial-with-tensorflow-2.html\">Tensorflow \ube14\ub85c\uadf8<\/a>\uc5d0 \uc790\uc138\ud788 \uc124\uba85\ub418\uc5b4 \uc788\uc2b5\ub2c8\ub2e4.)<\/p>\n<p><img loading=\"lazy\" class=\"alignnone size-large wp-image-581\" src=\"http:\/\/www.mukgee.com\/wp-content\/uploads\/2020\/08\/D468B7A4-4329-420D-929F-BB33331EF4D6-769x1024.jpeg\" alt=\"\" width=\"769\" height=\"1024\" \/><\/p>\n<p>Transformer \ubaa8\ub378\uc740 \ud06c\uac8c Encoder\uc640 Decoder\ub85c \uad6c\uc131\ub429\ub2c8\ub2e4. \uac01\uac01\uc758 \ubaa8\ub378\uc740 \uc815\ud655\ud788 \uc704\uc758 \uc774\ubbf8\uc9c0\uc640 \uac19\uc740 layer\ub97c \uac00\uc9c0\uace0 \uc788\uace0 RNN cell\uc774 \uc544\ub2cc attention\uc744 \uacc4\uc0b0\ud558\ub294 layer\uc640 FC \ub808\uc774\uc5b4 \ub9cc\uc73c\ub85c seq2seq\ub97c \ud559\uc2b5\ud569\ub2c8\ub2e4.<\/p>\n<p>Transformer \ubaa8\ub378\uc5d0 \ub300\ud55c \uc790\uc138\ud55c \ub0b4\uc6a9\uc740\u00a0<a href=\"https:\/\/blog.tensorflow.org\/2019\/05\/transformer-chatbot-tutorial-with-tensorflow-2.html\">Tensorflow \ube14\ub85c\uadf8<\/a>\u00a0\uc5d0\uc11c \ud655\uc778\uac00\ub2a5\ud569\ub2c8\ub2e4.<\/p>\n<p>(RNN\uacfc LSTM \uadf8\ub9ac\uace0 Attention \ubaa8\ub378\uc5d0 \ub300\ud55c \uc0ac\uc804 \uc774\ud574\uac00 \uc5b4\ub290 \uc815\ub3c4 \uc788\uc5b4\uc57c \ub530\ub77c \uac08 \uc218 \uc788\uc2b5\ub2c8\ub2e4.)<\/p>\n<p>&nbsp;<\/p>\n<p>\ub2e4\uc2dc LOL skill \ud559\uc2b5 \ubaa8\ub378\ub85c \ub3cc\uc544\uac00\uaca0\uc2b5\ub2c8\ub2e4.<\/p>\n<p>\uae30\ubcf8\uc801\uc778 \uad6c\uc131\uc740 \uae30\uacc4\ud559\uc2b5 \ucf54\ub4dc\uc640 \ube44\uc2b7\ud558\uac8c \uad6c\uc131\ud558\uc600\uc2b5\ub2c8\ub2e4. Transformer\uc758 \uc7a5\uc810\uc911 \ud558\ub098\ub294 RNN \ucc98\ub7fc window \uc0ac\uc774\uc988\ub97c \uc815\ud558\uc9c0 \uc54a\uc544\ub3c4 \ub41c\ub2e4\ub294 \uc810\uc785\ub2c8\ub2e4. Transformer \ubaa8\ub378\uc5d0\uc11c\ub294 \uac01 time \ubcc4(LOL \ub370\uc774\ud130\uc5d0\uc11c\ub294 level\ubcc4) attention\uc744 \uacc4\uc0b0\ud569\ub2c8\ub2e4.<\/p>\n<p>\ucd5c\uc885 \ubaa9\uc801\uc740 Sequence Generator \uc774\uae30 \ub54c\ubb38\uc5d0 y\uc5d0 input\uacfc \ub3d9\uc77c\ud55c x \uac12\uc744 \uc8fc\uc5b4 encoder\/decoder\ub97c \ud559\uc2b5 \uc2dc\ud0a8 \ud6c4 decoder\ub9cc\uc744 \ud65c\uc6a9\ud558\uc5ec \uc2dc\ud000\uc2a4\ub97c \uc0dd\uc131\ud560 \uac83\uc785\ub2c8\ub2e4.<\/p>\n<p>(Auto Encoder \ubaa8\ub378\uacfc \ube44\uc2b7\ud55c \uc6d0\ub9ac\ub85c \ub3d9\uc791)<\/p>\n<p><img loading=\"lazy\" class=\"alignnone size-large wp-image-583\" src=\"http:\/\/www.mukgee.com\/wp-content\/uploads\/2020\/08\/32F7BEBE-76EB-45E6-BEEB-FA21E40A186D-1024x973.jpeg\" alt=\"\" width=\"843\" height=\"801\" \/><\/p>\n<p>\ud559\uc2b5 \uacb0\uacfc Loss\ub294 \ube60\ub974\uac8c \ub5a8\uc5b4\uc9c0\uace0 \uc788\uc2b5\ub2c8\ub2e4. \ud558\uc9c0\ub9cc Acurracy\uac00 \uc6d0\ud558\ub294 \ub9cc\ud07c \ub5a8\uc5b4\uc9c0\uc9c0 \uc54a\uc2b5\ub2c8\ub2e4. Batch Layer\ub098 Drop out \uc744 \ud1b5\ud574 over-fitting\uc744 \uac1c\uc120\ud558\uace0 hyper para tunning \uc744 \uc9c4\ud589\ud574\uc57c\ud560\uac83 \uac19\uc2b5\ub2c8\ub2e4.<\/p>\n<p>\uc544\uc9c1 \ucda9\ubd84\ud788 \ud559\uc2b5 \ub418\uc9c0 \uc54a\uc558\uace0 decoder\ub9cc \ubd84\ub9ac\ud558\uc9c0 \uc54a\uc558\uc9c0\ub9cc seq\ub97c \uc0dd\uc131 \ud574\ubcf4\uc558\uc2b5\ub2c8\ub2e4.<\/p>\n<p><img loading=\"lazy\" class=\"alignnone size-large wp-image-584\" src=\"http:\/\/www.mukgee.com\/wp-content\/uploads\/2020\/08\/2744679D-F77B-4012-B8B6-DB4814ED8A87-1024x861.jpeg\" alt=\"\" width=\"843\" height=\"709\" \/><\/p>\n<p>\uae30\ub300\ud588\ub358 seq\uac00 \uc544\ub2d9\ub2c8\ub2e4.<\/p>\n<p><del>hyper para tuning \uacfc decoder\ubd84\ub9ac \uc774\ud6c4 \ub2e4\uc2dc \uc2dc\ud000\uc2a4\ub97c \uc0dd\uc131\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.<\/del><\/p>\n<p>&nbsp;<\/p>\n<p>2020.08.18 \ucd94\uac00<\/p>\n<p>\uc774\uae00\uc5d0\uc11c \uc0ac\uc6a9\ud55c greedy selection\uc744 \uc774\uc6a9\ud55c generator\ub294 \ubc18\ubcf5\uc801\uc778 \ubb38\uc7a5\uc744 \uac00\uc9c0\ub294\uac8c common \ud55c \ubb38\uc81c\ub77c\uace0 \ud569\ub2c8\ub2e4.(\ub2e4\uc74c \ub2e8\uc5b4 \uc120\ud0dd\uc2dc np.argmax \ub85c prediction)<\/p>\n<p>transformer\ub97c \uc9c1\uc811 \uad6c\ud604\ud558\ub294\uac83 \ubcf4\ub2e4\ub294 \uc798 \ub9cc\ub4e4\uc5b4\uc9c4 API \ub97c \ud65c\uc6a9\ud558\ub294\uac8c \ud6e8\uc52c \uc88b\uc740 \ubc29\ubc95\uc77c\uac83 \uac19\uc2b5\ub2c8\ub2e4.<br \/>\n<a href=\"https:\/\/huggingface.co\/blog\/how-to-generate\" target=\"_blank\" rel=\"noopener noreferrer\">https:\/\/huggingface.co\/blog\/how-to-generate<\/a> \uae00\uc5d0\uc11c \uc18c\uac1c\ud558\ub294 beam search\/Top-K\/Top-p \uac19\uc740 \ubc29\ubc95\uc744 \ud65c\uc6a9\ud55c generator\ub97c \ub9cc\ub4e4\uc5b4 \ubcfc \uc608\uc815\uc785\ub2c8\ub2e4.<\/p>\n<p>(huggingface library \uc0ac\uc6a9)<\/p>\n","protected":false},"excerpt":{"rendered":"<p>LOL \uac8c\uc784 \ub370\uc774\ud130\ub97c \ubc14\ud0d5\uc73c\ub85c RNN\uc744 \uc774\uc6a9\ud574 18\ub808\ubca8\uae4c\uc9c0\uc758 skill \uc2dc\ud000\uc2a4\ub97c \ub9cc\ub4e4\uc5b4\ubcf4\ub294 \uc791\uc5c5\uc744 \ud558\uace0 \uc788\uc2b5\ub2c8\ub2e4. \uc774 toy \uacfc\uc81c\uc758 \ubaa9\uc801\uc740 \ud559\uc2b5\ub41c \ubaa8\ub378\uc774 \uc0dd\uc131\ud558\ub294 \uc2dc\ud000\uc2a4\uac00 \uac8c\uc784\uc758 \uaddc\uce59\uc744 \uc774\ud574\ud55c skill&hellip;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":""},"categories":[3],"tags":[6,18,24,31],"aioseo_notices":[],"jetpack_featured_media_url":"","_links":{"self":[{"href":"http:\/\/mukgee.com\/index.php?rest_route=\/wp\/v2\/posts\/580"}],"collection":[{"href":"http:\/\/mukgee.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/mukgee.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/mukgee.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/mukgee.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=580"}],"version-history":[{"count":1,"href":"http:\/\/mukgee.com\/index.php?rest_route=\/wp\/v2\/posts\/580\/revisions"}],"predecessor-version":[{"id":594,"href":"http:\/\/mukgee.com\/index.php?rest_route=\/wp\/v2\/posts\/580\/revisions\/594"}],"wp:attachment":[{"href":"http:\/\/mukgee.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=580"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/mukgee.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=580"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/mukgee.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=580"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}