{"id":1014,"date":"2024-06-27T23:42:50","date_gmt":"2024-06-27T14:42:50","guid":{"rendered":"http:\/\/mukgee.com\/?p=1014"},"modified":"2024-06-27T23:42:50","modified_gmt":"2024-06-27T14:42:50","slug":"peft-%eb%aa%a8%eb%8d%b8%ec%9d%84-llama-cpp-%ec%9c%bc%eb%a1%9c-%ec%b6%94%eb%a1%a0-%ed%95%b4%eb%b3%b4%ec%9e%90","status":"publish","type":"post","link":"http:\/\/mukgee.com\/?p=1014","title":{"rendered":"PEFT \ubaa8\ub378\uc744 llama.cpp \uc73c\ub85c \ucd94\ub860 \ud574\ubcf4\uc790"},"content":{"rendered":"

huggingface \ubaa8\ub378\ub4e4\uacfc PEFT \ub77c\uc774\ube0c\ub7ec\ub9ac\uc758 \uc870\ud569\uc740 \uba87 \uc904\uc758 Python \ucf54\ub4dc\ub85c \ub098\ub9cc\uc758 \ubaa8\ub378\uc744 \ub9cc\ub4e4 \uc218 \uc788\uac8c \ud574\uc8fc\uc5c8\uc2b5\ub2c8\ub2e4.<\/p>\n

\ud2b9\ud788, PEFT(Parameter-Efficient Fine-Tuning)\ub294 \uc2dc\uac04\uacfc \uba54\ubaa8\ub9ac\ub97c \ud6a8\uc728\uc801\uc73c\ub85c \uc0ac\uc6a9\ud558\uac8c \ub3c4\uc640\uc8fc\uba70 \ub2e4\uc591\ud55c Downstream application \uc744 \ub9cc\ub4e4 \uc218 \uc788\ub294 \ud658\uacbd\uc744 \uc81c\uacf5\ud558\uc600\uc2b5\ub2c8\ub2e4.<\/p>\n

( PEFT\uc5d0 \ub300\ud574 \uad81\uae08\ud558\uc2dc\ub2e4\uba74, devocean \uc5d0 \uc62c\ub77c\uc628 PEFT \uc18c\uac1c\uae00\uc744 \ucc38\uace0\ud574\uc8fc\uc138\uc694.\u00a0https:\/\/devocean.sk.com\/search\/techBoardDetail.do?ID=164779\u00a0<\/a>)<\/p>\n

\ud558\uc9c0\ub9cc, PEFT \ub85c \ud2b9\uc815 Task\uc5d0 Align \ub41c \ubaa8\ub378\uc744 \uac1c\ubc1c \ud6c4\uc5d0 \uc2e4\uc81c \uc11c\ube44\uc2a4\ub97c \ud558\ub824\uace0 \ud558\uba74, \ucd94\ub860 \uc18d\ub3c4\uac00 \uace0\ubbfc\ub429\ub2c8\ub2e4.<\/p>\n

\uadf8\ub798\uc11c \uc624\ub298\uc740 PEFT library \ub85c fine tune \uc774 \uc9c4\ud589\ub41c \ubaa8\ub378\uc744 llama.cpp \ub85c GGUF \ub85c \ubcc0\ud658\ud55c \ud6c4 \uc591\uc790\ud654\ub97c \ud1b5\ud574 \ucd94\ub860 \uc18d\ub3c4\ub97c \uac1c\uc120\ud558\ub294 \uc791\uc5c5\uc744 \uc9c4\ud589\ud574\ubcf4\ub824\uace0 \ud569\ub2c8\ub2e4.<\/p>\n

\uba3c\uc800 huggingface \ub85c base model \uc744 \ubc1b\uc740 \ub4a4\uc5d0 PEFT library \ub85c fine tune\uc744 \uc9c4\ud589\ud558\uba74, adapter \ud30c\uc77c\ub4e4\uc774 \uc0dd\uc131\ub429\ub2c8\ub2e4.<\/p>\n

PEFT \ubc29\ubc95\uc73c\ub85c fine tune \ub41c \ubaa8\ub378\uc744 \uc0ac\uc6a9\ud558\uae30 \uc704\ud574\uc11c\ub294 \uc544\ub798 \ucf54\ub4dc\ucc98\ub7fc base model \uc744 load \ud558\uace0 \ub2e4\uc2dc adapter \ub97c \ubd99\uc5ec\uc57c\ud558\ub294 \uad6c\uc870\uac00 \ub429\ub2c8\ub2e4.<\/p>\n

\"image.png\"<\/p>\n

\uc544\ub798\uc640 \uac19\uc774 adapter\ub97c Merge \ud55c fine tune \ubaa8\ub378\uc744 \uc0dd\uc131\ud55c\ub2e4\uba74, \ubaa8\ub378\uc744 \ud55c\ubc88 \ub9cc\uc5d0 load \ud560 \uc218 \uc788\uc744\uac83\uc785\ub2c8\ub2e4.<\/p>\n

\"image.png\"<\/p>\n

Merge \ub41c \ud30c\uc77c\ub4e4\uc740 huggingface model \ub4e4\ucc98\ub7fc safetensors \ud639\uc740 bin \ud30c\uc77c\ub4e4\ub85c \uc800\uc7a5\ub429\ub2c8\ub2e4.<\/p>\n

transformer \ub77c\uc774\ube0c\ub7ec\ub9ac\ub97c \uacc4\uc18d \uc0ac\uc6a9\ud558\uba74\uc11c \ud559\uc2b5\ud574\uc57c\ud558\uac70\ub098, \uac80\uc99d\ud574\uc57c\ud558\ub294 \uc2dc\uc810\uc5d0\ub294 \ubb38\uc81c\uac00 \uc5c6\uaca0\uc9c0\ub9cc, \ucd94\ub860 \uc2dc\uc810\uc774\ub77c\uba74 python \uc758 transformer \ub77c\uc774\ube0c\ub7ec\ub9ac\ub85c \ubaa8\ub378\uc744 load \ud558\uace0 \ucd94\ub860\ud558\ub294 \uc791\uc5c5\uc740 \uc0dd\uac01\ubcf4\ub2e4 \uc2dc\uac04\uc774 \ub9ce\uc774 \uac78\ub824\uc11c \uace0\ubbfc\uc774\ub420 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n

\"image.png\"<\/p>\n

\uc774\ub7f4 \ub54c llama.cpp \ub97c \ud65c\uc6a9\ud574\ubcfc \uc218 \uc788\ub294\ub370\uc694. (\u00a0https:\/\/github.com\/ggerganov\/llama.cpp<\/a>\u00a0)<\/p>\n

llama.cpp \ub294 \uc624\ud508\uc18c\uc2a4 llm \ub4e4\uc744 pure C\/C++ \ud658\uacbd\uc5d0\uc11c \uc2e4\ud589\uc2dc\ucf1c \ucd5c\uc18c\ud55c\uc758 setup \uc73c\ub85c \ucd94\ub860 \uc18d\ub3c4\ub97c \ud5a5\uc0c1\uc2dc\ucf1c\uc8fc\ub294 \uc5ec\ub7ec \uae30\ub2a5\uc744 \uc81c\uacf5\ud569\ub2c8\ub2e4.<\/p>\n

llama.cpp \uc5d0\uc11c \ubaa8\ub378\uc744 \uc2e4\ud589\ud558\uae30 \uc704\ud574\uc11c\ub294 huggingface\uc758 \ubaa8\ub378\uc744 GGUF \ud30c\uc77c\ub85c \ubcc0\ud658\ud558\ub294 \uc791\uc5c5\uc774 \ud544\uc694\ud569\ub2c8\ub2e4.<\/p>\n

GGUF\ub294 Georgi Gerganov Unified Format \uc758 \uc57d\uc790\ub85c llama.cpp github \uc8fc\uc18c\uc5d0 \ud45c\uc2dc\ub41c Georgi Gerganov \uac1c\ubc1c\uc790 \ubcf8\uc778\uc758 \uc774\ub984\uc744 \ub534 \ud30c\uc77c \ud655\uc7a5\uc790\uba85\uc785\ub2c8\ub2e4. ( \uc0ac\ub78c\uc740 \uc8fd\uc5b4\uc11c \uc774\ub984\uc744…. )<\/p>\n

\uc774\uc81c \uc21c\uc11c\ub300\ub85c llama.cpp \ub97c \ud65c\uc6a9\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.<\/p>\n

1.\uba3c\uc800 llama.cpp \ub97c \ubcf8\uc778\uc758 \ud658\uacbd\uc5d0 \ub9de\uac8c \uc124\uce58\ud569\ub2c8\ub2e4.<\/p>\n

\uc800\ub294 llama.cpp github \uc5d0\uc11c \uc548\ub0b4\ud55c cmake \ub97c \ud65c\uc6a9\ud574 \uc124\uce58\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.<\/p>\n

\n
git<\/span> clone https:\/\/github.com\/ggerganov\/llama.cpp\r\ncd<\/span> llama.cpp\r\ncmake -B build\r\ncmake --build build --config Release<\/code><\/pre>\n<\/div>\n
2. llama.cpp \ub97c clone \ud558\uba70 \ubc1b\uc544\ub454 convert-hf-to-gguf.py \ub85c GGUF\ub85c \ubcc0\ud658\ud569\ub2c8\ub2e4.<\/p>\n
python \ud30c\uc77c\uc744 -h \uc635\uc158\uacfc \uac19\uc774 \uc2e4\ud589\ud558\uba70 \uc5b4\ub5a4 \uc635\uc158\ub4e4\uc774 \uc788\ub294\uc9c0 \ud655\uc778\ud574\ubd05\ub2c8\ub2e4.<\/p>\n
\n
python convert-hf-to-gguf.py -h<\/code><\/pre>\n<\/div>\n
\"image.png\"<\/p>\n
document \uc548\ub0b4\uc5d0\uc11c\ub294 \uac04\ub2e8\ud558\uac8c convert \ud560 \ubaa8\ub378\ub9cc \uc704\uce58\ub9cc \uc785\ub825\ud558\ub124\uc694.<\/p>\n
–outfile \uc635\uc158\ub9cc \uc8fc\uc5b4\uc11c \uc800\uc7a5\ub420 \uc704\uce58\ub9cc \uc9c0\uc815\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.<\/p>\n
\n
python convert-hf-to-gguf.py \"merged_model_path\"<\/span> --outfile \"merged_model.gguf\"<\/span><\/code><\/pre>\n<\/div>\n
\"image.png\"<\/p>\n
\uc774\uc81c bin \ud30c\uc77c\ub85c \uc800\uc7a5\ub41c \ubaa8\ub378\uc744 gguf \ud30c\uc77c\ub85c \ubcc0\ud658\ud558\uc600\uc2b5\ub2c8\ub2e4.<\/p>\n
bin \ud30c\uc77c\uc73c\ub85c \uc800\uc7a5\ub420\ub54c\uc640 \ub2e4\ub974\uac8c \uac01\uc885 config , tokenizer \ud30c\uc77c\ub4e4\uc774 \uc0ac\ub77c\uc9c0\uace0 \ud558\ub098\uc758 gguf \ud30c\uc77c\ub9cc \uc0dd\uc131\ub41c\uac83\uc744 \ubcfc \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n
3. \uadf8\ub7fc \uc591\uc790\ud654\ub97c \uc9c4\ud589\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.<\/p>\n
\ubaa8\ub378\uc758 weight \ub97c lower bit \ub85c \ubcc0\ud658\ud558\ub294 \uc791\uc5c5\uc740 \ubaa8\ub378\uc758 \ud06c\uae30\ub97c \uc904\uc774\uace0 \ucd94\ub860 \uc18d\ub3c4\ub97c \ud5a5\uc0c1 \uc2dc\ucf1c \uc8fc\uc9c0\ub9cc, \uc791\uc740 \ud06c\uae30\uc758 \ubaa8\ub378\ub4e4\uc744 \uc2e4\ud5d8\ud588\uc744 \ub54c \ud559\uc2b5\ub41c \ub2a5\ub825\uc744 \uc644\uc804 \uc783\uc5b4\ubc84\ub9ac\ub294 \ubd80\uc791\uc6a9\ub3c4 \uc874\uc7ac\ud569\ub2c8\ub2e4.<\/p>\n
\ub3d9\uc77c\ud558\uac8c llama.cpp \ub97c clone \ud560 \ub54c \ub0b4\ub824\ubc1b\uc740 llama-quantize \ud30c\uc77c\uc744 -h \uc635\uc158\uc73c\ub85c \uc2e4\ud589\ud558\uc5ec usgage\ub97c \uc0b4\ud3b4 \ubd05\ub2c8\ub2e4.<\/p>\n
\n
llama-quantize -h<\/code><\/pre>\n<\/div>\n
\"image.png\"<\/p>\n
\uc0c1\ud669\uc5d0 \ub530\ub77c \uc801\uc808\ud55c type\uc744 \uc801\uc6a9\ud558\ub294\uac8c \ud544\uc694\ud558\uaca0\uc9c0\ub9cc, \uc624\ub298\uc740 document \uc5d0 \uc548\ub0b4\ub41c\ub300\ub85c Q4_K_M \ub85c \ud55c\ubc88 \uc9c4\ud589\ud558\uaca0\uc2b5\ub2c8\ub2e4.<\/p>\n
\n
llama-quantize merged_model.gguf merged_model-Q4_K_M.gguf Q4_K_M<\/code><\/pre>\n<\/div>\n
\"image.png\"<\/p>\n
\ub4dc\ub514\uc5b4 \uc591\uc790\ud654\ub41c \ubaa8\ub378\uae4c\uc9c0 \uc0dd\uc131\ub418\uc5c8\uc2b5\ub2c8\ub2e4.<\/p>\n
4. \ubaa8\ub378\uc758 \uc131\ub2a5\uc744 \uce21\uc815\ud558\ub294 \ub2e4\uc591\ud55c benchmark \ub4e4\uc774 \uc788\uaca0\uc9c0\ub9cc, \uac04\ub2e8\ud55c prompt\ub97c \uc9c1\uc811 \uc2e4\ud589\ud558\uba70 \uc2e4\ud589\uc2dc\uac04\uc744 \ud55c\ubc88 \ube44\uad50\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.<\/p>\n
fine tune \ub41c \ubaa8\ub378\uc774 \uc544\ub2cc gemma-2b-it \ubaa8\ub378\uc744 \ud65c\uc6a9\ud569\ub2c8\ub2e4.<\/p>\n
\"image.png\"<\/p>\n
llama.cpp \ub85c \uc591\uc790\ud654\ub41c \ubaa8\ub378\uc5d0 \ub3d9\uc77c prompt\ub97c \uc785\ub825\ud574\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.<\/p>\n
\n
llama-cli gemma-2b-it_Q4_0.gguf --log-disable --predict 128<\/span> -p \"\ub300\ud55c\ubbfc\uad6d\uc758 \uc11c\uc6b8\uc744 \uce5c\uad6c\ub4e4\uc5d0\uac8c \uc18c\uac1c\ud574\uc918\"<\/span><\/code><\/pre>\n<\/div>\n
\"image.png\"<\/p>\n
transformer \ub77c\uc774\ube0c\ub7ec\ub9ac\ub85c \ubaa8\ub378\uc744 \ub85c\ub4dc\ud558\uace0 \ucd94\ub860\ud558\ub294 20s \uc5d0\uc11c llama.cpp \ub85c gguf \ub97c \uc2e4\ud589\ud558\ub294\ub370 109 ms \ub85c \ucd94\ub860 \uc18d\ub3c4\uac00 \ud06c\uac8c \uac1c\uc120\ub41c\uac78 \ubcfc \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n
\ud558\uc9c0\ub9cc, \uc11c\uc6b8\uc758 \uc804\ud1b5 \uc74c\ub8cc\uac00 \ud55c\uc6b0\ub77c\uace0 \ub2f5\ubcc0\ud55c\uac74 \ubd84\uba85\ud55c \ubaa8\ub378\uc758 \ud658\uac01\uc785\ub2c8\ub2e4.<\/p>\n
gemma-2b \ucc98\ub7fc \uc791\uc740 \ubaa8\ub378\uc744 \uc591\uc790\ud654\ud560\ub54c\ub294 \ubaa8\ub378\uc758 \uc131\ub2a5\uc774 \ud06c\uac8c \uc548 \uc88b\uc544\uc9c4\ub2e4\ub294 \uc810\uc744 \ud655\uc778\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.<\/p>\n
\n
\n<\/div>\n
\uc624\ub298\uc740 llama.cpp \ub85c gguf \ud30c\uc77c\ub85c convert \ud558\uace0 \uc591\uc790\ud654\ud558\ub294 \uc791\uc5c5\uc744 \uc9c4\ud589\ud574\ubcf4\uc558\uc2b5\ub2c8\ub2e4.<\/p>\n
gguf \ud30c\uc77c\uc758 \uacbd\uc6b0 localAI \ub098 ollama \uac19\uc740 \ud504\ub85c\uadf8\ub7a8\ub4e4\uc5d0\uc11c\ub3c4 \ub9ce\uc774 \uc0ac\uc6a9\ud558\ub294 type\uc73c\ub85c \ud544\uc694\uc5d0 \ub530\ub77c localAI \ub4f1\uc5d0 fine tune \ub41c \ubaa8\ub378\uc744 \ud1b5\ud569\ud558\uae30 \uc704\ud574\uc11c\ub294 llama.cpp \ub97c \ud65c\uc6a9\ud558\uc5ec gguf \ud30c\uc77c\ub85c \ubcc0\ud658\ud558\ub294 \uc791\uc5c5\uc744 \uc9c4\ud589\ud560 \uc218 \uc788\uc744 \uac83 \uac19\uc2b5\ub2c8\ub2e4.<\/p>\n
llama.cpp \ucc98\ub7fc Open Source \ubaa8\ub378\uc744 \ud65c\uc6a9\ud55c \ub2e4\uc591\ud55c \uae30\uc220\ub4e4\uc774 \ubc1c\uc804\ud560\uc218\ub85d OpenAI \uc640\uc758 \uaca9\ucc28\ub294 \uc810\uc810 \uc904\uc5b4\ub4e4\uc9c0 \uc54a\uc744\uae4c \uae30\ub300\ud574\ubd05\ub2c8\ub2e4.<\/p>\n
 <\/p>\n
 <\/p>\n
\uc774 \uae00\uc740 SK \uac1c\ubc1c\uc790 \ube14\ub85c\uadf8 Devocean\uc5d0 \ub3d9\uc77c\ud558\uac8c \uc791\uc131\ub41c \uae00\uc785\ub2c8\ub2e4.<\/p>\n
https:\/\/devocean.sk.com\/internal\/board\/viewArticleForAjax.do?id=166201<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
huggingface \ubaa8\ub378\ub4e4\uacfc PEFT \ub77c\uc774\ube0c\ub7ec\ub9ac\uc758 \uc870\ud569\uc740 \uba87 \uc904\uc758 Python \ucf54\ub4dc\ub85c \ub098\ub9cc\uc758 \ubaa8\ub378\uc744 \ub9cc\ub4e4 \uc218 \uc788\uac8c \ud574\uc8fc\uc5c8\uc2b5\ub2c8\ub2e4. \ud2b9\ud788, PEFT(Parameter-Efficient Fine-Tuning)\ub294 \uc2dc\uac04\uacfc \uba54\ubaa8\ub9ac\ub97c \ud6a8\uc728\uc801\uc73c\ub85c \uc0ac\uc6a9\ud558\uac8c \ub3c4\uc640\uc8fc\uba70 \ub2e4\uc591\ud55c Downstream…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":""},"categories":[3],"tags":[113,102,114],"aioseo_notices":[],"jetpack_featured_media_url":"","_links":{"self":[{"href":"http:\/\/mukgee.com\/index.php?rest_route=\/wp\/v2\/posts\/1014"}],"collection":[{"href":"http:\/\/mukgee.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/mukgee.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/mukgee.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/mukgee.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1014"}],"version-history":[{"count":1,"href":"http:\/\/mukgee.com\/index.php?rest_route=\/wp\/v2\/posts\/1014\/revisions"}],"predecessor-version":[{"id":1015,"href":"http:\/\/mukgee.com\/index.php?rest_route=\/wp\/v2\/posts\/1014\/revisions\/1015"}],"wp:attachment":[{"href":"http:\/\/mukgee.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1014"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/mukgee.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1014"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/mukgee.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1014"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}