{"id":2738,"date":"2023-12-03T19:45:43","date_gmt":"2023-12-04T00:45:43","guid":{"rendered":"https:\/\/maximumsettings.com\/?p=2738"},"modified":"2023-12-04T23:30:22","modified_gmt":"2023-12-05T04:30:22","slug":"oobabooga-llm-yi-34b-chat-q3_k_l-gguf-llama2-13b-psyfighter2-q4_k_m-gguf-7900xtx-gpu-7800x3d","status":"publish","type":"post","link":"https:\/\/maximumsettings.com\/?p=2738","title":{"rendered":"oobabooga LLM | yi-34b-chat.Q3_K_L.gguf | LLaMA2-13B-Psyfighter2.Q4_K_M.gguf | 7900XTX GPU | 7800X3D"},"content":{"rendered":"

\"\"<\/a><\/p>\n

Hello fellow gamers!<\/strong> Did you know that you can also run an AI Language Model like ChatGPT on your gaming PC? In fact, you can fully install your own large language model in under 10 minutes! This video tutorial will walk you through the entire setup. Let’s start by searching for “oobabooga LLM” on Google and it will take you to this site. Please scroll down a bit and click “Download.”<\/p>\n

Now go to your download folder, copy the oobabooga file to a better location such as the “Documents” folder. Then extract the files by right-clicking on the extracted folder, selecting “Extract,” and following the prompts. Next, open a terminal in the extracted directory by right-clicking again and selecting “Open Terminal Here.” Finally, type “ls” to list the files. https:\/\/github.com\/oobabooga\/text-generation-webui<\/a><\/p>\n

Now let’s begin the setup process by executing the command “sudo .\/start_linux.sh<\/strong>” After that completes loading, you will be presented with several options. Since your gaming PC has a high-end graphics card, choose option B. This step might take a few minutes; however, we can skip ahead to downloading and setting up the AI model:<\/p>\n

You’ll need to download these files for this purpose: https:\/\/huggingface.co\/TheBloke\/Yi-34B-Chat-GGUF<\/a>
\nCopy the name to the clip board and past it on the “Download model or LoRA\u201d<\/p>\n

Next, click on the “Get File List” button in the terminal and wait for it to retrieve the list of files present in that directory. Now paste the file named yi-34b-chat.Q3_K_L.gguf<\/strong>
\ninto the text box labeled GGUF and click “Download.”<\/p>\n

This process might take some time, but once it’s finished, you can proceed to load the model by clicking on “Refresh” in the terminal window and selecting your newly downloaded AI model from the drop-down menu that appears.
\nBefore loading the model, ensure that you increase the value of n-gpu-layers to 65 as only the AMD 7900xtx supports larger models like this one. If your GPU is an AMD Radeon RX 6800 XT, you should use a smaller model such as https:\/\/huggingface.co\/KoboldAI\/LLaMA2-13B-Psyfighter2-GGUF<\/a>
\nLLaMA2-13B-Psyfighter2.Q4_K_M.gguf<\/strong> instead. Once everything’s set up correctly, click “Load” and wait for the model to load into memory of the GPU on your gaming PC.<\/p>\n

Finally, you can ask the AI a question by typing it into the terminal after pressing enter: “what places should I visit in Toronto?” The model will provide you with an answer based on its knowledge and learning. Enjoy exploring Toronto through the eyes of your newly installed large language model!<\/p>\n","protected":false},"excerpt":{"rendered":"

Hello fellow gamers! Did you know that you can also run an AI Language Model like ChatGPT on your gaming PC? In fact, you can fully install your own large language model in under 10 minutes! This video tutorial will walk you through the entire setup. Let’s start by searching for “oobabooga LLM” on Google…<\/p>\n","protected":false},"author":1,"featured_media":2739,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"video","meta":{"footnotes":""},"categories":[2],"tags":[84,85,87],"table_tags":[],"class_list":["post-2738","post","type-post","status-publish","format-video","has-post-thumbnail","hentry","category-demos","tag-cloud-gaming","tag-linux-cloud-gaming","tag-mini-streaming-pc","post_format-post-format-video"," c-blog-1-item nt-post-class"],"_links":{"self":[{"href":"https:\/\/maximumsettings.com\/index.php?rest_route=\/wp\/v2\/posts\/2738","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/maximumsettings.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/maximumsettings.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/maximumsettings.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/maximumsettings.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2738"}],"version-history":[{"count":13,"href":"https:\/\/maximumsettings.com\/index.php?rest_route=\/wp\/v2\/posts\/2738\/revisions"}],"predecessor-version":[{"id":2752,"href":"https:\/\/maximumsettings.com\/index.php?rest_route=\/wp\/v2\/posts\/2738\/revisions\/2752"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/maximumsettings.com\/index.php?rest_route=\/wp\/v2\/media\/2739"}],"wp:attachment":[{"href":"https:\/\/maximumsettings.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2738"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/maximumsettings.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2738"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/maximumsettings.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2738"},{"taxonomy":"table_tags","embeddable":true,"href":"https:\/\/maximumsettings.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftable_tags&post=2738"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}