𝗧𝗵𝗲 𝗛𝗮𝗿𝗱 𝗥𝗲𝗮𝗹𝗶𝘁𝘆 𝗢𝗳 𝗔𝗜 𝗖𝗼𝘀𝘁𝘀

𝗧𝗵𝗲 𝗛𝗮𝗿𝗱 𝗥𝗲𝗮𝗹𝗶𝘁𝘆 𝗢𝗳 𝗔𝗜 𝗖𝗼𝘀𝘁𝘀

I built an AI bot named Monika. I used a Qwen 2.5-7B model with 687 dialogues. I had no budget. I hit many walls with free hardware.

• I tried Kaggle. I ran out of VRAM.
• I tried Lightning AI. The libraries clashed.
• I used Google Colab. I used QLoRA to fit the model on a 16GB GPU.
• I merged the model on Kaggle. I needed 30GB of RAM.
• I split the final file into 3GB parts to upload it.

The model is ready. It sits on Hugging Face. I lack the money to run it. Hosting a 14GB model costs money.

I have the technical skills. I lack the funds for the server. Infrastructure is a paywall.

Build your models. Plan for the server bill.

Source: https://dev.to/akshat-ray/fine-tuned-7b-llm-as-a-broke-student-and-cant-even-use-it--de7 Optional learning community: https://t.me/GyaanSetuAi