𝗦𝗲𝗹𝗳-𝗛𝗼𝘀𝘁𝗶𝗻𝗴 𝗔𝗜 𝘄𝗶𝘁𝗵 𝗝𝗲𝘁𝘀𝗼𝗻 𝗢𝗿𝗶𝗻 𝗡𝗮𝗻𝗼 𝗮𝗻𝗱 𝗢𝗹𝗹𝗮𝗺𝗮
Cloud AI servers are expensive.
Connecting cloud services to local agents creates friction. I faced this problem while building my apps, Govend and Flippy.
I wanted to use Ollama to power chat interfaces and flashcard generation. I did not want to pay monthly fees for cloud servers. I wanted a local solution.
I chose the Nvidia Jetson Orin Nano. It handles small AI models efficiently. It is a one-time cost instead of a recurring bill.
The setup was not easy. I learned several hard lessons:
• Avoid SD cards. Use an NVMe hard drive instead. • Watch your monitor timing. My screen went to sleep during boot. I missed crucial input prompts because the monitor hibernated too fast. • Prepare for hardware friction. I had to sync the power button and the Nano boot perfectly to get a signal.
Once the hardware worked, the real work began. I moved away from services like Heroku and Render. I built a central server hub to host my applications locally.
This transition taught me about security. Using AI introduces new risks. I had to learn how to protect my setup.
What I implemented to secure my hub:
- Rate limiting to prevent abuse.
- Progressive account lockouts.
- Security headers and strict Content Security Policies.
- A server firewall to close unused ports.
- A security.txt file for responsible disclosure.
I used Claude Code to help build my server hub page. It helped me identify vulnerabilities before I went public.
Building this local AI infrastructure was a massive learning experience. It turned my development process from relying on external clouds to owning my entire stack.
Check out my new hub here: annavillarreal.com
Source: https://dev.to/annavi11arrea1/self-hosting-experience-with-jetson-orin-nano-and-ollama-5a9c
Optional learning community: https://t.me/GyaanSetuAi