# I'm going to start my journey into the world of fine tuning llms, with an RTX 3080 (don't laugh)

By Morgan Linton · Published 2026-04-28 · X No. 12 · 618 words · https://blog.morganlinton.com/on-x/im-going-to-start-my-journey-into-the-world-of-fine-tuning-llms-with-an-rtx-3080/

![](../../assets/on-x/HG_3DJvbYAAXfNf.jpg)

So lately I've been reading tweet after tweet from people who are optimizing models like Qwen, so they can run better on local hardware. And over the last few weeks my curiosity has grown so much, I'm at the point now where I feel like I need to just dive in and start doing it myself.

But the problem is, I have a ~5 year old  gaming PC with a RTX 3080, which I know, isn't much, but after doing a little research, I think it's enough to get started. Of course, I probably won't be doing anything very impressive, but I do think it will be a great way to learn.

For the last couple of weeks I've thought - if I want to get into fine tuning and putting my own versions of models on HuggingFace, I need to spend $5,000 - $10,000, but I realized, nah - that's just making an excuse and delaying doing something I want to do now.

So here's my plan. I figured I'd just start sharing my journey as I'd imagine there are probably other people like me out there, that don't have a 3090, aren't ready to spend $5k+ on an at home AI lab, but want to start playing around with fine tuning and releasing models on HuggingFace.

And as I hope you can tell at this point in the article, this isn't written by AI, I'm old school, sitting here in my house in Tahoe, typing away like a caveman, still on my first cup of coffee.

Okay, so here's my plan.

I think to start with, I'm going to pick Qwen 2.5 3B. Sure, it's not the most cutting edge model on the planet, I get that, but I have to work with what I've got and it sounds like the 3080 could do some fun stuff with this one.

I'm going to use Unsloth initially because I keep hearing it's awesome and I really want to try it. As far as setting up the environment, here's a rough idea of what I'll do:

```shell
conda create -n qwen-ft python=3.11 -y
conda activate qwen-ft

pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121
pip install unsloth
pip install --no-deps "trl<0.9.0" peft accelerate bitsandbytes
pip install datasets transformers huggingface_hub
```

And to start with, I'm not going to use my own data, I'll pick some public dataset, no idea which one yet, but I'll take suggestions from anyone reading this who knows a lot more than me.

It looks like most of the work is building the dataset. I won't go into too much detail here because honestly, I know very little about how to do this, still learning, so I'll have to report back once I do it.

What I do know is, once I have the dataset built, then I can start training. Once the training is done, then I can do some side-by-side comparisons with the base Qwen 2.5 3B Instruct, and see how it looks.

If I'm not totally embarrassed by the results, I can export a LoRA adapter, and GGUF, and publish on Huggingface. Of course, I don't think anyone is dying to use an optimized version of Qwen 2.5 3B Instruct, esp. created by a noob like me, but that's okay, I think it will just be fun to get something out there and document the process.

Okay, that's all I got, time for my second cup of coffee. I'd love feedback and advice so share what you've got. If you think I should pick a different model to start, change part of the process, etc. let me know, as you can tell, this is all new to me, I'm here to learn, and I learn best by doing.
