So I picked up a gaming laptop on Black Friday to use as an AI PC for running local LLMs, trying out image generation, agentic flows, and possibly video and speech processing. Additionally, I’ll be using it for non-AI tasks like video editing and general use.
I bought an MSI Cyborg 14 with an RTX 4060 featuring 8GB VRAM, an i7 processor, 512GB SSD, and 16GB RAM. It’s definitely a gaming PC, complete with features like a backlit keyboard. However, it weighs a manageable 1.6 kg and offers good hardware for its price (equivalent to $870 in Europe).
Stable Diffusion
I first tried image generation with Stable Diffusion. I started with Easy Diffusion, which has a one-click setup. It comes bundled with Stable Diffusion 1.4, which is a bit underwhelming. I downloaded “Absolute Realistic Vision” from https://civitai.com/, and it was a noticeable improvement in output quality. I attempted to use an SDXL model, but it didn’t work. I’ll try it again once I’ve upgraded to 64GB of RAM.
LLMs with Ollama
For local LLMs, I downloaded Ollama and installed Llama 3.1 8B using the command
ollama run llama3.1. The speed was impressive.
I then installed Docker and ran Open Web UI in a Docker container. The setup was streamlined and integrated well with the previously installed Ollama.
A few models I have tried
- Qwen 2.5 Coder 14B: Promising for code. Its speed is acceptable, but it’s the upper limit of what my system can currently handle.
- Llama 3.2 Vision 11B: Works fine and can interpret the images I’ve tested. Speed is okay.
- DeepSeek Coder 6.7B and Opencoder 8B: High speed, but the quality is, unsurprisingly, below that of Qwen 2.5 Coder 14B. It’s fairer to compare them with the 7B version, but I’m running the highest version I can.
Gaming
I installed and ran Star Wars Battlefront 2 with default settings and a 1920×1200 resolution, and it worked fine. I haven’t tried tweaking it yet and will probably uninstall it due to its size of over 70GB versus how often I will play. It’s good to know the computer can run it, though.
Next step
I have ordered 64GB of DDR5 RAM, which is the maximum for my system. This will hopefully enable me to run Stable Diffusion SDXL and larger LLMs like Qwen 2.5 Coder 32B. I may also be able to run 70B models like Llama 3.3 70B through extended/shared GPU or by using settings from LM Studio/llama.cpp.
I will post an update once the RAM upgrade pushes the boundaries of what my system can do.