For years I’ve had a dream of building a rack mounted PC capable of splitting its resources to host multiple GPU intensive VMs:

  • a few gaming VMs
  • a VM for work that can run Davinci Resolve and Blender renders
  • an LLM server
  • a Stable Diffusion server
  • media server

Just to name a few possibilities…

Everytime I’ve looked into it, it seemed like the technology just wasn’t there yet. I remember a few years ago Linus TT took a shot at it, but in the end suggested the technology (for non-commercial entities) just wasn’t in a comfortable spot yet.

So how far off are we? Obviously AI focused companies seem to make it work, but what possibilities exist for us self-hosters who might also want to run multiple displays in addition to the web gui LLM servers? And without forking out crazy money for GPU virtualization software licenses?

  • yggstyle@lemmy.world
    link
    fedilink
    English
    arrow-up
    3
    ·
    edit-2
    6 months ago

    The two workstation nooks (spaces) have the capability to have a second monitor but I’ve since retired them in favor of ultrawide monitors which I find are a better experience in general. My current working solution is a split between two technologies: one thin client (second monitors) and one network distribution solution using multicast (primary displays and USB). Both run on copper 1 gig but the multicast traffic requires a switch that doesn’t suck and vlan usage. On average a single port can reach 70-85% usage sustained. I believe my longest run is 150’ ish.

    Cost per node is roughly 300- so comparable to what you are experiencing. If I went stupid cheap I could probably cut that to maybe 150-250 depending on my luck with eBay and patience.

    In terms of capabilities you could argue that this could be done without distribution using a nuc solution… but you’d have to split resources to reach node you’d need a full feature set at.

    My central server is a threadripper build with 2 gpus for direct passthrough to ‘gaming’ vms and a split gpu handling the rest of the needs of the other systems. Thanks to the matrix capabilities any given seat can be any system… or in some cases 2 seats can be a single rig (2 room gaming off the same display). There is a cost savings to be found in splitting resources from a more expensive build out to cheaper nodes… but ymmv depending on active seats and specific needs. I believe as a general rule it should be less costly and more efficient (power/heat) than individual solutions.

    • Takumidesh@lemmy.world
      link
      fedilink
      English
      arrow-up
      2
      ·
      6 months ago

      Thanks for the breakdown! This is probably the most helpful breakdown I’ve seen of a build like this.

      • yggstyle@lemmy.world
        link
        fedilink
        English
        arrow-up
        2
        ·
        6 months ago

        Absolutely 👍. I’ll just add that there are a lot of alternate routes to get the result you want so research and experiment but ideally set a deadline which can help with decision paralysis. Later changes are a problem for future you 😁.