Running QEMU with port redirection through libvirt

Like many developers these days, I use a bunch of different virtual machine images to give me access to multiple development and test environments without having a room full of computers. My VM of choice is QEMU KVM, the kernel-based virtual machine bundled with Fedora. I’ve always used simple shell scripts to start my VMs, which has allowed me great flexibility, but yesterday I set one up using the nice Virtual Machine Manager (VMM) GUI application. And remembered why I wasn’t using it: there’s no way to tell it how to do TCP port redirection! But now, there is a way…

The main reason I want TCP port redirection is so that I can use RDP to access Windows. RDP gives a much faster, smoother GUI experience than the standard VNC connection does, so I’ve set up all of my Windows VMs to use RDP. I remember now that I looked at this a couple of years ago, and there was no way to tell libvirt, the library underneath VMM, that I wanted TCP redirection. Thus, I stuck with my trusty shell scripts.

Looking again today, I was excited to find a post by Stefan Hajnoczi about passing QEMU command line arguments through libvirt. This is just what I need!

So, to the command line! First, take a backup of the configuration file for my VM. On Fedora, the files are in /etc/libvirt/qemu. Next, the message at the top of the file tells you in no uncertain terms to make edits using virsh edit or similar, so:

virsh edit my-vm-name

The first line needs to be changed, so that it uses the XML namespace for the QEMU command line elements:

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>

Next, the network type has to be changed. TCP port redirection is a feature of QEMU user-mode networking, and the VMM configures tap networks by default. So, I have to change this line:

<interface type='network'>

to this:

<interface type='user'>

And finally, I need to add my custom command line arguments. I stuck them at the bottom of the file, just before the closing tag for domain:

<qemu:commandline>
   <qemu:arg value='-redir'/>
   <qemu:arg value='tcp:5564::3389'/>
</qemu:commandline>

Job is done; faster, smoother, and remotely.

Facebooktwittergoogle_plusredditlinkedinmailFacebooktwittergoogle_plusredditlinkedinmail
  • Thanks a lot for this trick which is just working fine ! I was trying to use the new hostfwd option, but was unable to make it work with the same conf compared to what you did with -redir.

    Using:

    or adding the user param before the hostfwd:

    Which is working on CLI doesn’t with libvirt.

    I always get monitor related errors:Unable to read from monitor: Connexion ré-initialisée par le correspondant

    Traceback (most recent call last):
    File “/usr/share/virt-manager/virtManager/asyncjob.py”, line 45, in cb_wrapper
    callback(asyncjob, *args, **kwargs)
    File “/usr/share/virt-manager/virtManager/asyncjob.py”, line 66, in tmpcb
    callback(*args, **kwargs)
    File “/usr/share/virt-manager/virtManager/domain.py”, line 1114, in startup
    self._backend.create()
    File “/usr/lib64/python2.7/site-packages/libvirt.py”, line 606, in create
    if ret == -1: raise libvirtError (‘virDomainCreate() failed’, dom=self)
    libvirtError: Unable to read from monitor: Connexion ré-initialisée par le correspondant

    Anyway you’re proposal to use -redir, as long as it’s supportd I think, is a good workaround !

    • Sadly, it looks like the comment system ate your code example, but I’m sure it doesn’t matter. I too tried with hostfwd and failed, for some reason. So I have stuck with -redir for now.

      cheers,
      Ross

      • chenwj

        Work for me.

  • kvmuser

    Hi,

    thanks for this tip! Tried to forward port 80 ion the host to port 80 on the guest, but this does not work:

    Error: internal error process exited while connecting to monitor: char device redirected to /dev/pts/1
    kvm: -netdev user,id=hostnet0: could not set up host forwarding rule ‘tcp:80::80’
    kvm: -netdev user,id=hostnet0: Device ‘user’ could not be initialized

    All the other port redirects I tried are working, but not port 80 (although no webserver is running on the host).

    • G’day, sounds like something else is using port 80 on your system. Run the following command to see if you can track it down.

      lsof -i tcp:80

      Alternatively, you might want to try running your VM via a tap device, which allows the VM to run on the network with its own IP address and with all ports accessible.

      http://wiki.qemu.org/Features/HelperNetworking
      https://en.wikibooks.org/wiki/QEMU/Networking#TAP_interfaces

      cheers,
      Ross

      • kvmuser

        Hi,
        thanks for the quick reply. lsof did not return anything as expected. Port 80 is not in use on the host. Could it be that the privileged ports 0-1024 cannot be forwarded? Or could it be a problem with missing rights (although I am root when trying to make this port forward).

        With tun/tap: Do I need then a public IP for the guest? This is what I want to avoid…

        Thanks for your help!

        • G’day, maybe it’s in your iptables. I can’t really think of anything else.

          With tun/tap, you could set it up with an IP address local to your network, e.g. 192.168.0.x etc. You’re getting into Linux networking at that point, not my strong suit. It would be possible to firewall it off from accepting anything other than port 80 however.

          Simpler still is just redirecting 8080 to the VM’s 80. You can keep the VM on QEMU user mode networking and still access the webserver on the VM, just append the port to the URL, e.g. http://localhost:8080/

          cheers,
          Ross

          • kvmuser

            There are no port-related iptables rules on the host:

            Chain INPUT (policy ACCEPT)
            target prot opt source destination
            ACCEPT udp -- anywhere anywhere udp dpt:domain
            ACCEPT tcp -- anywhere anywhere tcp dpt:domain
            ACCEPT udp -- anywhere anywhere udp dpt:bootps
            ACCEPT tcp -- anywhere anywhere tcp dpt:bootps

            Chain FORWARD (policy ACCEPT)
            target prot opt source destination
            ACCEPT all -- anywhere 192.168.122.0/24 state RELATED,ESTA BLISHED
            ACCEPT all -- 192.168.122.0/24 anywhere
            ACCEPT all -- anywhere anywhere
            REJECT all -- anywhere anywhere reject-with icmp-p ort-unreachable
            REJECT all -- anywhere anywhere reject-with icmp-p ort-unreachable

            Chain OUTPUT (policy ACCEPT)
            target prot opt source destination

            I tried already with 8080 which is working, but it is kind of uncomfortable for visitors to the site… nobody will know that 8080 has to be added to the URL…

            Anyway, thanks for helping.

  • Staticn0de

    Hi there,

    I followed this guide and it worked a treat.

    Just wondering, can I forward several ports at the same time? do you have an example of how I would add:

    With several ports? I have it working with ssh at the moment.

    Cheers

    • I imagine that you can keep passing arguments, like this:

      <qemu:commandline>
      <qemu:arg value='-redir'/>
      <qemu:arg value='tcp:5564::3389'/>
      <qemu:arg value='-redir'/>
      <qemu:arg value='tcp:5565::80'/>
      </qemu:commandline>

      cheers,
      Ross

      • Staticn0de

        I’ll give that a go. Thanks for the fast reply.

      • Staticn0de

        Not sure why but it wont forward to my webserver.

        From inside the VM, I can navigate to 192.168.122.233 (VM IP and IP of webserver) and it connects to the webpage.

        When I forward to SSH using the method shown above, it works.

        When I forward as you have done above:

        It times out.

        Any idea why SSH would work and web wont?

        • I can’t see any of your example code, the comment system in WordPress is stripping it. Perhaps post it on Pastebin or an alternative.

          A simple diagnosis of your problem could be had by looking at the arguments given to qemu when you launch it from virsh:

          ps ax | grep qemu

          See if it is actually passing those arguments.

          cheers,
          Ross

          • Staticn0de

            Hi,

            Thanks for your help.

            Here is the pastebin of my VM.xml file. At the bottom you can see how I have forwarded the port.

            http://pastebin.com/DEMTy2RU

            Here are the results of ps ax | grep qemu. Looks like the commands are being passed.
            http://pastebin.com/qTniQRz1

            To connect to the VM, I am using 192.168.0.50:5565 (The Host IP) and that connection is timing out. Normally I could use 192.168.0.50:5564 to access SSH on the VM but I have disabled that while I was looking into what the issue was.

          • The kvm args look like they’re specifing a virtio network, which is in conflict with user mode networking. Simplify the interface block in your xml to just have the mac and address elements.

            cheers,
            Ross

      • Staticn0de

        Thanks Ross,

        I had a read here https://en.wikibooks.org/wiki/QEMU/Networking#User_mode_networking and changed my network adapter to an E1000.

        Still didn’t work.

        Changed the port I was forwarding from 5565::80 to 10133:80 and it worked.

        No idea why the change fixed it.

  • Vignette Hyena

    I love you.

    I’ve been looking all over the place for this solution.

  • Duron

    thanks, port forwarded!

    but, vms can not visit the internet, how to resolve?

  • kourampies

    Thanks, this guide works perfectly! Its strange that virt-manager has no wizard for this, but now I have a solution. Thanks!

  • Alistair Bill

    Thank you ever so much. Pointers – in virt-manager you must be using the qemu user session, not the system session, for this to work properly.