OVH UK Bare-Metal Proxmox Additional IP’s and Enhance Control Panel

Introduction

This article talks about deploying the Enhance Control Panel using Proxmox on OVH’s bare-metal instances in their UK datacenter with additional IP’s. Wow, that’s a mouth full.

Virtualization and OVH bare-metal Instances with Additional IP’s

When you order a bare-metal instance in OVH’s UK data center with additional IPs, and want to use virtulzation such as Proxmox. OVH UK’s data-center only supports internal addresses behind a bridge with the src option used for routing on your guest VM’s. They do not support virtual macs which other OVH data-centers do support. The UK data center is suppose to support virtual macs in December 2024

When you try to add a virtual mac address to an IP Address you will receive the following message.

An error occurred on generating a new virtual MAC on the IP 51.89.x.x
The virtual MAC address will be available on this server from December 2024
For more information on managing IPs and virtual MACs with OVHcloud, please refer to the Network Configuration guides, here: Proxmox VE and here: Windows Server. This guide provides instructions for configuring your OVHcloud servers to optimise network performance in virtualised environments with Proxmox VE. This is done using techniques like routing additional IPs via the vRack​.

It links to the following article, which goes in-depth into setting up source routing. Please refer to “ADVANCE range” examples.

Configuring the network on Proxmox VE on the High Grade, Scale & Advance ranges – OVHcloud

Find out how to configure the network on Proxmox VE

help.ovhcloud.com

Why must I use internal address for my guests/virtual machines?

Typically OVH would lock an IP to a mac-address, of which then you then move the mac address to different machines and use your IP Address. This also stops other OVH customers from taking over your IP.

Unfortunately this isn’t support in the UK DC, instead you have to use private internal addressing such as 192.168.0.0/24 for each of your guest instances within your virtualization hypervisor such as Proxmox. And custom networking configuration on each guest instance.

How do I setup my bare-metal server using Proxmox?

Using the image provided in the article above, this is how your network should look.

Here’s an example Proxmox /etc/network/interfaces file using the following.

• Bare-metal host IP: 57.128.1.1/32
• Bare-metal host Gateway: 100.64.0.1
• Additional IP’s: add 51.89.2.1/29 ( 8 Total)

auto lo
iface lo inet loopback

auto enp1s0f0np0
iface enp1s0f0np0 inet static
    address 57.128.1.1/32
    gateway 100.64.0.1

iface enp1s0f1np1 inet manual

iface enx5e929256fa0e inet manual

auto vmbr0
iface vmbr0 inet static
    address 192.168.0.1/24
    bridge-ports none
    bridge-stp off
    bridge-fd 0
    mtu 1500
    up ip route add 51.89.2.1/29 dev vmbr0

post-up iptables -t nat -A POSTROUTING -s 192.168.0.2 -o enp1s0f0np0 -j SNAT --to-source 51.89.2.1
pre-down iptables -t nat -D POSTROUTING -s 192.168.0.2 -o enp1s0f0np0 -j SNAT --to-source 51.89.2.1

Here’s a breakdown of the above configuration

• The enp1s0f0np0 interface is the internet facing connection from OVH with an IP address 57.128.1.1/32 and gateway of 100.64.0.1
• The vmbr0 bridge has no interfaces and the IP address 192.168.0.1/24
• There is a route under vmbr0 for 51.89.2.1/29 which are the additional IP’s, these IP’s are to be routed to vmbr0
• The last two iptables entries are important for later, if the IP 192.168.0.2 is seen on any packets on the interface enp1s0f0np0 change the source IP to 51.89.2.1 on the packets.

How do I configure my Proxmox guests/virtual machines?

There is an example in the OVH guide I linked above, but here is an example

network:
    ethernets:
        eth0:
            addresses:
            - 51.89.2.1/32
            - 192.168.0.2/24
            match:
                macaddress: bc:24:11:ca:87:1a
            nameservers:
                addresses:
                - 8.8.8.8
                - 1.1.1.1
            routes:
            -   from: 51.89.2.1
                to: default
                via: 192.168.0.1
            set-name: eth0
    version: 2

Here’s a breakdown of the above configuration

• We’re setting up eth0 to have both address 51.89.2.1/32 and 192.168.0.2/24
• We’re matching the mac address bc:24:11:ca:87:1a for this interface. This is the randomly generated mac address proxmox gives you.
• We’re creating a default route that sends all traffic to 192.168.0.1 (Proxmox host) and changing the src address to 51.89.2.1 so it doesn’t use 192.168.0.2

What about Enhance Control Panel?

Everything in Enhance works fine out of the box, except for local users and websites.

Enhance User Namespaces

The Enhance architecture uses what are called user namespaces, which are linux containers for each user on the system (above 2048 uid). This locks down the container, and as such there is no network route to make sure any packets coming from a user has the src set properly (in this example 51.89.2.1).

So when you try and update your site, you might get DNS or network errors, if you su – into a user account, you’ll notice you can’t ping the internet. This is because the packet from the Proxmox guest is reaching the Proxmox host and the src address is 192.168.0.2 and so it never reaches the internet.

Changing the source IP Address

How do we fix this? We just need to use iptables and create a source nat rule. If you recall above, I provided a Proxmox host network configuration and it had some iptables rules. Here are those rules again.

post-up iptables -t nat -A POSTROUTING -s 192.168.0.2 -o enp1s0f0np0 -j SNAT --to-source 51.89.2.1
pre-down iptables -t nat -D POSTROUTING -s 192.168.0.2 -o enp1s0f0np0 -j SNAT --to-source 51.89.2.1

These rules will make sure that if a packet is seen on interface enp1s0f0np0 with a src IP of 192.168.0.2, it will transform the IP to 51.89.2.1

That’s it, your website should be able to access the internet!