Difference between revisions of "Proxmox All-in-One"

From Da Nerd Mage Wiki
Jump to navigation Jump to search
 
(19 intermediate revisions by the same user not shown)
Line 1: Line 1:
= Installing PVE =
= Installing PVE =
Start with the port you will want as your LAN port (secondary) from pfSense connected to your network
Start with the port you will want as your WAN port (primary) from pfSense connected to your network


Install PVE
Install PVE
Line 9: Line 9:
* Refresh updates
* Refresh updates
* Upgrade
* Upgrade
Add a second Linux Bridge
 
== A Note about Clustering ==
<span style="color: rgb(132, 63, 161);">(I have not yet fully verified whether this works or not... Watch for either this line or this entire sub-section to disappear.)</span>
 
If you plan to add this server to a cluster, '''do it now'''. Once you install the first VM or LXC, Proxmox will not let you join a cluster.
 
<span style="color: rgb(132, 63, 161);">Bummer: So far, pfSense VM doesn't start because PVE is waiting for quorum in the cluster...&nbsp; :{{!}}</span>
 
Apparently, PVECM will not allow any VMs to start until it can contact the rest of the cluster and establish Quorate
 
It is quite possible to get it going from the console by logging in and running:
* <code>pvecm expected 1</code>
Sadly, still need to figure out the whole addressing issue.
 
= Configure your second Network Port =
Add a second Linux Bridge (This will be named '''vmbr1''')
* (Datacenter / Server ... System / Network)
* No addresses or gateways
* No addresses or gateways
* Assign your, as yet unused, second physical Ethernet port to this bridge
* Assign your, as yet unused, second physical Ethernet port to this bridge
** For good measure, put something along the lines of "Local Network" in the comment for this one
** (You could edit vmbr0 & put "The Interwebs" in it's comment too...)
* <span style="color: rgb(132, 63, 161); font-size: 18pt;" >Do NOT Forget to '''Apply Configuration'''...</span>


= pfSense =
= pfSense =
Installing [[pfSense{{!}}pfSense]] ([https://getlabsdone.com/how-to-install-pfsense-on-proxmox-step-by-step/ a link])
Installing [[pfSense{{!}}pfSense]] ([https://getlabsdone.com/how-to-install-pfsense-on-proxmox-step-by-step/ a link])
([[PfSense{{!}}More information about setting up pfSense]])
* Download the pfSense DVD ISO to your desktop
* Un-gzip it
* Upload it to the '''ISO Images''' folder on your '''Local''' datastore
* Build the pfSense VM
* Build the pfSense VM
** ISO image: pfSense-CE-2.7.0-RELEASE-amd64.iso
** Guest OS Type: Other
** 8GB drive, 4 cores, 4096MB RAM
** 8GB drive, 4 cores, 4096MB RAM
** Use the second bridge (vmbr1) as the first network port & the original (vmbr0) as the second port
*** (I'd suggest setting Processor Type to '''host''')
** Use the original bridge (vmbr0) as the first network port & the new bridge (vmbr1) as the second port.
*** You'll need to go into Hardware for the VM to add in the second Network Device AFTER creating the VM.
*** For some reason, when you choose "other" as your OS type, PVE defaults the network device model to "Intel E1000".<br>This seems unreliable for pfSense. Select "VirtIO (paravirtualized)" instead.
** Configure the pfSense VM to start at boot.
** Configure the pfSense VM to start at boot.
*** go into Options for the VM...
*** Strongly reccomend setting it to boot FIRST & give a startup delay of at least a couple of minutes.
*** Strongly reccomend setting it to boot FIRST & give a startup delay of at least a couple of minutes.
** <span style="color: rgb(186, 55, 42);">Do a BACKUP</span>
** <span style="color: rgb(186, 55, 42);">Do a BACKUP</span>
** Open the VM console & pretend you're building a normal pfSense router
** Open the VM console & pretend you're building a normal pfSense router<br>
** Once the VM is booted into pfSense...
** '''<span style="color: rgb(186, 55, 42);">Do a BACKUP</span>'''<br>
** '''<span style="color: rgb(186, 55, 42);">Do a BACKUP</span>'''
** Then move on to:


= Management VM =
= Management VM =
* Pick your favourite OS & build a VM <span style="color: rgb(132, 63, 161);">(Or... Ya know... Since ya gave that second network bridge a physical NIC (Ya did, right?)... You could just plug a computer in there.)</span>
* Pick your favourite OS & build a VM <br><span style="color: rgb(132, 63, 161);">(Or... Ya know... Since ya gave that second network bridge a physical NIC (Ya did, right?)... You could just plug a computer in there.)</span>
** Point its network device at the second network bridge
** Point its network device at the second network bridge (vmbr1).
** Sign into '''<nowiki>https://192.168.0.1</nowiki>''' (from the Management VM)
** In '''Services / DHCP Server / LAN'''
*** Under Servers, add in your DNS server(s) address(es)
** Restart networking on the Management VM
** Feed access to this VM through the pfSense firewall
* '''<span style="color: rgb(186, 55, 42);">Do a BACKUP</span>'''
* '''<span style="color: rgb(186, 55, 42);">Do a BACKUP</span>'''
== Further VMs ==
'''Note:''' Any further VMs created on this server need to have their network interfaces on '''vmbr1''' or else they'll appear on the WAN port.


= pfSense Configuration =
= pfSense Configuration =


*Sign into '''<nowiki>https://192.168.0.1</nowiki>''' (from the Management VM)
(assumption: pfSense LAN network is the default of 192.168.1.0/24)
*In '''Services / DNS Resolver / General Settings''', under '''Host Overrides'''
*Sign into '''<nowiki>https://192.168.1.1</nowiki>''' (from the Management VM) (or... [[PfSense#pfSense with_WAN_inside_a_LAN{{!}}If working inside your LAN]])
**set up a DNS entry for PVE
**In '''Services / DNS Resolver / General Settings''', under '''Host Overrides'''
* In '''Firewall / NAT / Port Forward'''
***set up a DNS entry for PVE. (I like 192.168.1.2)
** set up port forwarding for the pfSense UI (port 443)
** In '''Firewall / NAT / Port Forward'''
** set up port forwarding for the PVE UI (port 8006)
*** set up port forwarding for the pfSense UI (port 443)
** set up port forwarding for SSH (port 22) to the Management VM (if used...)
*** set up port forwarding for the PVE UI (port 8006)
*** set up port forwarding for SSH (port 22) to the Management VM (if used...)
** QEMU Guest Agent would be handy too...
*** [https://forum.netgate.com/topic/162083/pfsense-vm-on-proxmox-qemu-agent-installation PfSense VM on ProxMox : Qemu-agent installation]
*'''<span style="color: #ba372a;">Do a BACKUP</span>'''
*'''<span style="color: #ba372a;">Do a BACKUP</span>'''
'''Do note:''' These port forwards may be a security risk when you take the system live. They are here for ease of access while configuring the system.


= Taking it LIVE =
= Taking it LIVE =
Line 55: Line 85:


* Sign into the physical machine (PVE)
* Sign into the physical machine (PVE)
** Make backups of the 2 files we're about to modify
*** <code>cp /etc/network/interfaces /etc/network/interfaces.BAK</code>
*** <code>cp /etc/hosts /etc/hosts.BAK</code>
** edit /etc/network/interfaces
** edit /etc/network/interfaces
*** Change vmbr0 to vmbr1 & vmbr1 to vmbr0
*** Move the address & gateway from vmbr0 to vmbr1 and change them to those assigned for PVE on the pfSense VM.
*** Change the address & gateway (of what is now vmbr1) to those assigned for PVE on the pfSense VM
** edit /etc/hosts
** edit /etc/hosts
*** Change the address to that assigned for PVE on the pfSense VM
*** Change the address to that assigned for PVE on the pfSense VM<br>
Reboot the machine
Reboot the machine


Line 70: Line 102:
(Where '''MachineAddress''' is the address or name assigned to it by your local network...)
(Where '''MachineAddress''' is the address or name assigned to it by your local network...)


<br>
Congratulations!&nbsp; You have built a router out of a server.&nbsp; If you plug the first Ethernet port into the Internet and connect a switch to the second Ethernet port, this machine can replace the router provided by your ISP...

Latest revision as of 14:26, 24 November 2023

Installing PVE

Start with the port you will want as your WAN port (primary) from pfSense connected to your network

Install PVE

Fix repositories

  • Add "pve-no-subscription"
  • Disable "pve-enterprise"
  • Refresh updates
  • Upgrade

A Note about Clustering

(I have not yet fully verified whether this works or not... Watch for either this line or this entire sub-section to disappear.)

If you plan to add this server to a cluster, do it now. Once you install the first VM or LXC, Proxmox will not let you join a cluster.

Bummer: So far, pfSense VM doesn't start because PVE is waiting for quorum in the cluster...  :|

Apparently, PVECM will not allow any VMs to start until it can contact the rest of the cluster and establish Quorate

It is quite possible to get it going from the console by logging in and running:

  • pvecm expected 1

Sadly, still need to figure out the whole addressing issue.

Configure your second Network Port

Add a second Linux Bridge (This will be named vmbr1)

  • (Datacenter / Server ... System / Network)
  • No addresses or gateways
  • Assign your, as yet unused, second physical Ethernet port to this bridge
    • For good measure, put something along the lines of "Local Network" in the comment for this one
    • (You could edit vmbr0 & put "The Interwebs" in it's comment too...)
  • Do NOT Forget to Apply Configuration...

pfSense

Installing pfSense (a link)

(More information about setting up pfSense)

  • Download the pfSense DVD ISO to your desktop
  • Un-gzip it
  • Upload it to the ISO Images folder on your Local datastore
  • Build the pfSense VM
    • ISO image: pfSense-CE-2.7.0-RELEASE-amd64.iso
    • Guest OS Type: Other
    • 8GB drive, 4 cores, 4096MB RAM
      • (I'd suggest setting Processor Type to host)
    • Use the original bridge (vmbr0) as the first network port & the new bridge (vmbr1) as the second port.
      • You'll need to go into Hardware for the VM to add in the second Network Device AFTER creating the VM.
      • For some reason, when you choose "other" as your OS type, PVE defaults the network device model to "Intel E1000".
        This seems unreliable for pfSense. Select "VirtIO (paravirtualized)" instead.
    • Configure the pfSense VM to start at boot.
      • go into Options for the VM...
      • Strongly reccomend setting it to boot FIRST & give a startup delay of at least a couple of minutes.
    • Do a BACKUP
    • Open the VM console & pretend you're building a normal pfSense router
    • Do a BACKUP

Management VM

  • Pick your favourite OS & build a VM
    (Or... Ya know... Since ya gave that second network bridge a physical NIC (Ya did, right?)... You could just plug a computer in there.)
    • Point its network device at the second network bridge (vmbr1).
  • Do a BACKUP

Further VMs

Note: Any further VMs created on this server need to have their network interfaces on vmbr1 or else they'll appear on the WAN port.

pfSense Configuration

(assumption: pfSense LAN network is the default of 192.168.1.0/24)

  • Sign into https://192.168.1.1 (from the Management VM) (or... If working inside your LAN)
    • In Services / DNS Resolver / General Settings, under Host Overrides
      • set up a DNS entry for PVE. (I like 192.168.1.2)
    • In Firewall / NAT / Port Forward
      • set up port forwarding for the pfSense UI (port 443)
      • set up port forwarding for the PVE UI (port 8006)
      • set up port forwarding for SSH (port 22) to the Management VM (if used...)
    • QEMU Guest Agent would be handy too...
  • Do a BACKUP

Do note: These port forwards may be a security risk when you take the system live. They are here for ease of access while configuring the system.

Taking it LIVE

Up to this point, your server works fine on an internal network. Unfortunately, as far as the world outside the box is concerned, there are 2 machines there. The Proxmox install AND a pfSense install. They both show up on the network.

So...

Let's fix that.

  • Sign into the physical machine (PVE)
    • Make backups of the 2 files we're about to modify
      • cp /etc/network/interfaces /etc/network/interfaces.BAK
      • cp /etc/hosts /etc/hosts.BAK
    • edit /etc/network/interfaces
      • Move the address & gateway from vmbr0 to vmbr1 and change them to those assigned for PVE on the pfSense VM.
    • edit /etc/hosts
      • Change the address to that assigned for PVE on the pfSense VM

Reboot the machine

Wait at least a couple minutes for pfSense to fully boot.

At this point, the machine shows up on your network as a single device (The pfSense VM!)

You can now browse to https://MachineAddress/ to access pfSense or https://MachineAddress:8006/ to access the PVE UI to do further setup of the system.

(Where MachineAddress is the address or name assigned to it by your local network...)

Congratulations!  You have built a router out of a server.  If you plug the first Ethernet port into the Internet and connect a switch to the second Ethernet port, this machine can replace the router provided by your ISP...

Retrieved from ‘https://wiki.nerdmage.ca/index.php?title=Proxmox_All-in-One&oldid=1909’