Difference between revisions of "Proxmox All-in-One"

From Da Nerd Mage Wiki
Jump to navigation Jump to search
(Created page with " * Install pfSense ([https://getlabsdone.com/how-to-install-pfsense-on-proxmox-step-by-step/ a link]) ** Set up a second network bridge internally (& give it a secondary physical NIC if you have one) ** Build the pfSense VM *** 8GB drive, 4 cores, 4096MB RAM *** Add a second network device (on the second bridge... duh...) ** Do a BACKUP ** boot the VM & run through the defaults ** Do a BACKUP *** Open the VM console & pretent you're building a normal pfSense router ****...")
 
 
(32 intermediate revisions by the same user not shown)
Line 1: Line 1:
= Installing PVE =
Start with the port you will want as your WAN port (primary) from pfSense connected to your network


* Install pfSense ([https://getlabsdone.com/how-to-install-pfsense-on-proxmox-step-by-step/ a link])
Install PVE
** Set up a second network bridge internally (& give it a secondary physical NIC if you have one)
 
** Build the pfSense VM
Fix repositories
*** 8GB drive, 4 cores, 4096MB RAM
* Add "pve-no-subscription"
*** Add a second network device (on the second bridge... duh...)
* Disable "pve-enterprise"
** Do a BACKUP
* Refresh updates
** boot the VM & run through the defaults
* Upgrade
** Do a BACKUP
 
*** Open the VM console & pretent you're building a normal pfSense router
== A Note about Clustering ==
**** Go into Services / DHCP Server / LAN
<span style="color: rgb(132, 63, 161);">(I have not yet fully verified whether this works or not... Watch for either this line or this entire sub-section to disappear.)</span>
***** Under Servers, add in your DNS server(s) address(es)
 
If you plan to add this server to a cluster, '''do it now'''. Once you install the first VM or LXC, Proxmox will not let you join a cluster.
 
<span style="color: rgb(132, 63, 161);">Bummer: So far, pfSense VM doesn't start because PVE is waiting for quorum in the cluster...&nbsp; :{{!}}</span>
 
Apparently, PVECM will not allow any VMs to start until it can contact the rest of the cluster and establish Quorate
 
It is quite possible to get it going from the console by logging in and running:
* <code>pvecm expected 1</code>
Sadly, still need to figure out the whole addressing issue.
 
= Configure your second Network Port =
Add a second Linux Bridge (This will be named '''vmbr1''')
* (Datacenter / Server ... System / Network)
* No addresses or gateways
* Assign your, as yet unused, second physical Ethernet port to this bridge
** For good measure, put something along the lines of "Local Network" in the comment for this one
** (You could edit vmbr0 & put "The Interwebs" in it's comment too...)
* <span style="color: rgb(132, 63, 161); font-size: 18pt;" >Do NOT Forget to '''Apply Configuration'''...</span>
 
= pfSense =
Installing [[pfSense{{!}}pfSense]] ([https://getlabsdone.com/how-to-install-pfsense-on-proxmox-step-by-step/ a link])
 
([[PfSense{{!}}More information about setting up pfSense]])
* Download the pfSense DVD ISO to your desktop
* Un-gzip it
* Upload it to the '''ISO Images''' folder on your '''Local''' datastore
* Build the pfSense VM
** ISO image: pfSense-CE-2.7.0-RELEASE-amd64.iso
** Guest OS Type: Other
** 8GB drive, 4 cores, 4096MB RAM
*** (I'd suggest setting Processor Type to '''host''')
** Use the original bridge (vmbr0) as the first network port & the new bridge (vmbr1) as the second port.
*** You'll need to go into Hardware for the VM to add in the second Network Device AFTER creating the VM.
*** For some reason, when you choose "other" as your OS type, PVE defaults the network device model to "Intel E1000".<br>This seems unreliable for pfSense. Select "VirtIO (paravirtualized)" instead.
** Configure the pfSense VM to start at boot.
*** go into Options for the VM...
*** Strongly reccomend setting it to boot FIRST & give a startup delay of at least a couple of minutes.
** <span style="color: rgb(186, 55, 42);">Do a BACKUP</span>
** Open the VM console & pretend you're building a normal pfSense router<br>
** '''<span style="color: rgb(186, 55, 42);">Do a BACKUP</span>'''<br>
 
= Management VM =
* Pick your favourite OS & build a VM <br><span style="color: rgb(132, 63, 161);">(Or... Ya know... Since ya gave that second network bridge a physical NIC (Ya did, right?)... You could just plug a computer in there.)</span>
** Point its network device at the second network bridge (vmbr1).
* '''<span style="color: rgb(186, 55, 42);">Do a BACKUP</span>'''
 
== Further VMs ==
'''Note:''' Any further VMs created on this server need to have their network interfaces on '''vmbr1''' or else they'll appear on the WAN port.
 
= pfSense Configuration =
 
(assumption: pfSense LAN network is the default of 192.168.1.0/24)
*Sign into '''<nowiki>https://192.168.1.1</nowiki>''' (from the Management VM) (or... [[PfSense#pfSense with_WAN_inside_a_LAN{{!}}If working inside your LAN]])
**In '''Services / DNS Resolver / General Settings''', under '''Host Overrides'''
***set up a DNS entry for PVE. (I like 192.168.1.2)
** In '''Firewall / NAT / Port Forward'''
*** set up port forwarding for the pfSense UI (port 443)
*** set up port forwarding for the PVE UI (port 8006)
*** set up port forwarding for SSH (port 22) to the Management VM (if used...)
** QEMU Guest Agent would be handy too...
*** [https://forum.netgate.com/topic/162083/pfsense-vm-on-proxmox-qemu-agent-installation PfSense VM on ProxMox : Qemu-agent installation]
*'''<span style="color: #ba372a;">Do a BACKUP</span>'''
'''Do note:''' These port forwards may be a security risk when you take the system live. They are here for ease of access while configuring the system.
 
= Taking it LIVE =
Up to this point, your server works fine on an internal network. Unfortunately, as far as the world outside the box is concerned, there are 2 machines there. The '''Proxmox''' install AND a '''pfSense''' install. They both show up on the network.
 
So...
 
Let's fix that.
 
* Sign into the physical machine (PVE)
** Make backups of the 2 files we're about to modify
*** <code>cp /etc/network/interfaces /etc/network/interfaces.BAK</code>
*** <code>cp /etc/hosts /etc/hosts.BAK</code>
** edit /etc/network/interfaces
*** Move the address & gateway from vmbr0 to vmbr1 and change them to those assigned for PVE on the pfSense VM.
** edit /etc/hosts
*** Change the address to that assigned for PVE on the pfSense VM<br>
Reboot the machine
 
Wait at least a couple minutes for pfSense to fully boot.
 
<span style="color: rgb(132, 63, 161);">'''At this point, the machine shows up on your network as a single device (The pfSense VM!)'''</span>
 
You can now browse to https://'''MachineAddress'''/ to access pfSense or https://'''MachineAddress''':8006/ to access the PVE UI to do further setup of the system.
 
(Where '''MachineAddress''' is the address or name assigned to it by your local network...)
 
Congratulations!&nbsp; You have built a router out of a server.&nbsp; If you plug the first Ethernet port into the Internet and connect a switch to the second Ethernet port, this machine can replace the router provided by your ISP...

Latest revision as of 14:26, 24 November 2023

Installing PVE

Start with the port you will want as your WAN port (primary) from pfSense connected to your network

Install PVE

Fix repositories

  • Add "pve-no-subscription"
  • Disable "pve-enterprise"
  • Refresh updates
  • Upgrade

A Note about Clustering

(I have not yet fully verified whether this works or not... Watch for either this line or this entire sub-section to disappear.)

If you plan to add this server to a cluster, do it now. Once you install the first VM or LXC, Proxmox will not let you join a cluster.

Bummer: So far, pfSense VM doesn't start because PVE is waiting for quorum in the cluster...  :|

Apparently, PVECM will not allow any VMs to start until it can contact the rest of the cluster and establish Quorate

It is quite possible to get it going from the console by logging in and running:

  • pvecm expected 1

Sadly, still need to figure out the whole addressing issue.

Configure your second Network Port

Add a second Linux Bridge (This will be named vmbr1)

  • (Datacenter / Server ... System / Network)
  • No addresses or gateways
  • Assign your, as yet unused, second physical Ethernet port to this bridge
    • For good measure, put something along the lines of "Local Network" in the comment for this one
    • (You could edit vmbr0 & put "The Interwebs" in it's comment too...)
  • Do NOT Forget to Apply Configuration...

pfSense

Installing pfSense (a link)

(More information about setting up pfSense)

  • Download the pfSense DVD ISO to your desktop
  • Un-gzip it
  • Upload it to the ISO Images folder on your Local datastore
  • Build the pfSense VM
    • ISO image: pfSense-CE-2.7.0-RELEASE-amd64.iso
    • Guest OS Type: Other
    • 8GB drive, 4 cores, 4096MB RAM
      • (I'd suggest setting Processor Type to host)
    • Use the original bridge (vmbr0) as the first network port & the new bridge (vmbr1) as the second port.
      • You'll need to go into Hardware for the VM to add in the second Network Device AFTER creating the VM.
      • For some reason, when you choose "other" as your OS type, PVE defaults the network device model to "Intel E1000".
        This seems unreliable for pfSense. Select "VirtIO (paravirtualized)" instead.
    • Configure the pfSense VM to start at boot.
      • go into Options for the VM...
      • Strongly reccomend setting it to boot FIRST & give a startup delay of at least a couple of minutes.
    • Do a BACKUP
    • Open the VM console & pretend you're building a normal pfSense router
    • Do a BACKUP

Management VM

  • Pick your favourite OS & build a VM
    (Or... Ya know... Since ya gave that second network bridge a physical NIC (Ya did, right?)... You could just plug a computer in there.)
    • Point its network device at the second network bridge (vmbr1).
  • Do a BACKUP

Further VMs

Note: Any further VMs created on this server need to have their network interfaces on vmbr1 or else they'll appear on the WAN port.

pfSense Configuration

(assumption: pfSense LAN network is the default of 192.168.1.0/24)

  • Sign into https://192.168.1.1 (from the Management VM) (or... If working inside your LAN)
    • In Services / DNS Resolver / General Settings, under Host Overrides
      • set up a DNS entry for PVE. (I like 192.168.1.2)
    • In Firewall / NAT / Port Forward
      • set up port forwarding for the pfSense UI (port 443)
      • set up port forwarding for the PVE UI (port 8006)
      • set up port forwarding for SSH (port 22) to the Management VM (if used...)
    • QEMU Guest Agent would be handy too...
  • Do a BACKUP

Do note: These port forwards may be a security risk when you take the system live. They are here for ease of access while configuring the system.

Taking it LIVE

Up to this point, your server works fine on an internal network. Unfortunately, as far as the world outside the box is concerned, there are 2 machines there. The Proxmox install AND a pfSense install. They both show up on the network.

So...

Let's fix that.

  • Sign into the physical machine (PVE)
    • Make backups of the 2 files we're about to modify
      • cp /etc/network/interfaces /etc/network/interfaces.BAK
      • cp /etc/hosts /etc/hosts.BAK
    • edit /etc/network/interfaces
      • Move the address & gateway from vmbr0 to vmbr1 and change them to those assigned for PVE on the pfSense VM.
    • edit /etc/hosts
      • Change the address to that assigned for PVE on the pfSense VM

Reboot the machine

Wait at least a couple minutes for pfSense to fully boot.

At this point, the machine shows up on your network as a single device (The pfSense VM!)

You can now browse to https://MachineAddress/ to access pfSense or https://MachineAddress:8006/ to access the PVE UI to do further setup of the system.

(Where MachineAddress is the address or name assigned to it by your local network...)

Congratulations!  You have built a router out of a server.  If you plug the first Ethernet port into the Internet and connect a switch to the second Ethernet port, this machine can replace the router provided by your ISP...

Retrieved from ‘https://wiki.nerdmage.ca/index.php?title=Proxmox_All-in-One&oldid=1909’