Proxmox All-in-One Guru Notes

Part 1

• TheGuruOfNothing — 2022-07-12 at 11:03

Ok, wiping the PfSense VM and building a new one (for simplicity)
Already hit the wall

Being as I have never used more than one port on any of my servers, I am not sure how to set up and utilize the other ports 
since it defaults to whatever it defaults to

• Tinker — 2022-07-12 at 11:09

Step 1...
go to the Web UI of the server...
Select the server itself...
See System/Network in the second column?

• TheGuruOfNothing — 2022-07-12 at 11:14

yep

• Tinker — 2022-07-12 at 11:15

See at least one each of "Network Device" & "Linux Bridge"?

• TheGuruOfNothing — 2022-07-12 at 11:15

yep

• Tinker — 2022-07-12 at 11:15

The Network Device marked as Active is the one you're talking to it through...
Same for the active bridge.
See a second (or more...) "Network Device"?

• TheGuruOfNothing — 2022-07-12 at 11:18

Sorry... phone call
Im back

• Tinker — 2022-07-12 at 11:18

Phone calls happen...
So, see a second Network Device?

• TheGuruOfNothing — 2022-07-12 at 11:19

• Tinker — 2022-07-12 at 11:20

Thare ya go...

• TheGuruOfNothing — 2022-07-12 at 11:20

vmbr0 is using eno1

• Tinker — 2022-07-12 at 11:20

eno1 is the NIC you have connected ATM...

• TheGuruOfNothing — 2022-07-12 at 11:20

I have cables connected to both actually

• Tinker — 2022-07-12 at 11:20

& vmbr0 is the internal connection to it that all of the VMs see.
eno2 isn't actually doing anything tho.

• TheGuruOfNothing — 2022-07-12 at 11:21

correct

• Tinker — 2022-07-12 at 11:22

If it's working like I think it is... They are in the same order as the physical NIC are listed on the machine.
For the moment, I'd just unplug the cable from the second one.
Simplify life...
Yer gonna swap them later anyhow.

• TheGuruOfNothing — 2022-07-12 at 11:23

lemme run down there

• Tinker — 2022-07-12 at 11:23

Exercise time!

• TheGuruOfNothing — 2022-07-12 at 11:30

I was mistaken
I have two cables going to each server ...
one to a NIC and one to iLO
So... we good

• Tinker — 2022-07-12 at 11:31

All-Righty then...
Step 2...
See the "Create" button?
Click it.

• TheGuruOfNothing — 2022-07-12 at 11:32

yep
yeop

• Tinker — 2022-07-12 at 11:32

You wanna make a new "Linux Bridge"
Only thing you want to fill in is the Bridge ports box.
tell it "eno2"
Once you tell it to go ahead & create it, poke the "Apply Configuration" button.

• TheGuruOfNothing — 2022-07-12 at 11:34

yep

• Tinker — 2022-07-12 at 11:35

You now have 2 network ports available to your shiny new pfSense VM...

• TheGuruOfNothing — 2022-07-12 at 11:35

SO...
I create the VM, tell it to put WAN on vmbr1 and LAN on vmbr0

• Tinker — 2022-07-12 at 11:36

nope...

• TheGuruOfNothing — 2022-07-12 at 11:36

lol

• Tinker — 2022-07-12 at 11:36

WAN goes on vmbr0 during construction.
& LAN on vmbr1.
When you get to the fugly step...
https://wiki.nerdmage.ca/index.php/Proxmox_All-in-One#Taking_it_LIVE
Proxmox All-in-One
You'll be swapping the bridges.

• TheGuruOfNothing — 2022-07-12 at 11:37

Fer giggles... why can't ya build it the other way around?
A way of inserting a PfSense box into an existing server

• Tinker — 2022-07-12 at 11:38

Coz then you'll have a VERY hard time reaching the machine through the unconfigured pfSense VM...

• TheGuruOfNothing — 2022-07-12 at 11:38

ah, ok
forgot that ya hafta have a console to config on

• Tinker — 2022-07-12 at 11:39

The bridge swap puts the management port of the PVE box itself behind the pfSense server & kinda hides it from the real world.
Just had a thought...
Your local network... It contains the 192.168.1.xxx range?

• TheGuruOfNothing — 2022-07-12 at 11:40

yes
.0.xxx -.3.xxx
/22

• Tinker — 2022-07-12 at 11:41

If so... You may have to do an extra step while setting up pfSense.
Basically, you'll need to tell it to use a range that's not on your local LAN.
It might figure it out itself, but I'm not sure.

• TheGuruOfNothing — 2022-07-12 at 11:42

Not thinking so
Ok, so I am not mucking this about...
I just had a thought...........
hold on

• Tinker — 2022-07-12 at 11:45

Yer workin on a trick question... aren't you...

• TheGuruOfNothing — 2022-07-12 at 11:45

Well, I was
then I found my own flaw
Ya can't access PfSense from the WAN side of the setup

• Tinker — 2022-07-12 at 11:46

nope...
But telling your Management VM to use vmbr1 means you can go to its console & play...
Another reason you need PVE to stay accessible during the build.

• TheGuruOfNothing — 2022-07-12 at 11:47

I can build the VM and tell it to use vmbr1 as it's WAN port and vmbr0 as the LAN and use the console to access it.... right up  until I save the config then EVERYTHING on it will go dead stick because LAN will be dicked as long as it is connected to my network
I might be able to connect vmbr0 to a seperate switch and use a laptop to access it at that point however
it would then be a standalone server but I don't know if I can access PVE again though

• Tinker — 2022-07-12 at 11:49

WAN has to stay on vmbr0 until pfSense is ready to take over.

• TheGuruOfNothing — 2022-07-12 at 11:49

yeah... was spitballing
wrapping my head around all the option

• Tinker — 2022-07-12 at 11:50

I spent many hours & much stress trying to avoid the whole "Swap everything" step...
Damn near gave up on it...
Then, suddenly, decided to abuse it a bit & it worked.
Now both the servers I've done this on look like they're hooked up backwards if you pay too much attention to the port numbers 
on the back. 
But I have a label maker & I'm not afraid to use it.
Damn!
Bloody coffee cup has a big hole in the top...
BRB

• TheGuruOfNothing — 2022-07-12 at 11:57

Ok, what the literal hell is CENSORED going on?
I have the PfSense ISO on Datastore1 and that is the ONLY place it is...
I selected it for use for the building of the new VM...
and it CENSORED boot loops saying that the media is not present
I used it to build a VM on this box already so I know it worked
I deleted that VM so there should be no conflict if there was such a thing (edited)

• Tinker — 2022-07-12 at 11:59

Did you "Remove" the original VM & start from scratch? Or just try to tell it to buut from the image?

• TheGuruOfNothing — 2022-07-12 at 11:59

I deleted the original VM
then started from scratch
buut? Canadian?

• Tinker — 2022-07-12 at 12:01

Canadian?

• TheGuruOfNothing — 2022-07-12 at 12:01

yanno... like aBOOt

• Tinker — 2022-07-12 at 12:01

• TheGuruOfNothing — 2022-07-12 at 12:01

hehe
ok... I am done
for now
I might pull that one back out in a week or two
(it did feel kinda good though)

• Tinker — 2022-07-12 at 12:02

Need a :slap: emoticon

• TheGuruOfNothing — 2022-07-12 at 12:02

indeed

• Tinker — 2022-07-12 at 12:03

musta been some simple error setting up the vm...

• TheGuruOfNothing — 2022-07-12 at 12:03

lemme keel it and do it again

• Tinker — 2022-07-12 at 12:07

Did you remember to tell it OS Type = OTHER?

• TheGuruOfNothing — 2022-07-12 at 12:11

no
was Linux 5x

• Tinker — 2022-07-12 at 12:12

pfSense ain't Linux... Just sayin'

• TheGuruOfNothing — 2022-07-12 at 12:12

it didn't even try to install, said media not present
I deleted the ISO and downloading new copy directly to server
will use OTHER

• Tinker — 2022-07-12 at 12:21

Just realized why running through the steps on a test server is fighting me...
Doing it on the silly little Celery machine...

• TheGuruOfNothing — 2022-07-12 at 12:22

Still boot looping

• Tinker — 2022-07-12 at 12:22

It doesn't have hardware virtualisation OR more than 1 core OR more than 2GB of RAM...

• TheGuruOfNothing — 2022-07-12 at 12:22

Trying to use the PF sense aiso and it doesn't even acknowledge it is
Single core 4 gigs of ram and I don't know about hardware virtualization
I gotta go take wifey to a doctor's appointment so I will get back on this when I get back home in a few hours

• Tinker — 2022-07-12 at 12:24

I'll see if I can reproduce the bootloop (or find out why it might happen...)

Thoughts while waiting for Guru to return

• Probably need to follow along as he builds a VM again... From scratch...
  • Step-by-step check options during VM build...
• Possibly ISO problem?
  • Screencap immediately before hitting Download button...

pfSense configuration and addressing

Tested nesting pfSense VMs... It still puts LAN on 192.168.1.1/24 despite conflict.

Easily fixed after install tho. Just pick option 2 at the console.

Part 2

AAARRRGGG!!!

If you tell PVE to "Download from URL", click the "Query URL" button.  That way it will refuse to download a .gz file & you'll know it won't boot.

Part 3

• TheGuruOfNothing — Today at 19:36

So, PfSense is up and running
It is backed up
I have done nothing to it
I have a management VM already made that is on vmbr0 for network...

• Tinker — Today at 19:36

Looking at its console...  Bet it's doing 192.168.1.1/24 on the LAN side...

• TheGuruOfNothing — Today at 19:37

Do I need to shut it down and me it to vmbr1 since you said to put it on the second nic?

• Tinker — Today at 19:37

You need to do some configuration first.

• TheGuruOfNothing — Today at 19:38

yes, PfSense is on 1.1/24
LAN is

• Tinker — Today at 19:38

Right now, it can't properly do DHCP & DNS because it defaults to 1.1/24
which conflicts with the WAN address.

• TheGuruOfNothing — Today at 19:38

no, no... hold up

• Tinker — Today at 19:39

But it's an easy fix.

• TheGuruOfNothing — Today at 19:39

I was asking about the MANAGEMENT VM, not PfSense
Robin-Manager

• Tinker — Today at 19:39

ah...

• TheGuruOfNothing — Today at 19:39

It is up on vmbr0
previous build

• Tinker — Today at 19:40

yes, you're gonna move the manager to vmbr1...
But not until you fix the LAN addressing on pfSense.

• TheGuruOfNothing — Today at 19:40

k

• Tinker — Today at 19:40

So...  Go to it's console...
Pick option 2
Select an address range that doesn't conflict with your local network.
(This is why I moved my LAN to 192.168.8.1/23 a while ago...)
Since your LAN is 192.168.1.1/22...
I'd suggest putting robins internal LAN at 192.168.10.1/24

• TheGuruOfNothing — Today at 19:50

10.1/24 with DHCP starting at 150

• Tinker — Today at 19:50

Now, make sure Manager is shut down & change it's network to vmbr1

• TheGuruOfNothing — Today at 19:51

done
backing up

• Tinker — Today at 19:53

Now you can boot Manager back up, log into it in the console window & access the LAN side of pfSense.
Or...  ya know... connect a switch to the secondary NIC...
But you may as well do the main config of pfSense as it stands, then do the fugly step...
At which point, Robin is a standalone network.
Things to do to pfSense prior to the fugly part:

• TheGuruOfNothing — Today at 19:56

A standalone that I should be able to access via AD, correct?

• Tinker — Today at 19:56

Static DHCP for PVE & Manager...
Firewall NAT for PVE & Manager...
& then ya hafta wait until dinner is done for more tips...

• TheGuruOfNothing — Today at 19:57

k

Part 4

Example config files

Vanneck

/etc/network/interfaces	/etc/hosts
auto lo iface lo inet loopback iface eno1 inet manual iface enp2s0 inet manual auto vmbr0 iface vmbr0 inet static address 192.168.0.8/24 gateway 192.168.0.1 bridge-ports eno1 bridge-stp off bridge-fd 0 auto vmbr1 iface vmbr1 inet manual bridge-ports enp2s0 bridge-stp off bridge-fd 0	127.0.0.1 localhost.localdomain localhost 192.168.8.200 pveV.tinkernet.ca pveV # The following lines are desirable for IPv6 capable hosts ::1 ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters ff02::3 ip6-allhosts

Schizo-X

/etc/network/interfaces	/etc/hosts
auto lo iface lo inet loopback iface eno2 inet manual iface eno1 inet manual auto vmbr0 iface vmbr0 inet static address 192.168.1.2/24 gateway 192.168.1.1 bridge-ports eno1 bridge-stp off bridge-fd 0 #Internal Network (+ Eth0) auto vmbr1 iface vmbr1 inet manual bridge-ports eno2 bridge-stp off bridge-fd 0 #InterWebs (Eth1)	127.0.0.1 localhost.localdomain localhost 192.168.1.2 pve.tinkernet.ca pve # The following lines are desirable for IPv6 capable hosts ::1 ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters ff02::3 ip6-allhosts