Rebuilding a Proxmox cluster node

Recently, I had to rebuild my proxmox cluster after a disk failure. Against all advice, I installed Proxmox on USB keys. It worked well for ESX so I thought I could get away with it with Proxmox, but alas. Both of my nodes failed after a few months.

Rebuilding the cluster was easier than I expected, but probably because I had some backups of:

• /etc/pve
• /etc/corosync

So I installed Proxmox on the secondary node and set the same hostname and IP address. Then I started working on adding back the storage by following the instructions on my previous blog post.

Preventing Windows 11 / Windows Server 2022 crashes

For four months straight, I had stability issues with all of my modern Windows OS installs. Drove me crazy and I was considering giving up on Proxmox until I found some advice in the forums that said to update the kernel.

The following code solved all of my crashes.

1mv /etc/apt/sources.list.d/pve-enterprise.list /tmp/
2apt update && apt install pve-kernel-5.15

Using the local drive as a backup drive

Since my base OS disk was now 1TB, it'd be perfect to setup backups. This is some nice reassurance I didn't get with my VMware setup. I never had redundancy, not even RAID :O

Out of the box, Proxmox will partition 100GB of a 1TB disk. So that's 900GB doing something I'm not sure of. To make it into a Backup disk, I first tried doing it the wrong way and received the error:

directory is expected to be a mount point but is not mounted: '/var/lib/vz'

After a bit more research, I ran a few commands that I found on Proxmox's wiki.

1lvremove /dev/pve/data
2lvcreate --name data -l +100%FREE pve
3mkfs.ext4 /dev/pve/data

Then I added the following to /etc/fstab: /dev/pve/data /var/lib/vz ext4 defaults 0 1. Then I mount -a and voila! Go to Datacenter -> Storage -> Add Directory, and this is what you'll see.

I know I could have just used local but I wanted it to be backup. If you're curious about what my /etc/pve/storage.cfg looks like, it's this:

 1dir: m2
 2        path /mnt/pve/m2
 3        content images,snippets,rootdir,vztmpl,iso
 4        is_mountpoint 1
 5        nodes proxmox
 6        shared 0
 7
 8dir: ssd
 9        path /mnt/pve/ssd
10        content vztmpl,rootdir,snippets,images,iso
11        is_mountpoint 1
12        nodes proxmox
13        shared 0
14
15dir: prixssd
16        path /mnt/pve/prixssd
17        content images,snippets,rootdir,vztmpl,iso
18        is_mountpoint 1
19        nodes prixmix
20        shared 0
21
22dir: local
23        disable
24        path /var/lib/vz
25        content snippets
26        prune-backups keep-all=1
27        shared 0
28
29dir: backup
30        path /var/lib/vz
31        content backup
32        is_mountpoint 1
33        shared 0

Fixing KVM issues

My secondary node did run into KVM issues after a rebuild, while my primary node did not. Unsure what's up with that but I was able to fix it by running

pvecm updatecerts

I also threw in the following for good-measure because it was recommended in another thread when I was trying to figure out: edited /etc/modprobe.d/kvm.conf, added options kvm ignore_msrs=1 report_ignored_msrs=0 then ran update-initramfs -u -k all.