Hello,
sorry, but I’ve been sick for 2 days and didn’t feel like writing.
Anyway, this has now become a mixed thread: code and dealing with each other. For all readers following; I will deal with code first, 'cause that’s what most likely has brought you here - but not without my sincere apology for my words if they sounded rude.
The problem has been fixed. As @kermat said, the build pipeline was activated and the versions are matching again.
Maybe someone is looking for a summary, so here it is:
I aimed for a 2-node PVE cluster with a Raspberry Pi as the PVE quorum and diskless DRBD tie-breaker. Strictly home use, but this setup tries to achieve higher availability. If you’re using the (LINBIT-) recommended way with LINSTOR and DRBD9 and Proxmox, that means that you need a HA linstor-controller as well. As @kermat recommends: the controller should be running on the PVE nodes, not in a VM.
On the PVE nodes you should install the software as described by LINBIT: How to Setup LINSTOR on Proxmox VE
apt -y install pve-headers-$(uname -r) proxmox-default-headers drbd-dkms drbd-utils linstor-common linstor-client linstor-controller python-linstor linstor-satellite
Just follow the LINBIT docs and everything will be alright.
On the raspi, you need to know, that you can’t get the required software packages from the official LINBIT repos, if you’re not a (paying) customer. But fortunately some nice LINBIT people are running a ppa with the packages. But getting software from a ppa repo also means, that you can’t run Rasbian OS on your raspi, instead you have to use Ubuntu. (Thanks (a lot of) to @mtisza)
For the tie-breaker role the raspi needs:
add-apt-repository ppa:linbit/linbit-drbd9-stack && sudo apt update
apt install linux-headers-6.8.0-1017-raspi (make sure to install the ones matching your current kernel!)
apt install drbd-dkms drbd-utils linstor-common linstor-satellite
AND: make sure, that the version numbers in the ppa (linstor-* packages) are matching the ones in the official LINBIT repo to the last digit. If they are not matching, just wait a bit until LINBIT updates the ppa. Versions in the official repo are leading, they must be followed.
Again you can follow the LINBIT docs for configuring LINSTOR. Make only one PVE node the LINSTOR controller for now. Make both PVE nodes and the raspi LINSTOR satellite nodes, create DRBD resources on the PVE nodes. Maybe set
linstor set-property DrbdOptions/AutoEvictAllowEviction false
on controller or node level.
At this point you’ll have a working DRBD cluster on Proxmox. The LINSTOR PVE packages will create disk resources for LXC and VM from the Proxmox UI.
What you don’t have at this point is the LINSTOR controller HA, which I haven’t done yet.
Again, I like to refer to @mtisza and his valuable contribution: maybe you like to make sure that DRBD/LINSTOR related packages are under tighter control and apt-mark hold these packages
Now for the other part.
Family-resilience is the target;-). HA the way to get closer.
Energy costs are a leading factor, for this reason (continuously running) shared storage should be avoided.
Searching for a synchronous disk replication of local disks on different systems bought me to DRBD. Ceph isn’t an option for a 2-PVE-RasPi-HomeUse-Setup. DRBD has some reputation, a lot from being integrated into the Linux kernel.
Luckily I don’t have experienced open source support like @vik-t. I really appreciate the passion and devotion of open source developers. When choosing open source software I look at how many people are contributing, how lively the community is and factor what I see as “reputation”. This might help to choose software with good “community interaction”.
Proxmox forums are referring to LINBIT since the change to DRBD9. I guess you would agree, that the LINBIT forum is not the hotspot of the internet.
But I saw reputation and the contributors are a company. That convinced me. And the company sells support to their customers, so these customers wouldn’t appear in the forum.
I’ve gone up and down every LINBIT doc that seems to relate to my setup. I asked “How to config the raspi?” and now that I know I did a google search for “linbit ppa”. Although it has been talked about the raspi a lot, it took a community member @mtisza to disclose the needed information. Hint for LINBIT: although the raspi is mentioned in your docs, the ppa seems missing or is hidden too deep.
By coincidence then hit the build-version-mismatch. I guess at this moment, acting in short time frames during the holiday season, my thoughts were: what the…it’s supposed to be an automated build pipeline…how can there be a mismatch…by no way a mismatch in the third place should lead to such a problem…
Well, I guess the build pipelines are not running in the same cycle as the ones for the commercial repo, what I absolutely understand from a technical perspective.
Anyway: again my apology.
