Proxmox DRBD HA Controller problem

DRBD
1

Hello, I deleted linstor_db on node3. How can I safely add linstor_db again without damaging the system and ensure that linstor_db does not report as StandAlone?

root@node1:~# linstor storage-pool list
╭─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
┊ StoragePool          ┊ Node        ┊ Driver   ┊ PoolName       ┊ FreeCapacity ┊ TotalCapacity ┊ CanSnapshots ┊ State ┊ SharedName                       ┊
╞═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╡
┊ DfltDisklessStorPool ┊ node1       ┊ DISKLESS ┊                ┊              ┊               ┊ False        ┊ Ok    ┊ node1;DfltDisklessStorPool       ┊   
┊ DfltDisklessStorPool ┊ node2       ┊ DISKLESS ┊                ┊              ┊               ┊ False        ┊ Ok    ┊ node2;DfltDisklessStorPool       ┊   
┊ DfltDisklessStorPool ┊ node3       ┊ DISKLESS ┊                ┊              ┊               ┊ False        ┊ Ok    ┊ node3;DfltDisklessStorPool       ┊   
┊ drbd-zfs-sas0        ┊ node1       ┊ ZFS      ┊ data_sas0/drbd ┊   317.33 GiB ┊       464 GiB ┊ True         ┊ Ok    ┊ node1;drbd-zfs-sas0              ┊   
┊ drbd-zfs-sas0        ┊ node2       ┊ ZFS      ┊ data_sas0/drbd ┊   317.33 GiB ┊       464 GiB ┊ True         ┊ Ok    ┊ node2;drbd-zfs-sas0              ┊   
┊ drbd-zfs-ssd0        ┊ node1       ┊ ZFS      ┊ data_ssd0/drbd ┊     1.68 TiB ┊      1.73 TiB ┊ True         ┊ Ok    ┊ node1;drbd-zfs-ssd0              ┊   
┊ drbd-zfs-ssd0        ┊ node2       ┊ ZFS      ┊ data_ssd0/drbd ┊     1.68 TiB ┊      1.73 TiB ┊ True         ┊ Ok    ┊ node2;drbd-zfs-ssd0              ┊   
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
root@node1:~# linstor resource list
╭───────────────────────────────────────────────────────────────────────────────────────────────────────╮
┊ ResourceName ┊ Node        ┊ Port ┊ Usage  ┊ Conns                   ┊    State ┊ CreatedOn           ┊
╞═══════════════════════════════════════════════════════════════════════════════════════════════════════╡
┊ linstor_db   ┊ node1 ┊ 7000 ┊ InUse  ┊ StandAlone(node2)             ┊ UpToDate ┊ 2024-06-18 15:26:05 ┊
┊ linstor_db   ┊ node2 ┊ 7000 ┊ InUse  ┊ StandAlone(node1)             ┊ UpToDate ┊ 2024-06-18 15:26:05 ┊
┊ pm-0b7a526a  ┊ node1 ┊ 7003 ┊ InUse  ┊ Ok                            ┊ UpToDate ┊ 2024-06-18 22:58:44 ┊
┊ pm-0b7a526a  ┊ node2 ┊ 7003 ┊ Unused ┊ Ok                            ┊ UpToDate ┊ 2024-06-18 22:58:49 ┊
┊ pm-c497f25f  ┊ node1 ┊ 7004 ┊ InUse  ┊ Ok                            ┊ UpToDate ┊ 2024-06-18 23:07:47 ┊
┊ pm-c497f25f  ┊ node2 ┊ 7004 ┊ Unused ┊ Ok                            ┊ UpToDate ┊ 2024-06-18 23:07:52 ┊
┊ pm-d14f5a7f  ┊ node1 ┊ 7006 ┊ InUse  ┊ Ok                            ┊ UpToDate ┊ 2024-06-18 23:27:16 ┊
┊ pm-d14f5a7f  ┊ node2 ┊ 7006 ┊ Unused ┊ Ok                            ┊ UpToDate ┊ 2024-06-18 23:27:23 ┊
┊ pm-ebd55187  ┊ node1 ┊ 7005 ┊ InUse  ┊ Ok                            ┊ UpToDate ┊ 2024-06-18 23:20:18 ┊
┊ pm-ebd55187  ┊ node2 ┊ 7005 ┊ Unused ┊ Ok                            ┊ UpToDate ┊ 2024-06-18 23:20:25 ┊
╰───────────────────────────────────────────────────────────────────────────────────────────────────────╯
root@node1:~# linstor resource-group list
╭───────────────────────────────────────────────────────────────────────╮
┊ ResourceGroup  ┊ SelectFilter                  ┊ VlmNrs ┊ Description ┊
╞═══════════════════════════════════════════════════════════════════════╡
┊ DfltRscGrp     ┊ PlaceCount: 2                 ┊        ┊             ┊
╞┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╡
┊ drbd-zfs-sas0  ┊ PlaceCount: 2                 ┊ 0      ┊             ┊
┊                ┊ StoragePool(s): drbd-zfs-sas0 ┊        ┊             ┊
╞┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╡
┊ drbd-zfs-ssd0  ┊ PlaceCount: 2                 ┊ 0      ┊             ┊
┊                ┊ StoragePool(s): drbd-zfs-ssd0 ┊        ┊             ┊
╞┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄┄╡
┊ linstor-db-grp ┊ PlaceCount: 2                 ┊ 0      ┊             ┊
┊                ┊ StoragePool(s): drbd-zfs-sas0 ┊        ┊             ┊
┊                ┊ DisklessOnRemaining: True     ┊        ┊             ┊
╰───────────────────────────────────────────────────────────────────────╯
2

You will need to:

  1. Find the “good” controller instance:
    linstor controller which
  2. disable the HA configuration on the other node: we need to resolve the split brain by discarding the data there.
    drbd-reactorctl disable --now linstor_db
drbdadm disconnect linstor_db
drbdadm secondary linstor_db
drbdadm connect --discard-my-data linstor_db
  3. Reconnect the resource on the “good” controller:
    drbdadm disconnect linstor_db
drbdadm connect linstor_db
drbdadm status linstor_db
    The nodes should begin to sync and quickly be back to UpToDate state.
  4. Reset the tiebreaker flag:
    linstor resource-definition set-property linstor_db DrbdOptions/auto-add-quorum-tiebreaker
linstor resource list
    There should now be a diskless tiebreaker resource on node3.
  5. Re-enable the HA configuration on all nodes
    drbd-reactorctl enable linstor_db
1 Like
3

Hi, thank you very much. It saved my situation.

I have one last question: if I wanted to cancel the HA Linstor controller and move the Linstor controller to node3 only, what is the correct procedure?

I am interested in the technical aspect of how I would migrate.