# Snap restore on proxmox with lvm-thin as storage pool seems unstable **URL:** \ **Category:** LINSTOR\ **Created:** [October 17, 2024, 2:54pm UTC](https://forums.linbit.com/t/snap-restore-on-proxmox-with-lvm-thin-as-storage-pool-seems-unstable/375 "2024-10-17T14:54:14Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![c.duchenoy](https://yyz1.discourse-cdn.com/flex003/user_avatar/forums.linbit.com/c.duchenoy/32/137_2.png) [@c.duchenoy](https://forums.linbit.com/u/c.duchenoy)\ **Post date:** [October 17, 2024, 2:54pm UTC](https://forums.linbit.com/t/snap-restore-on-proxmox-with-lvm-thin-as-storage-pool-seems-unstable/375/1 "2024-10-17T14:54:14Z")
Hi, We have several proxmox install with linstor as SDS, we found out that if you take snapshot, and rollback with ZFS as a storage pool, everything works flawless. However, when you try the same thing with a lvm-thin storage pool, it fails almost everytime. Sometimes it will work but when it is not you are left with stuck ressources. We tried : - powered on or powered off VM - the oldest snap - the most recent one - snap in the middle of the “tree” - DrbdOptions/Net/allow-two-primaries set to yes or no : same problem We did not find a stable working method of rolling back snapshots via the proxmox web UI. I will try the same with the linstor CLI and report back EDIT : Yes, same problem with the linstor snapshot rollback cmd ```auto Description: (Node: 'vc-swarm3') Failed to rollback to snapshot pve/vm-700-disk-1_00000_snap_vm-700-disk-1_snap1 Details: Command 'lvconvert --config devices { filter=['a|/dev/sdg3|','r|.*|'] } --merge pve/vm-700-disk-1_00000_snap_vm-700-disk-1_snap1' returned with exitcode 5. Standard out: Error message: pve/vm-700-disk-1_00000_snap_vm-700-disk-1_snap1 is not a mergeable logical volume. ``` And we can see that we end up with a snap that is not “linked” to the parent lvm ```auto vm-700-disk-1_00000 pve Vwi---tz-- 10,00g thin-hdd vm-700-disk-1_00000_snap_vm-700-disk-1_snap1 pve Vwi-a-tz-k 10,00g thin-hdd 100,00 ``` We have three cluster in that situation one with like so : ```auto root@vc-swarm1:~# dpkg -l | grep linstor ii linstor-client 1.18.0-1 all Linstor client command line tool ii linstor-common 1.22.0-1 all DRBD distributed resource management utility ii linstor-controller 1.22.0-1 all DRBD distributed resource management utility ii linstor-proxmox 7.0.0-1 all DRBD distributed resource management utility hi linstor-satellite 1.22.0-1 all DRBD distributed resource management utility ii python-linstor 1.18.0-1 all Linstor python api library ``` And two other : ```auto └─$ dpkg -l | grep linstor ii linstor-client 1.23.0-1 all Linstor client command line tool hi linstor-common 1.29.0-1 all DRBD distributed resource management utility ii linstor-controller 1.29.0-1 all DRBD distributed resource management utility hi linstor-proxmox 8.0.4-1 all DRBD distributed resource management utility ii linstor-satellite 1.29.0-1 all DRBD distributed resource management utility ii python-linstor 1.23.0-1 all Linstor python api library ``` ---
**Author:** ![ghernadi](https://avatars.discourse-cdn.com/v4/letter/g/9e8a1a/32.png) [@ghernadi](https://forums.linbit.com/u/ghernadi)\ **Post date:** [October 21, 2024, 6:19am UTC](https://forums.linbit.com/t/snap-restore-on-proxmox-with-lvm-thin-as-storage-pool-seems-unstable/375/2 "2024-10-21T06:19:44Z")
Hello, Thank you for the report. We are aware of this issue. We have a few ideas we are currently testing that could address this issue. Some technical background: If you have an LVM volume, and create let’s say 2 snapshots of it, both snapshots will have the original volume as their “origin”. If you know run a `linstor snapshot rollback` (which internally runs a `lvconvert --merge $vg/$snapshot` as the error message states), it merges the snapshot into its origin. So far everything is as expected. After this command two things have changed: First, the LVM snapshot is now gone (since it got merged), but LINSTOR simply creates a new snapshot to “fix” this point. The second point is more problematic: The second snapshot we created in the beginning, which was completely untouched by our `linstor snapshot rollback` and `lvconvert --merge` commands, also “lost” its origin. The data is still there and fine, but this second snapshot can no longer be “merged” into the already rolled back volume. As a workaround, instead of using `linstor snapshot rollback` for LVM-THIN setups, you can manually delete the resources (not the -definition) and `linstor snapshot resource restore --fr $rsc --tr $rsc --fs $snapshot` the snapshot into the same resource-definition. This is actually one of our plans we are testing right now. The idea sounds fine on LVM\_THIN but unfortunately does not work that easily on ZFS setups since there you cannot just delete the volume while it has snapshots (there is a very strict parent-child dependency between ZVOLs and their snapshots). ---
**Author:** ![c.duchenoy](https://yyz1.discourse-cdn.com/flex003/user_avatar/forums.linbit.com/c.duchenoy/32/137_2.png) [@c.duchenoy](https://forums.linbit.com/u/c.duchenoy)\ **Post date:** [October 21, 2024, 2:27pm UTC](https://forums.linbit.com/t/snap-restore-on-proxmox-with-lvm-thin-as-storage-pool-seems-unstable/375/3 "2024-10-21T14:27:18Z")
Helo @ghernadi and thanks for your answer. We will do so test with the suggested method and will report back in the next days ! ---
**Author:** ![vik-t](https://avatars.discourse-cdn.com/v4/letter/v/ba8739/32.png) [@vik-t](https://forums.linbit.com/u/vik-t)\ **Post date:** [October 23, 2024, 9:40am UTC](https://forums.linbit.com/t/snap-restore-on-proxmox-with-lvm-thin-as-storage-pool-seems-unstable/375/4 "2024-10-23T09:40:01Z")
That sounds… scary. Isn’t LVM-THIN like THE storage pool technology for Linstor? How is it possible then that this issue was not noticed before? ---
**Author:** ![c.duchenoy](https://yyz1.discourse-cdn.com/flex003/user_avatar/forums.linbit.com/c.duchenoy/32/137_2.png) [@c.duchenoy](https://forums.linbit.com/u/c.duchenoy)\ **Post date:** [November 28, 2024, 10:20am UTC](https://forums.linbit.com/t/snap-restore-on-proxmox-with-lvm-thin-as-storage-pool-seems-unstable/375/5 "2024-11-28T10:20:07Z")
Hi @ghernadi Txu for your input 🙂 resource restore works as expected ! For proxmox user, we write a little script that restore selected snap Cant upload here so view [https://gist.github.com/cduchenoy/36d57d7d089f3553e15ba37fb7ed8258](https://gist.github.com/cduchenoy/36d57d7d089f3553e15ba37fb7ed8258) ### Script output ```bash $ geco-linstor-pve-snap rollback --vmid 100 --snap snap1 Rollback DRBD volume snapshot: "snap1" - Proxmox VMID: "100" Found snap snap1 on drbd resource pm-56c30604 Stop VMID 100 UPID:vcg12:000CA929:061858C5:674838B5:qmstop:100:root@pam: Remove drbd volume: pm-56c30604 on vcg11 SUCCESS: Description: Node: vcg11, Resource: pm-56c30604 preparing for deletion. ... Remove drbd volume: pm-56c30604 on vcg12 SUCCESS: Description: Node: vcg12, Resource: pm-56c30604 preparing for deletion. ... Remove drbd volume: pm-56c30604 on vcg13 SUCCESS: Description: Node: vcg13, Resource: pm-56c30604 preparing for deletion. Restore snap snap1 on pm-56c30604 ... +------------------------------------------------------------------+ | ResourceName | Port | ResourceGroup | State | Proxmox VMID | |==================================================================| | pm-56c30604 | 7004 | vmdisks-ssd-volumes | ok | 100 | +------------------------------------------------------------------+ +------------------------------------------------------------------------------------------------------------------+ | Node | Resource | StoragePool | VolNr | MinorNr | DeviceName | Allocated | InUse | State | |==================================================================================================================| | vcg12 | pm-56c30604 | pool-ssd-vmdisks-volumes | 0 | 1005 | /dev/drbd1005 | 630 KiB | Unused | UpToDate | | vcg13 | pm-56c30604 | pool-ssd-vmdisks-volumes | 0 | 1005 | /dev/drbd1005 | 630 KiB | Unused | UpToDate | +------------------------------------------------------------------------------------------------------------------+ +--------------------------------------------------------------------------------------------------------------------------------------------------+ | Resource | Issue | Possible fix | |==================================================================================================================================================| | pm-56c30604 | Resource has 2 replicas but no tie-breaker, could lead to split brain. | linstor rd ap --drbd-diskless --place-count 1 pm-56c30604 | +--------------------------------------------------------------------------------------------------------------------------------------------------+ Rollback Proxmox VM Config snapshot: "snap1" - Proxmox VMID: "100" - Prepare new current block config - Prepare block config for snap "snap1" - Prepare block config for snap "snap2" - Prepare block config for snap "snap3" Enable new VM config... ``` ---
**Author:** ![kastob](https://avatars.discourse-cdn.com/v4/letter/k/9de0a6/32.png) [@kastob](https://forums.linbit.com/u/kastob)\ **Post date:** [December 29, 2024, 11:00pm UTC](https://forums.linbit.com/t/snap-restore-on-proxmox-with-lvm-thin-as-storage-pool-seems-unstable/375/6 "2024-12-29T23:00:10Z")
I also noticed that on a LVM-Thin storage pool a snapshor rollback doesn’t work. It appears to do the disk rollback correctly but won’t roll back memory and just stops the VM. Error message is: “Error: start failed: QEMU exited with code 1 … blockdev: cannot open /dev/drbd/by-res/pm-e08830a6/0: Keine Daten verfügbar kvm: -drive file=/dev/drbd/by-res/pm-e08830a6/0,if=none,id=drive-scsi0,format=raw,cache=none,aio=io\_uring,detect-zeroes=on: Could not open ‘/dev/drbd/by-res/pm-e08830a6/0’: No data available” Is this the same issue as above and can we expect this to be fixed? I was very enthusiastic about the Linstor/Proxmox integration when starting the evaluation, but the nasty lowercase snapshot name issue, the inability to move storage (without temporarily tampering with storage.cfg) and the inability to rollback snapshots is disappointing. ---
**Author:** ![ghernadi](https://avatars.discourse-cdn.com/v4/letter/g/9e8a1a/32.png) [@ghernadi](https://forums.linbit.com/u/ghernadi)\ **Post date:** [April 16, 2025, 5:54am UTC](https://forums.linbit.com/t/snap-restore-on-proxmox-with-lvm-thin-as-storage-pool-seems-unstable/375/7 "2025-04-16T05:54:54Z")
Hello, sorry for the delay! I believe we have announced this already in other places but just stumbled across this post. The recently released LINSTOR 1.31.0 should properly deal with such rollback scenarios for LVM\_THIN as well as ZFS + ZFS\_THIN. ---
**Author:** ![c.duchenoy](https://yyz1.discourse-cdn.com/flex003/user_avatar/forums.linbit.com/c.duchenoy/32/137_2.png) [@c.duchenoy](https://forums.linbit.com/u/c.duchenoy)\ **Post date:** [April 16, 2025, 9:46am UTC](https://forums.linbit.com/t/snap-restore-on-proxmox-with-lvm-thin-as-storage-pool-seems-unstable/375/8 "2025-04-16T09:46:35Z")
Hi, Thanks for this feedback ! We’re still using 1.29 on our production clusters, but I’m working on trying this version on a test cluster. I’ll publish our feedback on this post for the community.