Proxmox snapshot rollback not working

Delgado · September 30, 2025, 4:07pm

Hello,

I have started experiencing issues with snapshot rollbacks. I am not sure what the problem may be. I tried rolling back my kernel version, as I recently upgraded it, but the problem is still present. I am not really sure if this is a Linstor or Proxmox issue. I have attached the log.

root@pve-03:~# dpkg -l | grep linstor
ii  linstor-client                           1.26.1-1                             all          Linstor client command line tool
ii  linstor-common                           1.32.1-1                             all          DRBD distributed resource management utility
ii  linstor-controller                       1.32.1-1                             all          DRBD distributed resource management utility
ii  linstor-gui                              1.9.9-1                              all          Administration GUI for LINSTOR clusters.
ii  linstor-proxmox                          8.1.3-1                              all          DRBD distributed resource management utility
ii  linstor-satellite                        1.32.1-1                             all          DRBD distributed resource management utility
ii  python-linstor                           1.26.1-1                             all          Linstor python api library

root@pve-03:~# dpkg -l | grep drbd
ii  drbd-dkms                                9.2.15-1                             all          RAID 1 over TCP/IP for Linux module source
ii  drbd-reactor                             1.9.0-1                              amd64        Monitors DRBD resources via plugins.
ii  drbd-utils                               9.32.0-1                             amd64        RAID 1 over TCP/IP for Linux (user utilities)

root@pve-03:~# uname -a
Linux pve-01 6.14.11-3-pve #1 SMP PREEMPT_DYNAMIC PMX 6.14.11-3 (2025-09-22T10:13Z) x86_64 GNU/Linux

modinfo drbd | grep ^version
version:        9.2.15

TASK ERROR: API Return-Code: 500. Message: Could not rollback cluster wide snapshot snap_pm-45d8fa00_snap-1 of pm-45d8fa00, because: [{"ret_code":34340867,"message":"Snapshot 'snap_pm-45d8fa00_snap-1' of resource 'pm-45d8fa00' marked down for rollback.","obj_refs":{"RscDfn":"pm-45d8fa00","Snapshot":"snap_pm-45d8fa00_snap-1"},"created_at":"2025-09-30T10:29:33.307468723-04:00"},{"ret_code":36962307,"message":"(pve-02) Resource 'pm-45d8fa00' [DRBD] adjusted.","obj_refs":{"RscDfn":"pm-45d8fa00","Snapshot":"snap_pm-45d8fa00_snap-1"},"created_at":"2025-09-30T10:29:33.368380764-04:00"},{"ret_code":34340867,"message":"Deactivated resource 'pm-45d8fa00' on 'pve-02' for rollback","obj_refs":{"RscDfn":"pm-45d8fa00","Snapshot":"snap_pm-45d8fa00_snap-1"},"created_at":"2025-09-30T10:29:33.368497176-04:00"},{"ret_code":36962307,"message":"(pve-01) Resource 'pm-45d8fa00' [DRBD] adjusted.","obj_refs":{"RscDfn":"pm-45d8fa00","Snapshot":"snap_pm-45d8fa00_snap-1"},"created_at":"2025-09-30T10:29:33.405882104-04:00"},{"ret_code":34340867,"message":"Deactivated resource 'pm-45d8fa00' on 'pve-01' for rollback","obj_refs":{"RscDfn":"pm-45d8fa00","Snapshot":"snap_pm-45d8fa00_snap-1"},"created_at":"2025-09-30T10:29:33.405999321-04:00"},{"ret_code":-4611686018393046042,"message":"(pve-03) Error executing lsblk.","error_report_ids":["68DBE2C9-249FF-000027"],"obj_refs":{"RscDfn":"pm-45d8fa00","Snapshot":"snap_pm-45d8fa00_snap-1"},"created_at":"2025-09-30T10:29:34.460052562-04:00"},{"ret_code":34340867,"message":"Rollback of 'pm-45d8fa00' aborted due to error deactivating","obj_refs":{"RscDfn":"pm-45d8fa00","Snapshot":"snap_pm-45d8fa00_snap-1"},"created_at":"2025-09-30T10:29:34.58835522-04:00"}]  at /usr/share/perl5/PVE/Storage/Custom/LINSTORPlugin.pm line 633. 	PVE::Storage::Custom::LINSTORPlugin::volume_snapshot_rollback("PVE::Storage::Custom::LINSTORPlugin", HASH(0x57ce6069dfc8), "linstor", "pm-45d8fa00_120", "snap-1") called at /usr/share/perl5/PVE/Storage.pm line 451 	PVE::Storage::volume_snapshot_rollback(HASH(0x57ce606683b8), "linstor:pm-45d8fa00_120", "snap-1") called at /usr/share/perl5/PVE/QemuConfig.pm line 503 	PVE::QemuConfig::__snapshot_rollback_vol_rollback("PVE::QemuConfig", HASH(0x57ce5a2ac4e8), "snap-1") called at /usr/share/perl5/PVE/AbstractConfig.pm line 1208 	PVE::AbstractConfig::__ANON__("scsi0", HASH(0x57ce5a2ac4e8)) called at /usr/share/perl5/PVE/AbstractConfig.pm line 483 	PVE::AbstractConfig::foreach_volume_full("PVE::QemuConfig", HASH(0x57ce606a8048), undef, CODE(0x57ce57d3be28)) called at /usr/share/perl5/PVE/AbstractConfig.pm line 492 	PVE::AbstractConfig::foreach_volume("PVE::QemuConfig", HASH(0x57ce606a8048), CODE(0x57ce57d3be28)) called at /usr/share/perl5/PVE/AbstractConfig.pm line 1210 	PVE::AbstractConfig::snapshot_rollback("PVE::QemuConfig", 120, "snap-1") called at /usr/share/perl5/PVE/API2/Qemu.pm line 6207 	PVE::API2::Qemu::__ANON__() called at /usr/share/perl5/PVE/Tools.pm line 270 	eval {...} called at /usr/share/perl5/PVE/Tools.pm line 270 	PVE::Tools::lock_file_full("/var/lock/pve-manager/pve-migrate-120", 10, 0, CODE(0x57ce606a8510)) called at /usr/share/perl5/PVE/Tools.pm line 278 	PVE::Tools::lock_file("/var/lock/pve-manager/pve-migrate-120", 10, CODE(0x57ce606a8510)) called at /usr/share/perl5/PVE/GuestHelpers.pm line 93 	PVE::GuestHelpers::guest_migration_lock(120, 10, CODE(0x57ce606a8510)) called at /usr/share/perl5/PVE/API2/Qemu.pm line 6216 	PVE::API2::Qemu::__ANON__("UPID:pve-03:00006205:00028D9E:68DBE94D:qmrollback:120:danielg"...) called at /usr/share/perl5/PVE/RESTEnvironment.pm line 630 	eval {...} called at /usr/share/perl5/PVE/RESTEnvironment.pm line 621 	PVE::RESTEnvironment::fork_worker(PVE::RPCEnvironment=HASH(0x57ce5a2a8af8), "qmrollback", 120, "danielgaraventa\@Authentik", CODE(0x57ce606bc808)) called at /usr/share/perl5/PVE/API2/Qemu.pm line 6219 	PVE::API2::Qemu::__ANON__(HASH(0x57ce6069a1d0)) called at /usr/share/perl5/PVE/RESTHandler.pm line 503 	PVE::RESTHandler::handle("PVE::API2::Qemu", HASH(0x57ce5e4f0438), HASH(0x57ce6069a1d0)) called at /usr/share/perl5/PVE/HTTPServer.pm line 187 	eval {...} called at /usr/share/perl5/PVE/HTTPServer.pm line 143 	PVE::HTTPServer::rest_handler(PVE::HTTPServer=HASH(0x57ce57d3ca10), "::ffff:172.16.0.11", "POST", "/nodes/pve-03/qemu/120/snapshot/snap-1/rollback", HASH(0x57ce60699c60), HASH(0x57ce606a8ab0), "extjs") called at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 980 	eval {...} called at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 950 	PVE::APIServer::AnyEvent::handle_api2_request(PVE::HTTPServer=HASH(0x57ce57d3ca10), HASH(0x57ce60681b20), HASH(0x57ce60699c60), "POST", "/api2/extjs/nodes/pve-03/qemu/120/snapshot/snap-1/rollback") called at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1224 	eval {...} called at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1216 	PVE::APIServer::AnyEvent::handle_request(PVE::HTTPServer=HASH(0x57ce57d3ca10), HASH(0x57ce60681b20), HASH(0x57ce60699c60), "POST", "/api2/extjs/nodes/pve-03/qemu/120/snapshot/snap-1/rollback") called at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1721 	PVE::APIServer::AnyEvent::authenticate_and_handle_request(PVE::HTTPServer=HASH(0x57ce57d3ca10), HASH(0x57ce60681b20)) called at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1423 	eval {...} called at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 1406 	PVE::APIServer::AnyEvent::__ANON__(AnyEvent::Handle=HASH(0x57ce6069dc38), "", "\x{d}\x{a}") called at /usr/lib/x86_64-linux-gnu/perl5/5.40/AnyEvent/Handle.pm line 1541 	AnyEvent::Handle::__ANON__(AnyEvent::Handle=HASH(0x57ce6069dc38)) called at /usr/lib/x86_64-linux-gnu/perl5/5.40/AnyEvent/Handle.pm line 1315 	AnyEvent::Handle::_drain_rbuf(AnyEvent::Handle=HASH(0x57ce6069dc38)) called at /usr/lib/x86_64-linux-gnu/perl5/5.40/AnyEvent/Handle.pm line 2015 	AnyEvent::Handle::__ANON__() called at /usr/lib/x86_64-linux-gnu/perl5/5.40/AnyEvent/Loop.pm line 248 	AnyEvent::Loop::one_event() called at /usr/lib/x86_64-linux-gnu/perl5/5.40/AnyEvent/Impl/Perl.pm line 46 	AnyEvent::CondVar::Base::_wait(AnyEvent::CondVar=HASH(0x57ce5a28b640)) called at /usr/lib/x86_64-linux-gnu/perl5/5.40/AnyEvent.pm line 2034 	AnyEvent::CondVar::Base::recv(AnyEvent::CondVar=HASH(0x57ce5a28b640)) called at /usr/share/perl5/PVE/APIServer/AnyEvent.pm line 2064 	PVE::APIServer::AnyEvent::run(PVE::HTTPServer=HASH(0x57ce57d3ca10)) called at /usr/share/perl5/PVE/Service/pvedaemon.pm line 56 	PVE::Service::pvedaemon::run(PVE::Service::pvedaemon=HASH(0x57ce60580a20)) called at /usr/share/perl5/PVE/Daemon.pm line 171 	eval {...} called at /usr/share/perl5/PVE/Daemon.pm line 171 	PVE::Daemon::__ANON__(PVE::Service::pvedaemon=HASH(0x57ce60580a20)) called at /usr/share/perl5/PVE/Daemon.pm line 390 	eval {...} called at /usr/share/perl5/PVE/Daemon.pm line 379 	PVE::Daemon::__ANON__(PVE::Service::pvedaemon=HASH(0x57ce60580a20), undef) called at /usr/share/perl5/PVE/Daemon.pm line 549 	eval {...} called at /usr/share/perl5/PVE/Daemon.pm line 547 	PVE::Daemon::start(PVE::Service::pvedaemon=HASH(0x57ce60580a20), undef) called at /usr/share/perl5/PVE/Daemon.pm line 657 	PVE::Daemon::__ANON__(HASH(0x57ce57d33a28)) called at /usr/share/perl5/PVE/RESTHandler.pm line 503 	PVE::RESTHandler::handle("PVE::Service::pvedaemon", HASH(0x57ce60580d68), HASH(0x57ce57d33a28), 1) called at /usr/share/perl5/PVE/RESTHandler.pm line 1005 	eval {...} called at /usr/share/perl5/PVE/RESTHandler.pm line 986 	PVE::RESTHandler::cli_handler("PVE::Service::pvedaemon", "pvedaemon start", "start", ARRAY(0x57ce58040278), ARRAY(0x57ce60581410), undef, undef, undef) called at /usr/share/perl5/PVE/CLIHandler.pm line 624 	PVE::CLIHandler::__ANON__(ARRAY(0x57ce57d34220), CODE(0x57ce58132b40), undef) called at /usr/share/perl5/PVE/CLIHandler.pm line 705 	PVE::CLIHandler::run_cli_handler("PVE::Service::pvedaemon", "prepare", CODE(0x57ce58132b40)) called at /usr/bin/pvedaemon line 27

Delgado · September 30, 2025, 7:18pm

I tried downgrading drbd-dkms, drbd-utils, and linstor-proxmox to the version right before the versions listed above, but that didnt seem to help.

Devin · September 30, 2025, 9:55pm

Is there any error within the LINSTOR logs on either the controller or Satellite? Any LINSTOR error reports recorded that seem relevant (linstor err list)?

If so, could you share the contents of those as well?

Delgado · September 30, 2025, 11:49pm

I regenerated the from a fresh snap and attempted rollback

ERROR REPORT 68DC2EA3-8B928-000005

============================================================

Application: LINBIT® LINSTOR

Module: Satellite

Version: 1.32.1

Build ID: e04f98efc3aeb643cf109ffd322a4f2506000da1

Build time: 2025-09-16T09:03:12+00:00

Error time: 2025-09-30 18:53:55

Node: pve-02

Thread: DeviceManager

============================================================

Reported error:

===============

Category: LinStorException

Class name: StorageException

Class canonical name: com.linbit.linstor.storage.StorageException

Generated at: Method ‘checkExitCode’, Source file ‘ExtCmdUtils.java’, Line #70

Error message: Failed to query symlinks of device /dev/drbd1039

Error context:

    An error occurred while processing resource 'Node: 'pve-02', Rsc: 'pm-88065fdc''

ErrorContext:

Details: Command ‘udevadm info -q symlink /dev/drbd1039’ returned with

exitcode 1.

Standard out:

Error message:

Unknown device “/dev/drbd1039”: No such device

Call backtrace:

Method                                   Native Class:Line number

checkExitCode                            N      com.linbit.extproc.ExtCmdUtils:70

genericExecutor                          N      com.linbit.linstor.storage.utils.Commands:108

genericExecutor                          N      com.linbit.linstor.storage.utils.Commands:68

genericExecutor                          N      com.linbit.linstor.storage.utils.Commands:56

getSymlinks                              N      com.linbit.linstor.core.UdevHandler:43

updateDeviceSymlinks                     N      com.linbit.linstor.core.devmgr.DeviceHandlerImpl:1593

processResources                         N      com.linbit.linstor.core.devmgr.DeviceHandlerImpl:430

dispatchResources                        N      com.linbit.linstor.core.devmgr.DeviceHandlerImpl:222

dispatchResources                        N      com.linbit.linstor.core.devmgr.DeviceManagerImpl:334

phaseDispatchDeviceHandlers              N      com.linbit.linstor.core.devmgr.DeviceManagerImpl:1140

devMgrLoop                               N      com.linbit.linstor.core.devmgr.DeviceManagerImpl:779

run                                      N      com.linbit.linstor.core.devmgr.DeviceManagerImpl:675

run                                      N      java.lang.Thread:1583

END OF ERROR REPORT.

What is strange is I see the device.

root@pve-02:~# ls -la /dev/drbd1039

brw-rw---- 1 root disk 147, 1039 Sep 30 19:03 /dev/drbd1039

Delgado · October 1, 2025, 5:36pm

@Devin I found this github issue. It seems to be throwing the same error and my backend storage in proxmox is ZFS. Can't rollback ZFS snapshot in a single node environment · Issue #464 · LINBIT/linstor-server · GitHub

Topic		Replies	Views
Proxmox: TASK ERROR: API Return-Code: 500. Message: Could not rollback cluster wide snapshot LINSTOR drbd	15	166	April 14, 2025
Snap restore on proxmox with lvm-thin as storage pool seems unstable LINSTOR	7	426	April 16, 2025
Proxmox Snapshots with ram Proxmox VE	3	298	May 14, 2024
Snapshot by backup not working General	0	75	October 18, 2024
Snapshots and proxmox_backup_server Proxmox VE linstor , proxmox , snapshots	0	35	April 3, 2025

Proxmox snapshot rollback not working

Related topics