# Load-balance-paths yes **URL:** \ **Category:** General\ **Tags:** drbd\ **Created:** [November 13, 2024, 8:04am UTC](https://forums.linbit.com/t/load-balance-paths-yes/430 "2024-11-13T08:04:19Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Anton](https://avatars.discourse-cdn.com/v4/letter/a/d9b06d/32.png) [@Anton](https://forums.linbit.com/u/Anton)\ **Post date:** [November 13, 2024, 8:04am UTC](https://forums.linbit.com/t/load-balance-paths-yes/430/1 "2024-11-13T08:04:19Z")
I configured 2 paths for load-balance and failover. But replication traffic goes only though single path. 2 nodes, Rocky 9.4 (5.14.0-427.42.1.el9\_4.x86\_64), drbd-9.28.0-1 ```plaintext [root@memverge ~]# cat /etc/drbd.d/resource0.res resource resource0 { net { load-balance-paths yes; protocol C; # fencing resource-and-stonith; } handlers { fence-peer "/usr/lib/drbd/crm-fence-peer.9.sh"; after-resync-target "/usr/lib/drbd/crm-unfence-peer.9.sh"; } on "memverge" { device /dev/drbd0; disk /dev/md0; meta-disk internal; # address 10.72.14.152:7789; node-id 0; } on "memverge2" { device /dev/drbd0; disk /dev/md0; meta-disk internal; # address 10.72.14.154:7789; node-id 1; } connection { path { host "memverge" address ipv4 192.168.0.6:7900; host "memverge2" address ipv4 192.168.0.8:7900; } path { host "memverge" address ipv4 192.168.0.7:7900; host "memverge2" address ipv4 192.168.0.9:7900; } } } [root@memverge ~]# ``` What am I doing wrong ? ---
**Author:** ![kermat](https://yyz1.discourse-cdn.com/flex003/user_avatar/forums.linbit.com/kermat/32/21_2.png) [@kermat](https://forums.linbit.com/u/kermat)\ **Post date:** [November 13, 2024, 5:19pm UTC](https://forums.linbit.com/t/load-balance-paths-yes/430/2 "2024-11-13T17:19:27Z")
Is the `drbd_transport_lb_tcp` kernel module loaded? ```plaintext # lsmod | grep drbd drbd_transport_tcp 28672 1 handshake 28672 1 drbd_transport_tcp drbd 700416 3 drbd_transport_tcp lru_cache 16384 1 drbd libcrc32c 16384 3 dm_persistent_data,drbd,sctp # modprobe drbd_transport_lb-tcp # lsmod | grep drbd drbd_transport_lb_tcp 28672 0 drbd_transport_tcp 28672 1 handshake 28672 1 drbd_transport_tcp drbd 700416 4 drbd_transport_lb_tcp,drbd_transport_tcp lru_cache 16384 1 drbd libcrc32c 16384 3 dm_persistent_data,drbd,sctp ``` I haven’t configured load balanced resources on the host I took those outputs from, just FYI in case there are some differences. Or, maybe you have yet to adjust (`drbdadm adjust all`) your resources? ---
**Author:** ![Anton](https://avatars.discourse-cdn.com/v4/letter/a/d9b06d/32.png) [@Anton](https://forums.linbit.com/u/Anton)\ **Post date:** [November 14, 2024, 8:34am UTC](https://forums.linbit.com/t/load-balance-paths-yes/430/3 "2024-11-14T08:34:51Z")
There is no drbd\_transport\_lb-tcp module available. [root@memverge ~]# modprobe drbd\_transport\_lb-tcp modprobe: FATAL: Module drbd\_transport\_lb-tcp not found in directory /lib/modules/5.14.0-427.42.1.el9\_4.x86\_64 [root@memverge ~]# [root@memverge ~]# find / -name drbd\_transport\_\* /sys/kernel/debug/printk/index/drbd\_transport\_tcp /sys/module/drbd\_transport\_tcp /sys/module/drbd/holders/drbd\_transport\_tcp /usr/lib/modules/5.14.0-427.13.1.el9\_4.x86\_64/extra/drbd9x/drbd\_transport\_tcp.ko /usr/lib/modules/5.14.0-427.42.1.el9\_4.x86\_64/weak-updates/drbd9x/drbd\_transport\_tcp.ko [root@memverge ~]# I have next _drbd_ packages installed - [root@memverge ~]# rpm -qa|grep -i drbd drbd-selinux-9.28.0-1.el9.x86\_64 drbd-utils-9.28.0-1.el9.x86\_64 drbd-udev-9.28.0-1.el9.x86\_64 collectd-drbd-5.12.0-24.el9.x86\_64 drbd-9.28.0-1.el9.x86\_64 drbd-bash-completion-9.28.0-1.el9.x86\_64 drbd-pacemaker-9.28.0-1.el9.x86\_64 drbdlinks-1.29-5.el9.noarch kmod-drbd9x-9.1.22-1.el9\_4.elrepo.x86\_64 [root@memverge ~]# ---
**Author:** ![kermat](https://yyz1.discourse-cdn.com/flex003/user_avatar/forums.linbit.com/kermat/32/21_2.png) [@kermat](https://forums.linbit.com/u/kermat)\ **Post date:** [November 14, 2024, 5:02pm UTC](https://forums.linbit.com/t/load-balance-paths-yes/430/4 "2024-11-14T17:02:35Z")
> [@Anton](#): > > kmod-drbd9x-9.1.22-1.el9\_4.elrepo.x86\_64 Aha, the load-balanced TCP module wasn’t added until [DRBD 9.2.6](https://github.com/LINBIT/drbd/blob/176bec24cde49b18796e08b4045e23e77a61ffe5/ChangeLog#L132). If ELRepo doesn’t have a newer kernel module package and you really want to try the load-balanced TCP transport, you can download and compile a 9.2.6 (or newer) kmod from LINBIT’s [prepared source code tarballs](https://pkg.linbit.com/) (or reach out to LINBIT for eval access to their customer repos). ---
**Author:** ![Anton](https://avatars.discourse-cdn.com/v4/letter/a/d9b06d/32.png) [@Anton](https://forums.linbit.com/u/Anton)\ **Post date:** [November 15, 2024, 6:29pm UTC](https://forums.linbit.com/t/load-balance-paths-yes/430/5 "2024-11-15T18:29:34Z")
Ok, thank you Matt. I’m surprised that Elrepo contains so old version… I compiled latest stable 9.2.11 and add it at boot for both nodes. So after nodes boot, I see on both nodes lsmod | grep drbd drbd\_transport\_lb\_tcp 40960 1 drbd 1007616 3 drbd\_transport\_lb\_tcp libcrc32c 16384 2 xfs,drbd All looks good, but when I generated replication traffic with simplest rsync -av --progress /home/anton/testfile /mnt/testfile replication traffic spread slightly unevenly across two paths, but OK. I checked with iptraf-ng monitoring “General interface statistics”. P. S., there two nodes mounted in the single rack and directly connected using Nvidia Connect-X5 NICs without any switches. ---
**Author:** ![kermat](https://yyz1.discourse-cdn.com/flex003/user_avatar/forums.linbit.com/kermat/32/21_2.png) [@kermat](https://forums.linbit.com/u/kermat)\ **Post date:** [November 15, 2024, 8:51pm UTC](https://forums.linbit.com/t/load-balance-paths-yes/430/6 "2024-11-15T20:51:08Z")
> [@Anton](#): > > replication traffic spread slightly unevenly across two paths, but OK. Yes. When I’ve looked it’s been pretty close, but never “perfectly even”. If you’re seeing wildly different numbers, then please do let us know. > P. S., there two nodes mounted in the single rack and directly connected using Nvidia Connect-X5 NICs without any switches. Not to send you down a totally different path, but DRBD’s [RDMA transport](https://linbit.com/drbd-user-guide/drbd-guide-9_0-en/#s-rdma_transport) balances traffic over multiple paths. ---
**Author:** ![Anton](https://avatars.discourse-cdn.com/v4/letter/a/d9b06d/32.png) [@Anton](https://forums.linbit.com/u/Anton)\ **Post date:** [November 15, 2024, 11:10pm UTC](https://forums.linbit.com/t/load-balance-paths-yes/430/7 "2024-11-15T23:10:28Z")
I like RDMA, but with RoCE there is a small problem. How to monitor RoCE traffic ?, standart tools such as nload doesn’t work, becase they looked at TCP/IP stack, with RoCE there is no TCP/IP stack. So how to monitor RoCE network traffic ? Something like, cat /sys/class/infiniband/mlx5\_0/ports/1/counters/port\_rcv\_data is not very informative…