We noticed that the Linstor placement of resources can be sub-optimal when the resources groups are configured with multiple storage pools per node.
Example case A)
The storage pools are of unequal size.
This leads to resources being placed only on the largest SP until the remaining free space equalizes. Combined with subsequent allocation growth on thin pools, this can lead to a situation where one pool becomes completely full while the smaller pools are still empty.
Example case B)
Increasing the placement count of resource groups with existing volume definitions.
This causes all the new resources to be created instantly on the one pool with the most free space, even if the difference in free space is minimal. Linstor does not consider the available information about the allocation sizes of the existing volumes to better distribute the new resource placements.
There are other situations where the same type of issue occurs, like during a node evacuation.
It seems that the placement algorithm could be “smarter” when dealing with the case of multiple pools on a node. For case A the solution could be to not only consider the absolute free space of a pool, but also the fullness percentage wise.
For case B the placement should consider the available data on volume sizes, to calculate the free SP capacity post expansion and place the resources in a more balanced fashion.
As a bandaid solution, we are setting a default reservation on the RGs ZfscreateOptions, and this seems to steer the placer to distribute the volumes a bit better, but it is not optimal.
More generally, this also obviates the need for a mechanism to rebalance resources in a clusters, to adapt to changes of node count, storage pool number and size, as well as changes in volume numbers, sizes and IO load. I am aware of the resource migration procedure, but it is a completely manual process. Drbd/Linstor already have all the data to make an informed balancing suggestion, so ideally we’d have a command that could suggest balancing operations, considering capacity and IO load distribution, that we could then apply. This could even work automatically, if some threshold of lopsided distribution is detected. Technically a feature request.