Skip to content

Scalable point location - #5396

Open
leo-collins wants to merge 139 commits into
mainfrom
leo/distributed-rtree-refactor
Open

Scalable point location#5396
leo-collins wants to merge 139 commits into
mainfrom
leo/distributed-rtree-refactor

Conversation

@leo-collins

Copy link
Copy Markdown
Contributor

Introduces the partition_rtree, which is an R-tree of bounding boxes covering each partition of the mesh. This R-tree is replicated on all ranks by allgathering the bounding boxes. The bounding boxes on each rank are obtained by traversing down the local R-tree. Each rank may submit more than one bounding box to the partition_rtree (we want to do this to minimise candidates since the mesh partitions may not be nice and convex, and might even be disconnected).

We choose the number of bounding boxes via a simple heuristic (we stop traversing when the total volume of the bounding boxes stop decreasing - see _box_ratio_heuristic). This should probably be improved to take into account the total number of bounding boxes so we don't blow up the allgather.

Constructing the partition rtree uses an allgather, although each rank normally submits O(10s) of boxes so I have not seen this kill us yet when testing up to 2048 ranks. If doing huge number of cores we might want to consider implementing some sort of hierarchy of partition rtrees.

Each rank queries this R-tree with the points local to that rank to determine the 'candidate' ranks for each point. We then build the candidate_sf which is a PETScSF mapping input points to their candidates. The local cell location process proceeds as usual, solving parallel conflicts with broadcasts/reductions via the candidate_sf.

This current implementation gets around 80% weak scaling efficiency in my tests up to 2048 cores.

@leo-collins leo-collins added ci:complex Run the test suite in complex mode ci:macos Run the test suite on the Mac runners ci:int64 Do a 64 bit integer build labels Aug 27, 2026
@leo-collins
leo-collins marked this pull request as ready for review August 29, 2026 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:complex Run the test suite in complex mode ci:int64 Do a 64 bit integer build ci:macos Run the test suite on the Mac runners

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant