Playing with the AABB tree and queries, there are a few things that have popped up. Such as, for doing ray and point tests, my BSP trees are 3 – 4x faster. But, when making broader queries such as testing a bound against an AABB tree, it is quite fast at handing you all the leafs within that volume. So, each solution has its wins and losses and best fits.
Making my editor give good previews in good time, I will still distribute the computations over a few frames to help it grow with the size of the models.
A. Make a broad sweep against the AABB tree to find all the leafs that fit within each others top level AABB.
B. Take the results from the former step and reject the leafs that do not collide with any other leafs in the other tree.
C. Cut the leafs from the AABB filtering with BSP tree and collect the in and out bits along with the intersection contour.
D. Process the remaining leafs by classifying their centers of mass against the BSP tree to see which ones are inside or outside of the volume.
E. Collect the in and out polygons to create a final preview mesh.
Each step could be distributed across frames and then I can also apply some effects and niceness to make it more visually appealing.
Then when the user applies the operation, it takes the input and feeds it to the robust and heavier back-end for good and correct results.
Anyway, time to sleep.