I realized something as I was running my tests after I had implemented all of my optimization algorithm. My initial optimization and cleanup phase removes some of the internal geometry that the second phase sort of depends on.
So those pesky invisible faces make RayCasts fail when going from row to row. I will need to alter it so that the RayCasts are not needed and I can just pick the cube up from an array instead. Well, it will make my algorithm more optimal anyway.
I should not take long to alter that particular behavior,
:p
V 