I would call a local tree trimmer they have great advice I know when they trimmed my tree they informed me to always tar the cut ends to ensure the branch lives and I think to pull on the branch and if they limb springs back that area of the tree is still good.

That is planted scary close to that house. Tipu trees get huge.

I always suggest that a tree be planted far enough away from any structure to allow for the full anticipated width and height of the tree (according to the nursery tag or actual research) at maturity.
If the tree pictured here will be "massive", now is the time to remove it. Your neighbor will appreciate it, and you will avoid causing future fence, foundation, roof or root problems by taking care of it right now. You may also avoid the expense of repairs for damage caused by a too-big-for-this-location tree.