OP, I say go for it with no fear! I bought PVB resale about a year ago and have zero regrets. The fact I can stay at the Polynesian, that I love, for about $250 a night vs. $850 cash seems like a gift. We just had our first stay, and the rooms/resort are awesome!
I will probably add on some direct points at the tower whether it is the same association or not. And if I end up with too many points, I can easily rent them, as Poly 1 is in demand vs the cash price. You might think it is undesirable vs the tower based on some of the comments I've seen in various threads, but I 100% disagree!