HI, Mine is by the very back fence, well away from any houses. We have mice, moles, shrews , raccoons .. you know , all the usual suspects. I find a bigger issue from my trash and recycling bins and bird feeders.
The trick is... no meat, bones, animal waste. IT is grass clippings, leaves, organic material from kitchen ( I don't bother in winter tho) like coffee grounds, onion, potato peelings etc. It is also in the shade. Flip it now and again, and water if it is dry, but don't soak. Good luck!
Oops, i meant to say, the varmints are in my area, but don't seem to bother with the compost. The birds might pop by for some stuff for nesting tho.