I had heard that driving to the MK could take longer than riding the bus', but I am so glad we tried it; we went there 4 out of our 7 days. We stayed at AoA and the drive from gate to gate was 6 minutes. Parking took about 2 mins, walking/taking tram to TTC was another 5 minutes (maybe a couple more when walking) and then we mostly took the ferry, but also took the monorail, and that was another 5-8 minutes. We drove to all the parks and they were all very easy access. Epcot too! In fact, Epcot and HS are the easiest.
We drive our own car too, and have always taken the buses and I have found that worrying about them had become a huge stress for me. With each child it became more stressful. More than once our last trip, we were told by drivers, "sure there is room", only to be left standing with my husband carrying a preschooler, me standing and carrying an infant, and hoping for the best for the other two. Then it dawned on me, we do not have to do this, we have a car!!!
We also found that driving (at least back from the parks) had a huge decompressing benefit for the kids. They were in their own seats and quickly relaxed and fell asleep on the way back to resort. The parking lots are so quiet and when we got to the room, they all just went to sleep. Buses at night can be noisy and then when you get off at your resort, if staying at a Value, you go through the main building where there is a lot of action and the kids get a second wind and are ready to go some more.