You don't need to micro manage everything at Universal like at Disney. It's one of the reasons we love it. However, if you are going at a busy time and don't have express pass, you will want to do some pre-planning to minimize waits.
How much you need to plan really depends on how much you are ok with waiting and what all you want to do. 4 days is more than enough time to do everything even if you wait in standby lines every day, but do you really want to?
Without express pass, arriving early is key to avoiding lines. If you can get ahead of the crowd in the morning, you can walk on and off a number of rides very quickly without waiting. Prioritizing the most popular rides early will help as well. You may also find that returning to your resort or even visiting a different resort (resort hopping is allowed) during mid-day is a good option as break from the crowds. When we don't have EP, we try to arrive an hour before early entry, ride everything we know will get a long line, stop for brunch / lunch about 10:30 / 11:00, and are usually heading back to the resort for a nap or a swim about the time everyone else is really starting to crowd the parks. This has always worked well for us.
I second the recommendation to look at the Touring Plans website. It should help give you an idea of which rides to prioritize when.
Another option is to look at possibly getting a throw-a-way room at one of the premier resorts for 1 night. This would give you 2 days of express passes for everyone on the room reservation. You get express passes for your check in and check out day. We did this our first trip, and it worked well. We focused on the rides with the longest waits and the ones the kids wanted to repeat ride over and over again on the days with express. We saved shows, parades, shopping, longer meals, play areas, characters meet-n-greets, and rides with shorter lines for the days we didn't have the passes. It worked great for us and gave us a little peace of mind to not stress so much our first trip as we got used to Universal.
As far as other planning not wait related, there are tons of fun Easter Egss, photo ops, play areas, and just little details in general that most people walk right past and never even notice. You might want to look into some of that before going. I posted a response here last week to someone asking for non-attraction activities that lists a number of those things, but there's way more out there if you do a google search.
Along those same lines, if you are HP fans, do some research on those areas as well. There's more to see besides just the main rides. For instance, bring a stamp and postcards or a letter, and the kids can send mail by owl post back home or to their friends. You can buy stamps and postcards there, but much cheaper to get them ahead of time and just mail them at the owlry.
Take time to familiarize yourself with all the dining options, table and quick service. Some of the quick service places are as good as table service with amazing theming. With everything at Universal so close together, it's no big deal to hop lands, parks, or go to Citiwalk if someone is cravings something specific.
Universal does offer a disability pass, but they recently changed the way you apply for it, and from what I have seen, people have had very mixed experiences. It is a lengthy process now involving a third party, documentation, and a phone interview that all needs to take place before you go, so that may be something you want to look into. It would need to be done at least a month ahead of time. If approved, then your entire family would either be directed to the express lines or allowed to virtual wait depending on the wait time of the attraction you want to visit. This could help with the child swap situation you mentioned.
Here's really the rest of what you need to know planning wise.
1. Citywalk. Citywalk is Universal's much smaller version of Disney Springs. It has a movie theater, mini golf, and an escape room as well as restuarnts and bars. Everything at Universal can be accessed from Citywalk. You enter both parks from Citywalk and transportation and walking paths to the resorts all come from Citywalk.
2. Security. You go through security before entering Citywalk, so you do not go through it again when entering either park. This makes switching parks and hopping to and from Citywalk and the parks, super quick and easy. You can take food and drinks (non-alcoholic) through security.
3. You have 3 options for transportation to the parks from CB. 1) You can take a bus from CB to the Citywalk drop off area. Then, you go through general off-site guest security and enter Citywalk by the mini golf and Bubba Gump area. 2) You can walk down the garden path from CB to Citywalk, arriving on the IoA side near Margarittaville. It's about a 20 minute walk and has it's own dedicated security at the end of the garden path. It's a very nice sidewalk path that never crosses any roads with plants and trees alll around you. You pass the Sapphire Falls and Royal Pacific resorts on the way, so it's easy to stop at one of their pools on an afternoon walk back if desired. 3) You can walk to Sapphire Falls and take the boat to Citywalk. Security is at the boat dock before you board in this case.
4. Universal allows resort hopping. You can use the pool or gym at any other resort on the main property. You simply go to concierge at the resort you want to visit, and they will give you a pass that is good for the resort.
5. Park hopping. Unlike Disney where it can take 2+ hours to change parks, it's very quick and easy at Universal. The walk between the two park emntrances is 5 minutes. It's more like switching lands at Disney than switching parks, so it's not a big deal to switch back and forth multiuple times in a day.
6. The water rides get you wet. I don't mean a little wet. I mean soaked like you jumped in a pool with your clothes on. I've even tried ponchos before and still gotten soaked because the water came up from below. We won't ride the water rides unless we are in our bathing suits. You must have actual clothes on, though, so I usually do a tankini and quick dry athletic wear over the bottoms. Just plan accordingly with a change or clothes or something.
7. Metal Detectors. Some rides require you to go through metal detectors before you ride. You cannot take anything with you on these rides, so be prepared to leave your stuff in a locker or with another person not riding to hold. You may want to look up the size of the lockers. It is quite small.
8. Universal purchases. You don't have to carry souvenirs around with you. You can either send them back to your resort, or you can send them to the Universal store in Citywalk to be picked up later. The Universal store is open much later than the actual parks are, so it is easy to stop in and get your packages at the end of the night, even after a late dinner.
9. Wear shoes that don't fall off. This is something I've noticed with people used to going to parks like Six Flags. They are use to just leaving their shoes at a ride and riding barefoot. This does not work at Universal as you do not enter and exit most rides from the same location, so you really need to have shoes on that are not going to fall off of your feet if your legs are dangeling down.
10. Motion Sickness Meds. Almost everything at Universal, even kiddie rides, try to make you sick. They have toned down the a few rides, and it is better, but they still have so many that are very motion sickness inducing. Nothing makes me sick at Disney. Everything does at Universal. I have to get a patch from my doctor for Universal. Just something to think about.
The last advice I will give, especially based on your husband's comment, is do what works for your family. If you guys normally plan and that makes everything go smoother, than plan. Our first trip, I also read that we didn't need to schedule every minute like we did for Disney, so we didn't, and it was a huge mistake. The kids were used to extremely scheduled vacations. Not being on a schedule ended up causing anxiety and behavioral issues. Worries about not being able to fit in something someone wanted to do, disagreements over where to eat, debates about what to do next, general anxiety about not knowing what's next, etc. It was a nightmare. I never would have guessed having a relaxing trip would turn into such a mess. Half way into the trip, in the middle of the night, I sat down and planned out a detailed itinerary for our last few days. I just couldn't take it anymore. I wish I had planned everything out all along. Was it neccessary for touring Universal? No, but was it neccessary for my sanity? Very much so! It was what they were used to and everyone just functioned better with a detailed itenary in place of what was happening when. I never would have dreamed not planning would cause such stress but it did, so just something to keep in mind if your family is used to very scheduled trips.