I am the exact opposite, if I were to buy one of the little toy SUVs, I would go for the bigger engine. 4 people plus luggage plus the AWD drivetrain would kill most of the 4 bangers . Thats just me though, I like having the ability to pass on a undivided, 2 lane highway.
Not to accuse you of any driving deficency, but there's a huge difference on driver ability to overtake on undivided 2-lane highways. We all know or see people who can't do it at all, not even on the 2-lane uphills.
A few years ago with us in our recent Grand Vitara, another party in anther recent Grand Vitara chose to pace us for several hours through BC's interior in summer traffic on 2-lane roads with 100km+ speed limits. This meant working our way past convoys of slightly slower traffic, including plenty of semis.
The Grand Vitara is no sports car, and ours, at least, was loaded with passengers and camping gear.
Passing distances were greatly reduced by technique: leaving a larger gap to the vehicle ahead when going into a curve, and closing the gap when coming out of the bend. The moment it was possible to see there was a passing opportunity free of oncoming traffic after the curve, I'd floor it. The driver of the other GV behind me did the same thing, sometimes losing a place if there wasn't room for both of us to pass, or if I could make it past one more vehicle than they could, sometimes gaining it back. We'd work our way past each convoy, usually with a semi being the last to be passed, and enjoy some open road before coming up behind another convoy.
We stopped for gas, and lost them. They must have stopped somewhere, because they showed up behind us again, and the passing resumed.
The point being that horsepower can be bought to substitute for technique, and even supposedly "underpowered" cars have plenty of power by any reasonable standards.
Quickly getting back to the subject at hand, I suspect the V6 Rav4 would have a seriously inferior weight distribution and so would be really bad on slippery surfaces and in snow.