We went car shopping last year for more or less the same reasons you state - sit higher; all-around easier on joints, in this case mine. Specifically, my left hip was giving out from driving a standard in town (way too much clutch action) and the very angled seating position in our rather low station wagon - not to mention the wear and tear from entry and egress, as I am in and out of the car a lot. We have two kids in their teens, and often pack a fair bit of gear, so we wanted something that basically was just like our Suzuki station wagon, only higher.
The two vehicles in the new car market that met our criteria were the Mazda 5 and the Kia Rondo. We bought the Rondo, with an automatic transmission, and although we do regret losing the manual, are more than delighted with the car - and after driving it for 10 months now, my hip is so much better I feel like a new woman. I'm doing stuff, like riding my bike, that I thought I'd never be able to do again.
Besides the relief from constant clutching that the automatic offers, the key attribute of the car that has made a difference for me is certainly the higher seating, which lets me sit folded to 90 degrees and not tighter. But another important aspect is the lower floor. That is something that absolutely differentiates these two vehicles from SUVs, which all purport to be off-roadable and so incorporate a high ground clearance. That means a big step up, and an entry/exit process that is as hard on joints in some ways as slinging yourself into a lower car. That off-roadability is also what boosts the price of some SUVs, as they have heavier-duty suspensions and so on (that vague ending is to conceal my lack of precise knowledge about SUVs - how am I doing?).
Other vehicles you might want to consider include the Jeep Compass and the Chevy HHR; they didn't do it for us for one reason or another but might work for you. Other models mentioned already, like the Vibe, are also good to investigate.
If the high floor doesn't bother you, you might want to investigate the compact SUV segment... Nissan X-trail (which I understand is about to be discontinued - shame), Subaru Forester or maybe even the Outback?. Kia and Hyundai also make small SUVs, and if price were no object, I'd be curious about what the Acura RDX is like inside. I'm not sure what else still fits into this category. The older Honda CRV and the Toyota Rav4 certainly did, but their newer versions are humungous.
You say you're used-car shopping in any event, and you can study the specifications, including floor height and often seating height, of most years of most models on-line.
Good luck, and keep the forum posted on your progress.
Edit: price-wise, the $23K range should get you into a new Mazda 5 or Kia Rondo, base models at least, or several other of the cars mentioned above.