Gravel is a big PITA to work on. Not only for the comfort factor, but also because floor jacks really don't like it. The wheels need to roll as the arm lifts up and back, and they just get stuck in the gravel, dragging the car sideways instead.
Hmm, I wonder if a thick (at least 3/4"?) piece of plywood under the jack would work?
And now that I think about it, using the floor jack on gravel was probably how I bent the pinch weld on the GTI.