IIRC the player's guide I have (which I can't presently locate) says something along the lines of, "You can go to the Mahogany Gym first, if you want an extra challenge," which makes me think you'll probably find Jasmine and Chuck easier than Pryce, but that's it, really.