Kimmeridge Bay is a marine nature reserve and much of its interest lies under the water but even if you are not a diver, and I am certainly not, then a visit here is well worthwhile. You may like to visit the Dorset Wildlife Trust's information centre here and perhaps join one of the frequent events the wardens host. The natural history of rock pools is quite unique and a knowledgeable guide is invaluable to helping you understand what you are looking at. If you do go along on your own then do take extreme care if you go rock pooling as the rock ledges can be very slippery and keep an eye on the tide. Do not take stuff out of the pools, look at it but leave it alone!
As I said, marine wildlife identification is a specialist job and way beyond any level of expertise I may have so for my purposes so I stick to the countryside above the actual bay and the cliff walks east and west. A steep walk up the hill to the east will take you past the Clavel Tower and along the beautiful Purbeck coastline towards Chapmans Pool. To the west, pass the oil pumping 'mule' and head out to the ledges under Tynham Cap (although this area may be closed off if the army are firing as it is the home of the tank regiment training school.
Be prepared to pay Smedmore Estate for access to the area. There is a very rough car park which gets very busy in summer and dreadful toilets; you do not get much for your money in terms of facilities but you do get to see a lovely place.