We second Makers Mark with Hammonasset Beach State Park in CT. Probably the crown jewel of CT state parks. It's like a home base for us.
Not really the ocean, but it is on Long Island Sound with really nice beaches that are not overcrowded. Sites have no hook-ups and are mostly open on grass, but there are some shaded areas too and water is usually within reach if you bring extra hose. The entire campground is clean and well patrolled.
Mystic Seaport is about 45 minutes away as are the Mohegan Sun and Foxwoods casinos.
Fisherman's Memorial is RI is also worth the trip. And it has hook-ups.
Lot's of good eats in the area and you can also take the ferry over to Block Island for a day trip. You can also spend a day in Newport touring the mansions of the gilded age or doing the Cliff Walk.
Either one is a good choice, but if you have the time, try to stay at both.