Explore downtown Portsmouth. There are a few cute shops along Market Street along the water. A few stores of note: Bliss Boutique, Pickwick's Mercantile, Stonewall Kitchen, Book & Bar. Make your way to Market Square to see the center of downtown.
As the 3rd oldest city in US, there are lots of historic homes (Moffatt-Ladd, Warner House, Strawberry Banke Museum with 10-acres of homes from the 1600s). While they close for tours in October, you can still take a look and may be able to roam the outside grounds.
One of Travel & Leisure's best ice cream places in America, Anabelle's a must for ice-cream lovers. Before you leave the town, try it. Joe Biden and his campaign recently stopped here while on the road.