Two wonderful herbaceous plants I have are American Spikenard (Aralia racemosa) and Baneberry (Actea pachypoda). Both flower in summer and fruit now. In both cases the berries are considered poisonous. They both seem to be pest free (if you don't count racoons knocking the baneberries over), never mildew and come back reliably every year.
|American spikenard - a relative of Sarsaparilla|
This is one of two plants and is the smaller at two feet tall. The other gets more sun and is close to five feet tall
Baneberries can have red fruit or white fruit. I have tried many times to buy the red fruited one but it always turns out to be white. These are also very difficult to propagate. They don't seem to do it own their own, and collecting mature fruit to clean and sow with stratification is not working for me at all. The white berries are affectionately called "Doll's eyes".