I also have this issue. My use case is to read a pdf document and do annotations.
After I finished the document study I would like to only work with the "complete" annotation summary.
If not all the annotations are shown in the summary the whole annotation process is useless.
I hope there will be a feature request soon...