The evidence we have for safety of low-level consumption doesn't look great.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7691614/

"The paucity of adequate studies, particularly related to long‐term use of A. muricata supplements, currently does not allow the establishment of a safe intake level."

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10092620/

"Cognitive performance assessed with the MDRS worsens above a cumulative consumption threshold of 0.2 fruit‐years of Annonaceae fruit/juice (ie, one fruit every 5 days for 1 year) or with consumption of any quantity of Annonaceae herbal tea."