For what it's worth, I can relate to your story very much, albeit with a slight twist. The last time I experienced such intense pain, I ended up fainting (first and only instance in my life) in the ER. Consequently, surgery became a must.
IMHO, considering your past experiences with similar symptoms and existing scarring, it seems advisable to forgo the appendix, especially given your susceptibility to infection. If you hadn't done it, perhaps one of the future occasions could have ended much worse.