It's trivial to brute force for anyone who has a weak password.
More importantly, it's trivial for an adversary who cares.
If I'm encrypting a note containing state secrets to send to a foreign intelligence officer, the NSA has the technology (and more importantly, the resources) to brute force their way in.
And if your password is too complex to crack (read: a 256-bit key), you probably can't remember it either, which means you have to write it down somewhere; so an adversary who cares would find an outside channel (subpoena, hack your personal computer) to determine your key.
What is your key derivation algorithm? PBKDF2?