What plugin are you using to "hide" Wordpress?
The first time it happened, I just deleted the file and went forward. The second time, I just deleted the app. I know I could write a script to deal with this, but it was just a test, playground type app. All it did was make me say I needed to move my "blog" off WordPress and stop trying to host Rails apps on Webfaction.
PS: The domain the Rails app was on had never hosted a WP site ever before. WP is just so ubiquitous that that they were just trying to hack any site -- even one that had like 3 organic visitors ever.