Me and my team understands the 300 lines of code that go into rochefort and can twist and modify it for our needs.
Performance will make or break our startup, which deals with real time user behaviour analytics, and doing high performant java for a while, I know very well how much time I will have to spend looking at G1 logs to fine tune it.
I am sure we wont use rochefort after we scale up, but for now I think gives us greather velocity than kafka (just because if we want to modify kafka we have to spend a week on a simple change).
I want to be able to add more meta information in the header, or read the files from another process, rsync and read them to my laptop, add custom reducers etc, all those things will take me minutes with rochefort and days with kafka.
Simple code with "obviously no bugs" vs complex code with "no obvious bugs".
This is one of our biggest headaches, and it's not even that a Kafka server itself is so configurable. We have hundreds of teams writing client applications, and jumping on bridges because Kafka clients have poor configurations is getting old. Too many knobs to twiddle, but I guess that's what happens if you're expecting to be able to tweak for high performance.
That just leaves the "simple work" of administering and tuning which, as you pointed out, is competence that's steadily growing in the industry.
Granted, I am not monitoring Kafka, but I do other processes.
The other nice thing is that now that I have ZK, other software that need it can just reuse the same process.
I think using the maintenance cost as a reason to write your own tool, is a short-sighted decision.