A breaker panel isn't just used for overcurrent protection, though—it's also a standard point to disconnect power when needed. The possibility of house wiring being energized even when all power to the house is cut is dangerous for anyone not familiar with the setup. This could be lethal if work needs to be done on it and OP isn't around to explain how it's wired up.