Probably not, the trajectories are simple parabolic and you can assume perfect elastic collision. Easy enough to implement, compared to interfacing with a physics engine. In fact, it sounded like he approximated the ball by a point and ignored the radius.
So yeah, you can reinvent the wheel but using specialized tools gets the job done too, usually with less painful mistakes, but that's the price to pay for the lack of learning you get with the tool.
If the physics engine can't efficiently solve the inverse problem here, it is probably only well-suited for simulating a final solution for validation, not iterating over the rather large space of possible shot trajectories times possible backboard angles at the point of impact.
Isn't solving the physics problem a straightforward ODE integration?
And that process needs to be done for all possible incoming ball trajectories that would hit that point of the backboard in order to figure out what backboard angle will maximize the number of shots that go in.
Is it though? Isn't the final shape just a paraboloid, like a parabolic antenna with the hoop in the focal point?