Steel is sooooooooooooo much harder than al, it’s not even in the same ballpark. Aluminum is about as hard to cut as wood. For non round holes, you are looking at approximations or broaching. Look up manual hex broaching