Core Matlab/Octave is great but almost inevitably you end up needing/wanting toolbox support. For Matlab the cost gets up there quite fast. I've found for Octave, support at the toolbox level is not as extensive. If Matlab is not an option it leads to searching for support in other languages. OpenCV for image processing/understanding and the python libs as mentioned become important for example. But development cycle time is longer and prototyping algorithms is a bit more complicated.
Its been a while since I've worked with Octave toolboxes but it would be great if they were on par with Matlab.
Performance is less of an issue with prototyping than production systems and Matlab has a path to get to compiled C/C++ (Java and perhaps other?) Is there Octave support for generating reasonably performant C++ or Java?