A bite of Python
access.redhat.com
access.redhat.com
"Contract conditions should never be violated during execution of a bug-free program. Contracts are therefore typically only checked in debug mode during software development. Later at release, the contract checks are disabled to maximize performance." - https://en.wikipedia.org/wiki/Design_by_contract
That's almost a tautology though.
In this particular case, you compiled your code with "-O", so it's not the "same code" used in production, but code compiled with a different flag. Shouldn't they check what the flag does?
I agree with the way it's done in Python, as it's consistent with most other languages. But the blog post is right to point out to inexperienced developers that the way assert behaves might give them a surprise.
Would be nice if they had a pointer to the reason, however.
"However, Python does not produce any instructions for assert statements when compiling source code into optimized byte code (e.g. python -O). That silently removes whatever protection against malformed data that the programmer wired into their code leaving the application open to attacks.
The root cause of this weakness is that the assert mechanism is designed purely for testing purposes, as is done in C++. Programmers must use other means for ensuring data consistency."
Python `-OO` strips docstrings, `-O` basically only disables assertions. See: http://stackoverflow.com/a/4777156/459543
Or just do "python -h":
-O : optimize generated bytecode slightly; also PYTHONOPTIMIZE=x -OO : remove doc-strings in addition to the -O optimizations
Same on both Py 2 and 3.
Real or imagined constraints? AFAICT, an assert only tells me what you wish your program did, but that has absolutely no bearing on what it will actually do.
An asserts checks DESIRED constraints.
>AFAICT, an assert only tells me what you wish your program did, but that has absolutely no bearing on what it will actually do.
Depending on the implementation, an assert can either merely log or absolutely stop a program that doesn't pass its test, so it very much has a bearing on what the program will actually do.
Then imagined it is. Your desires are totally a part of your imagination, unless you make them become real.
> Depending on the implementation, an assert can either merely log or absolutely stop a program that doesn't pass its test, so it very much has a bearing on what the program will actually do.
Point taken. Unfortunately, logging errors or aborting the program won't make assertions magically become true, though.
The assertion remains false for the final process state, before the process quits. Outside of the process, the assertion is simply meaningless (neither true nor false), because the assertion's free variables are only bound inside the process.
Which is inconsequential. Programmers don't expect automatic "recovery" from assertions, they expect them to notify them of the violated constraint, and/or to ensure that a program wont go on and use a value that violates an assertion further down -- which program termination achieves.
>Outside of the process, the assertion is simply meaningless (neither true nor false), because the assertion's free variables are only bound inside the process.
Captain Obvious?
Users don't expect failed asserts at all.
> Captain Obvious?
Apparently not obvious enough for JonnieCache, who said: “If you have the process quit it definitely stops them from being false though.”
He meant from being false subsequently in the program.
I see we were joined by Technically Correct Man too.
http://9gag.com/gag/a5PmrLq/technically-correct-man-the-man-...
But, you see, the assertion is no less false just because the process was aborted. The fact remains that there exists a reachable state for which the assert fails. So apparently what I meant is no more obvious to you than it was for JonnieCache.
Yes, Captain Obvious, and that reachable state is exactly what every programmer who uses an assert() statement expects when he writes it.
If there wasn't the potential for such a state, assert statements would do nothing ever in the first place -- so it would be kinda silly to even have them in.
Only in the most pedantic and useless sense of the term.
Asserts are not just some random imagination, they are added based on the program's specifications and expected/desired functionality and constraints. The CS term for those kind of constraints are "invariants", and asserts are a way to be notified if those invariants are violated.
>unless you make them become real.
Only there are no assurances for that. If the invariants in your program were somehow guaranteed to be "real" then you wouldn't need asserts.
Asserts are there because whether you tried to make your invariants "become real" or not, you'll still miss things, have bugs, have unexpected interactions with code/systems outside your control etc. So they are there to tell you about those misses.
>Point taken. Unfortunately, logging errors or aborting the program won't make assertions magically become true, though.
Assertions are not expected to "magically become true" -- just to (a) inform about anytime they are violated, and, optionally, (b) not be violated and still have the program continue to run.
An “invariant” is a function of the process state whose value remains constant (hence “invariant”) in spite of changes to the process state. Perhaps you meant “precondition” or “postcondition”?
> Only there are no assurances for that. If the invariants in your program were somehow guaranteed to be "real"
Guaranteeing that preconditions, postconditions and invariants hold when they're supposed to hold is your job, not the computer's.
> then you wouldn't need asserts.
I absolutely don't need asserts. An assert merely describes what you want, but that's useless to me, unless you establish a relation between what you want and what your program actually does - with proof.
> Asserts are there because whether you tried to make your invariants "become real" or not, you'll still miss things, have bugs,
It will become patently clear when the proof doesn't go through.
> have unexpected interactions with code/systems outside your control etc.
What happened to sanitizing input at system boundaries?
> Assertions are not expected to "magically become true"
Of course. Assertions are expected to always be true.
No, I meant invariant. An invariant is something that is supposed to hold true, not just things that are guaranteed to hold true (e.g. a constant that can't ever change anyway). That's why the need for assertions to check that invariants hold.
From Wikipedia:
"In computer science, an invariant is a condition that can be relied upon to be true during execution of a program, or during some portion of it. It is a logical assertion that is held to always be true during a certain phase of execution. (...) Programmers often use assertions in their code to make invariants explicit."
Preconditions and postconditions are similar in concept, but are supposed/wanted to hold true before (pre) or after (post) a method runs.
>I absolutely don't need asserts. An assert merely describes what you want, but that's useless to me, unless you establish a relation between what you want and what your program actually does - with proof.
Well, asserts weren't created specifically for you. Feel free not to use them.
They are useful to me, and assuming from their widespread use, others, even if they don't formally prove the program does 100% that it needs to (which nobody expected them to anyway).
Until we all program in Coq or similar, they will be useful for all kinds of checks. A correct program is a spectrum, not a binary option.
>Of course. Assertions are expected to always be true.
No, they are also expected to be false -- that's why we add assertion statements to check whether our assertions hold. But we're splitting hairs twice or three times here.
No, you need proof.
> Until we all program in Coq or similar
So you're saying humans are fundamentally incapable of establishing the logical validity of what they assert by themselves? This contradicts historical evidence that people have done this for well over 2 millennia, using various methods and tools.
> A correct program is a spectrum, not a binary option.
Some errors might be easier to fix or have less disastrous consequences than others, but a correct program is one that has no errors, so I don't see where the spectrum is.
You might need proof, but it doesn't mean you'll get it. In most languages in common use (e.g. not Coq and co) and for any larger than trivial program "proof" is impossible.
So, we'll continue to need all the tools we can realistically use, including assertions, unit tests and others.
>So you're saying humans are fundamentally incapable of establishing the logical validity of what they assert by themselves? This contradicts historical evidence that people have done this for well over 2 millennia, using various methods and tools.
This particular question is not even wrong in the context of the discussion. I don't usually throw around the term "troll", but you're either trolling or being alternatively naive on principle / too pedantic.
I any case, whether people are "capable of establishing the logical validity of what they assert by themselves" for trivial things or for narrow domains, the absolutely have not been able to manually do it, or do it fast enough to be practical, for software programs, especially any non trivial one. Even the best programmers introduce bugs and have behavior in their program that they didn't expect.
Which is also why even the best programmers use assertions. It's not some obscure feature relegated to newbies or bad programmers. It's a standard practice, even in the most demanding and hardcore programming environments, from the Linux kernel (which uses the BUG_ON assertion macro) to NASA rocket code.
Or I could turn "troll mode" on an answer on the same vein as the question: if "people have done this for well over 2 millennia, using various methods and tools" then they haven't been doing it "by themselves" any more so than when using assertions (which is also one of such "tools").
And of course, I haven't anywhere stated that "humans are fundamentally incapable of establishing the logical validity of what they assert by themselves".
The gist of my comment would be merely that humans are bad at establishing the logical validity of their computer programs by themselves -- for which there is ample "historical evidence".
>Some errors might be easier to fix or have less disastrous consequences than others, but a correct program is one that has no errors, so I don't see where the spectrum is.
The spectrum is obviously in that correctness is not black and white, and all non trivial programs have bugs in practice. Those programs with few and far between bugs are more correct than others.
Or, in other words: http://chem.tufts.edu/answersinscience/relativityofwrong.htm
> The root cause of this weakness is that the assert mechanism is designed purely for testing purposes, as is done in C++.
However, C and C++ are perhaps unique in how much undefined behavior is possible and in how simple it is to create. Inserting into a vector while iterating through it, for instance. Or an uninitialized pointer.
That's why many C++ experts believe in runtime assertions in production. Crashing the application with a core dump is generally preferable to trashing memory, corrupting your database, or launching the missiles.
I imagine they would say that your statement about crashing vs. e.g. launching the missiles is a false dilemma. You don't crash and you don't incorrectly launch the missiles.
I'm not a C++ developer so I can't say it with certainty. I more agree with what you're saying. I'm just relaying that my experience has been that out of many different language communities, C++ actually seems adamantly the opposite of what you're describing.
Higher level languages are better at reporting uncaught runtime errors than C/C++ is, because they'll automatically do things like print useful stack traces and then exit gracefully even if you don't catch an exception. The interpreter doesn't crash when your code does.
Also, if you don't catch an exception in C++, most systems will give you a full core, which includes a stack trace for all running threads. Catching an exception and 'exiting cleanly' actually loses that information.
Exactly. "Undefined behavior" includes showing private data to the wrong user and booking ten times more orders than the user originally indicated. I'll take crashing over that.
1. Stack trace at point of a contract violation tends to capture the most relevant context for debugging -- the faster it is to discover and debug an issue the easier it is to fix 2. Interacting code has to become sufficiently coupled to preserve "sane program state" -- an exception may or may not be recoverable -- a fatal error never is and there's no point in building code to try to recover. If the programmerer has to design the interaction among program components to avoid fatal errors then there must be fewer total states in the program vs a program which recovers from errors -- this makes the program easier to reason about. 3. On delivering good User experience -- id rather have clear and obvious crashes which are more likely to include the most relevant debug information -- than delivering the user some kind of non-crash but non-working, behavior (with possibly unknown security consequences) which may take longer to get noticed and fixed as a result of an error handling mechanism that deliberately _tries_ to paper over programming problems ...
I've actually modified third party libraries I've used to remove catch blocks or replace error handling within with fatal errors -- when dealing with unknown code it really can vastly speed up the learning process and the understanding based on observational behavior ... -- especially in understanding the behavior around edge cases.
fun_call('string1',
'string2'
'string3')
That is, missing commas and subsequent string concatenations can lead to nasty errors. I wish Python didn't nick this from C and would have just enforced the use of + to concat over-length strings, if they need to be split to multiple lines. subprocess.check_call([
"/usr/bin/env",
"echo"
"hello world"
])Then yes, I rely on the linter and testing.
I like this string catenation behavior and I prefer it, even if it causes some confusion in (IMO rare) cases.
Nothing mind-changing in here. Translations seem to take the biggest hit, but it's largely a matter of company conventions if this is a problem. IMO the grounds of rejection weren't discussed very thoroughly.
I could imagine the new f-string interpolation might have some nice synergy.
It will actually pick up a number of security issues listed in the post. It's useful in real world too - led to a number of CVEs being reported.
Don't suppose you know if that's gotten better?
- `bandit-config-generator` will give you a file filled with the current/default configuration, so it's a simple way to start with the defaults and modify just what you need
- if you just need to enable/disable tests rather than reconfigure, you can do that in command line options
- if you want to get rid of specific warning, you can mark the line with "# nosec" in the source
Merging various configs is possible, but rather complex to implement considering we aim for the config to be a complete description that won't ever need to change between versions.
If none of the above workarounds solve your use case, feel free to report an issue. (https://bugs.launchpad.net/bandit) I can't guarantee how/whether we'll fix this, but we'd definitely like to know what the problem is and how you're trying to use Bandit.
The config uses YAML format.
Regardless, config errors should be more verbose. I raised https://bugs.launchpad.net/bandit/+bug/1621552
pypy2-v5.3.1-linux64 / [PyPy 5.3.1 with GCC 4.8.2]
If you can't reproduce it with that tarball I'll dig deeper to see the mechanism of failure, maybe it's not pypy and it's just something local to my config or venv.
cd path/to/bandit
virtualenv -p path/to/pypy venv
venv/bin/pip install -r./requirements.txt -e .
venv/bin/bandit-config-generator -o tmp_file
venv/bin/bandit -c tmp_file -r path/to/some/project virtualenv -p `which pypy` ~/pypy_env
source ~/pypy_env/bin/activate
# indeterminate <but probably critical> changes to this venv
pip install bandit
bandit-config-generator -o tmp_file
bandit --help # "The following sets..." is empty
bandit -c tmp_file -r path/to/some/project # gives the error regarding config file parse failure
I can reproduce the parse error given the nearly empty config file, but it's not clear to me whether the parse error is expected in this case or not. $ echo -e 'tests:\nskips:\n{}' > parse_err.cfg
$ bandit -c parse_err.cfg .
[main] ERROR parse_err.cfg : Error parsing file.So, basically, I can write such a list for every language I know.
https://stackoverflow.com/questions/tagged/hidden-features?s...
This can be said for every industry involving people.
Here's an example of its expressiveness a colleague and mine I discussing the other day: Python: [os.remove(i.local_path) for i in old_q if i not in self.queue] Java: old_q.stream().filter(i -> !self.queue.contains(i)).map(i -> new Path(i.local_path)).forEach(Files::delete);
I've programmed in both languages but joked I could only understand the Java line by using the Python line as documentation!
The Python is example is both short and obvious, whereas the other examples tend to be either cryptic or complicated.
I think your Java example only looks gross because it's using ugly APIs, and isn't indented well, but otherwise, apart from contrived examples, pipelining is superior.
I find Python's lack of pipeline capability, whilst every other modern language supports it, very frustrating. JavaScript, Scala, Swift, Rust, Ruby, Elixir, C#, F#, Java, Kotlin <-- all support pipelines.
Meanwhile, Python has borked, 1-line, lambdas that compose awkwardly with map/filter (if you do a map over the result of a filter, they'll be written in reverse order), and refuses to implement useful methods on lists, that would allow pipelining. It's like it can't decide to pick the OO solution to the problem (and add the methods to lists) or to go the FP route (and fix its lambdas), so has done neither.
So we're stuck hoping our problem at hand fits neatly into a list comprehension, which still won't be composable when we come back to it and realise we want to add another operation.
I like Python very much, but this is one of it's weakest areas in my opinion, so I'm surprised you bring it up as a strength.
For example: does Python pass function arguments by value or by reference? Neither! It passes them by object reference - not by variable reference like C/C++.
Check out:
>>> def foo(a):
... a = 2
...
>>> value = 1
>>> value
1
>>> foo(value)
>>> value
1
and: >>> def mutate(dct):
... dct['foo'] = 'bar'
...
>>> value = {}
>>> value
{}
>>> mutate(value)
>>> value
{'foo': 'bar'}
This apparent contradiction confuses a lot of people. The first example would imply that Python's pass-by-value, but the second looks a lot like pass-by-reference. If you don't know the actual answer, it looks magical and inconsistent, and I've heard all sorts of explanations like "mutable arguments are passed by reference while immutable objects are passed by value".In reality, the object itself - not the variable name referring to the object - is passed to a function arguments. In the first example we're passing in the object `int(1)`, not the variable `value`, and creating a new variable `a` to refer to it. When we then run `a = 2`, we're creating a new object `int(2)` and altering `a` to point to the new object instead of the old one. Nothing happens to the old `int(1)` object. It's still there, and the top-level `value` variable still points to it. `a` is just a symlink: it doesn't have a value of its own. Neither does `value` or any other Python variable name. That's why the second example works: we're passing in the actual dictionary object and then mutating it. We're not passing in the variable `value`; we're passing in the object that `value` refers to.
The point of this long-windedness is that Python's rules tend to be very, very simple and consistent. Its behavior can be unexpected if you don't truly understand the details or if you try to infer parallels to other languages by observing it and hoping you're right.
dicts are mutable, so, it doesn't create a new object to make an assignment
numbers are immutable, so, it does.
Python is always pass-by-object-reference, and assignment is always a pointer operation. It's not special-cased like you're describing.
>>> v=2
>>> id(v)
4148001784
>>> id(2)
4148001784
>>> def m(a):
... a=3
...
>>> m(v)
>>> id(3)
4148001800
>>> id(v)
4148001784
>>> v
2
>>> l=[]
>>> id(l)
4140973392
>>> def m2(lst):
... lst.append(1)
...
>>> m2(l)
>>> l
[1]
>>> id(l)
4140973392When I switched my analysis to Python, I became so much more productive. And other science researchers I have known have echoed this sentiment. Even writing a C module to speed up my Python was pretty straightforward, if tedious.
Python had the fewest surprises. And debugging other people's Python is exponentially less annoying than debugging other people's Fortran or C. It's still my go-to language to get stuff done without fuss.
Perhaps your experience with objects in other languages has given you a different mental model for what an object is. I find Python objects to be more straightforward than in other languages, especially because classes are objects, too.
And as for method declaration, if you aren't satisfied with implicit self, I much prefer Go's choice of having you declare the self reference for methods before the method name, instead of in the argument spec list (which then doesn't match the calling list). Python's way makes it feel like the compiler writer couldn't be bothered to hide the OO implementation on the declaration side, but embraced it on the calling side.
Meh, I know these have been hashed over a thousand times here. Just some of the things that rub me the wrong way when I've tried to deal with Python.
The explicit self came from Modula-3. https://docs.python.org/2/faq/design.html#why-must-self-be-u...
The explanation for len is a little lacking. https://docs.python.org/2/faq/design.html#why-does-python-us...
Guido once explained further in an email to the mailing list. I've forgotten some of it, but the gist is that he didn't want anyone to accidentally create or override a .len() method to do something other than tell the number of elements in the container.
And... almost everything is a method. Even ``len(obj)`` is just sugar for ``obj.__len__()``.
Not "almost". Just "everything is a method", in terms of function calls. Even user-defined standalone functions. Consider:
def my_func(arg):
return arg + 5
The following are equivalent, and show how things work: my_func(3)
and my_func.__call__(3)
and types.FunctionType.__call__(my_func, 3)Python is a multi-paradigm language in the sense that it does not explicitly force the programmer to write all code in a particular way. So, for example, Python does not forbid the existence of standalone functions, or the execution of functions without looking them up through a class of which they happen to be a member.
But given that it is inescapably true that every function call in Python is translated to a call of a method of an object, it's hard to argue that it isn't "really" OO.
so is str and other shortcuts
I have to maintain a codebase of php/perl/java/python. "Pythonistic" programming seems to encourage finding the shortest/fastest way to code things at the expense of clarity.
Plus dependencies can get headachy. This might just be the code I have to work with, but while better than perl, in my case its harder to maintain than java or php, (the global scope thing in python seems to get me).
It's because the comparison operators are defined for every value. That is, "True < []" is valid in Python 2.7, along with any other 2 values, regardless of type. This is a surprising instance of weak typing in Python, which is otherwise strongly typed, which is why this was fixed in Python 3 (https://docs.python.org/3.0/whatsnew/3.0.html#ordering-compa...).
This is also not a case of Python doing something useful, like with '"foo"*2'. The result of the comparison is defined, but it's not useful. I suppose it was useful for making sure that you can always sort a list, but there are better ways to do that.
Do you mean this example? (it's the only one I can find about floating point comparison)
> 2.2 * 3.0 == 3.3 * 2.0
It's definitely due to accuracy error. (rather than type comparison) How would you explain it otherwise?
And you're completely right there. Any language using floating-point numbers will have the same issue regardless of its typing discipline e.g. Rust: https://is.gd/4BNoWa
>>> float > float('infinity')
TrueBasically, autoconvert the above to this (and make Fraction a builtin instead of in the standard library, of course):
>>> Fraction('2.2') * Fraction('3.0') == Fraction('3.3') * Fraction('2.0')
TrueAlso, I think that '2.2' is better represented as `Decimal`, as it's a decimal number (which is a subset of rational numbers, that are usually better represented using `Decimal`) (edit: that of course depends on the use case, as Decimal uses fixed-point precision).
Intertype comparison is useful for sorting a list of heterogenous types. But for Python 3 they decided that use case wasn't so important.
There are nonetheless good remarks about poor design choices of Python which can lead to misconceptions to newbies, such as naming `input` the function that does `eval(raw_input(prompt))` (as casually documented[0]), and the existence of such function in a first place.
[0] https://docs.python.org/2/library/functions.html?highlight=i...
"Being easy to pick up and progress quickly towards developing larger and more complicated applications, Python is becoming increasingly ubiquitous in computing environments".
Why would you change the order of the subject in such an unreadable way? Isn't much easier to say:
"Python is becoming increasingly ubiquitous in computing environments, as it's easy to pick up and progress quickly towards developing larger and more complicated applications"
I'm not expert in writing, it just sounded weird. If anyone can explains what's going on there, really appreciated.
It's best to avoid passive voice, but sometimes that requires difficult thinking about who is taking action. In this case, who is causing Python to be deployed in more computing environments? How about this revision:
"With increasing frequency, software engineers and system administrators are choosing Python, because the language is easy to learn and productive for developing large, complex applications."
I'm not sure that's how I'd explain Python's popularity.
If Python used the Perl 6 model, you could still use floats by writing your literals in scientific notation, so if you want floating-point performance, you can still get it. For example, 2.2 would be a Fraction, but 2.2e0 would be a float. I don't want to eliminate floats from the language, just hide them from average users by default.
And it's not like rationals-as-default are just some weird Perl 6-ism. Haskell does the same thing, and the language is fairly well-received.
> Also, while nice, Fractions have their own pitfalls due to potentially catastrophic runtime behavior.
Elaborate?
Why is this relevant for this article? The article doesn't say anything about attackers having write access to the source.
> The mitigation is to maintain secure access permissions on all directories and package files in search path to ensure unprivileged users do not have write access to them.
Check out the "yes"es in the "fixed" column in comment at https://bugs.python.org/msg85966
The same is true for module imports... If you have write access to the same directory as the code itself there's all sorts of havoc one can cause beyond merely substituting your own os.py.
The feature is just some syntactic sugar.
When within a class, private fields such as:
class Foo:
def __init__(self):
self.__bar
are accessible from within other methods of class Foo as `self.__bar`. But that's just syntactic sugar, the real name of `self.__bar` is `self._Foo__bar`.So from the outside "world", including `hasattr()`, you can still access `self.__bar` as `Foo()._Foo__bar`.
>>> class Foo():
... def __init__(self):
... self.__bar = 'hello'
... def show(self):
... print(1, self.__bar)
... print(2, getattr(self, '__bar'))
...
>>> foo = Foo()
>>> foo._Foo__bar
True
>>> foo.show()
1 hello
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 6, in show
AttributeError: 'Foo' object has no attribute '__bar'
>>> foo.__bar = 'world'
>>> foo.show()
1 hello
2 world
In the end, when `x.__private` is setup outside of the class definition, obviously, it's a new member as its name differs from the internal name `__private` (which really is `_X__private`).From within the code doing `getattr('X', '__private')` will return the `__private` setup from outside the class, and `getattr('X', '_X__private')` the one defined from within the class.
The whole point of that feature is to ensure that members defined within a class that are not part of the public API are left untouched when that class get subclassed, to avoid unexpected behaviours.
Here's an example of why this has been designed:
>>> class A:
... def __init__(self):
... self.__internal = "this is a"
... def show(self):
... print(1, "A", self.__internal)
...
>>> class B(A):
... def __init__(self):
... super(B, self).__init__()
... self.__internal = "this is b"
... def show(self):
... super(B, self).show()
... print(2, "B", self._A__internal)
... print(3, "B", self.__internal)
...
>>> B().show()
1 A this is a
2 B this is a
3 B this is b
>>>
There's nothing that should be surprising or asymmetrical to anybody who've read the python documentation, and use that feature appropriately. It's maybe a weird feature, but it's still a coherent and homogeneous behaviour and actually adding more safety to codes.Documentation references:
* https://docs.python.org/3/faq/programming.html#i-try-to-use-spam-and-i-get-an-error-about-someclassname-spam
* https://docs.python.org/3/reference/expressions.html#atom-identifiers/That/ includes double underscores mangled members and and "magic" methods.
I would not call it "syntactic sugar", but rather a leak of implementation details. It could be deliberate, like Perl did for its OO (showing the entrails of all its objects), but it's not particularly sugary-sweet-yummy.
> Explicit is better than implicit
So there's no "leaking" of implementation details, because the implementation shall always be fully exposed.
As said in a sibling post, the private fields are just public fields, which are not documented as part of the public API and start with a `_`.
And as I said in the parent post, the reason to use the name mangling mechanism on top of that is to ensure that those variables won't be used by descendants in the class hierarchy, when a given class is intended to be subclassed by a peer.
The the sugar /is/ actually sugary-sweet-yummy, as it's preventing potential faults from people who blindly subclass stuff they haven't read the source code of.
In c++ and likes, I know there's a vtable somewhere, and I get why there must be one for virtual funcs, but I don't want to deal with it directly, it's the compiler job. Same for Python, I know it must prevent fields from getting clobbered when inheriting, but I don't want to be exposed to its mangling or whatever other mechanism it uses.
Python does not prevent anything, it gives the developer a name mangling tool, it's up to the developer whether they want to use it or not. By default, identical names will conflict and you will clobber supertype fields or methods.
ref: https://docs.python.org/2/reference/expressions.html#atom-id...
ref: https://docs.python.org/2/faq/programming.html#i-try-to-use-...
It has nothing whatsoever to do with field visibility.
Now I like python, it has many useful libraries, in fact it is one of the language that has the most libraries for any purpose. I wish, even as a dynamically typed language, it was stricter sometimes though.
> Input function
Yes, in Python 2, input() is a shortcut for eval(raw_input(...)), and documented as such. Obviously that is not a safe way to parse user input, and therefore it has been changed in Python 3. So this has been fixed, but if you don't read the documentation you probably will keep introducing security issues with whatever programming language.
> Assert statement
If you want to effectively protect against a certain condition, raise an exception! Asserts, on the other hand, exist to help debugging (and documenting) conditions that should never occur by proper API usage. Stripping debugging code when optimizing is common practice, not only with Python.
> Reusable integers
First of all, this behavior isn't part of the Python programming language, but an implementation detail, and a feature as it reduces memory footprint. But even when small integers wouldn't be cached, you would still have the same situation when using the is operator on variables holding the same int object. On the other hand, caching all integers could easily cause a notable memory leak, in particular considering that ints in Python 3 (like longs in Python 2) can be as large as memory available. But either way, there is no good reason to check for identify if you want to compare values, anyway.
> Floats comparison
floats in Python use essentially the native "double" type. Hence they have whatever precision, your CPU has for double precision floating point numbers, actually it is specified in IEEE 754. That way floating point numbers are reasonable fast, while as precise as in most other programming languages. However, if that still isn't enough for your use case, Python also comes with the decimal module (for fixed-point decimal numbers) and the fractions module (for infinite precision fractions).
And as for infinity, while one would expect float('infinity') to be larger than any numerical value, the result of comparing a numerical value with a non-numerical type is undefined. However, Python 3 is more strict and raises a TypeError.
> Private attributes
Class-private attributes (those starting with __) exist to avoid conflicts with class-private attributes of other classes in the class hierarchy, or similar accidents. From my experience that is a feature that is rarely needed, even more rarely in combination with getattr()/setattr()/delattr(). But if you need to dynamically lookup class-private attributes you can still do so like hastattr('_classname__attrname'). After all, self.__attrname is just syntactical sugar for self._classname__attrname.
Also note that private attributes aren't meant as a security mechanism, but merely to avoid accidents. That's not specific to Python; in most object-oriented languages it is possible to to access private attributes, one way or another. However, Python tries to be transparent about that fact, by keeping it simple.
> Module injection
Yes, Python looks in a few places for modules to be imported. That mechanism is quite useful for a couple of reasons, but most notably it's necessary to use modules without installing them system-wide. It can only become a security hole if a malicious user has write access to any location in sys.path, but not to the script, importing the modules, itself. I can hardly think about a scenario like that, and even then I'd rather blame the misconfiguration of the server.
> Code execution on import
Yes, just like every other script language, Python modules can execute arbitrary code on import. That is quite expected, necessary, and not limited to Python. Even if module injection is an issue, it doesn't make anything worse, as you you don't necessarily have to run malicious code on module import but could do it with whatever API is being called. But as outlined above, this is a rather theoretical scenario.
> Shell injection via subprocess
Yes, executing untrusted input, is insecure. That is why the functions in Python's subprocess module, by default, expect a sequence of arguments, rather than a string that is parsed by the system's shell. The documentation clearly explains the consequences of using shell=True. So introducing a shell injection vulnerability by accident, in Python, seems less likely than with most other programming languages.
> Temporary files
If anything, Python is as unsecure as the underlying system, and therefore as most other programming languages too. But CWE-377, the issue the author is talking about, isn't particular easy to exploit in a meaningful way, plus it requires the attacker to already have access to the local temporary directory. Moreover, Python's tempfile module encourages the use of high-level APIs that aren't effected.
> Templating engines
The reason jinja2 doesn't escape HTML markup by default is that it is not an HTML template engine, but a general purpose template engine, which is meant to generate any text-based format. Of course, it is highly recommended to turn on autoescaping when generating HTML/XML output. But enforcing autoescaping would break other formats.
However, it had also never occurred to me to make anything I cared about the security of in python. Perhaps this article is aimed at people who are writing system utilities for linux distributions, and are considering Python? Presumably some such utilities are written that way already.
It comes down to doing a proper security analysis before you define the requirements of the software: Specifically what attack vectors you want to defend against. A valid conclusion for some types of software, given the list of "bugs" in the post, would be don't write it in Python. (Indeed, I have done exactly this before writing 200 lines of C instead of 20 lines of Python.)
Do you have some specific types in mind? I know some types of protection are not reachable from python directly and require native modules, but I'm not sure what would cause you to drop Python altogether. I'd be interested to hear some examples.
The problematic code here is Python's `import` mechanism and mutable global references to standard library functions. You can cut out the "buggy" code by writing in another language.
dnf for example is widely used (Fedora's default package manager) and it's written in Python: https://github.com/rpm-software-management/dnf
Also input is truly baffling to me. Such a small mistake that could allow write access to your code.
Then we got stuck with it because backwards compatibility.
Assert is a statement, not an expression, so do not use it as an expression.
One should never compare floats. This is taught in any freshman CS course. The limitation is due to the standard encoding of floats - IEEE 754 - not Python's fault.
Everything else are features of a truly dynamic language, designed for a really quick prototyping. Python3.x got rid of many inconsistencies and caveats of 2.x
Shall we re-read the classic now?
Logically, they should refer to the same entity. It is "natural" - when people are trying to communicate a concept to one another they assume they are referring to the same concept. Not to an instance of it.)
I find the concept of special-casing ints to behave that way to be surprising and inconsistent. If ints act that way, shouldn't strings? And if they (very much unexpectedly) did, why not every other type?
"is" is very useful on its own. "variable is None" is a common and powerful idiom entirely distinct from "variable == None". There are many cases when you want to compare object identity. None of those use cases apply to ints where "==" is always the correct way to compare them, so the fact that "a == b" and "a is b" might occasionally be the same or different doesn't affect anything at all in practice.
`str' can be interned in some situations, though the rules vary across implementations and versions. Most of these things just boil down to unintuitive caching optimizations. Like you mention, it's pretty rare to check the object identity for integers or strings, but if you are doing so, you probably want the real answer.
Aside Python's small integers, True, False, and None, Java has these rules for boxing in the specification [1]:
> If the value p being boxed is an integer literal of type int between -128 and 127 inclusive (§3.10.1), or the boolean literal true or false (§3.10.3), or a character literal between '\u0000' and '\u007f' inclusive (§3.10.4), then let a and b be the results of any two boxing conversions of p. It is always the case that a == b.
> Ideally, boxing a primitive value would always yield an identical reference. In practice, this may not be feasible using existing implementation techniques. The rule above is a pragmatic compromise, requiring that certain common values always be boxed into indistinguishable objects. The implementation may cache these, lazily or eagerly. For other values, the rule disallows any assumptions about the identity of the boxed values on the programmer's part
[1] http://docs.oracle.com/javase/specs/jls/se8/html/jls-5.html#...