The Wrong Kind of Paranoia
prog21.dadgum.com
prog21.dadgum.com
If I mark a method as internal, that means I intend it for reuse within the library but don't expect it to be used by any external caller, which means another developer can come along and make changes to it without needing to worry about anything outside the library (short of those in C# using [InternalsVisibleTo] or similar, which hopefully is restricted heavily to obvious test projects).
By decorating my code appropriately, I don't need to write comments most of the time, which means I don't run the risk or filling my code with out-of-date or poorly understood annotations that become useless almost as soon as I finish writing them.
Even in Python, a very dynamic language, there is a convention that methods that start with _ are private. And I often also separate them into a different (bottom) section with a big header saying this is the private block of methods. So in that case there isn't a compiler rule but rather a common convention (which is kind of a step above comments if you wish).
__mydef -> _myClass__mydef
> const
I don't have to worry as much about iterator invalidation with containers, because I've documented that I'm not going to change it. Bonus points: I know the documentation isn't lying like it usually is. (Caveat: Someone const_cast<>ed it away, and my trust is forever crushed ;_;)
I don't have to scan for side effects to figure out if this const variable is an alias for "the initial size of the container" or "the number of elements remaining to process, which just happens to start as the initial size of the container".
It's clearer if the vector normalization function is "mutate in place" or "return a copy". I've seen enough code that does both that I can't necessarily rely on eyeballing the return type.
I can slap it onto existing, likely side-effecting code, to help make sure I don't miss anything when refactoring to an immutable style, because e.g. I'm preparing for multithreading.
That it helps catch accidental mutation is merely a happy side benefit.
> protected/internal
These are worth it just to unclutter my intellisense, which is how I read most of my documentation on some projects. Because it contains all the documentation, or because the API is straightforward enough that it contains all the necessary documentation.
> static
I've had this both help and harm. I'm typically using C++'s anonymous namespaces instead. I have collisions frequently - localized log methods, error tables, local module allocators, etc.
This is less about "I want to prohibit external calls" and more about "This is throwaway local code that I don't want to painstakingly disambiguate from every other possible chunk of throwaway local code." - I see frequent enough collisions with local log methods (which annotate before calling global log functions), error tables, shorthand aliases to specific allocators, etc.
And now to quote from the article instead:
>> What all of these fine-grained controls have done is to put the focus on software engineering in the small.
I'll certainly agree that fetishizing these micro tools to the detriment of larger system issues is bad. That's more along the lines of "insufficiently wide paranoia" than "the wrong kind" though.
>> There's an architecture used in video games for a long time now where rendering and other engine-level functions are decoupled from the game logic, and the two communicate via a local socket.
You can overcouple via RPC over a socket just fine. Like module level protections, it's a mere speedbump against making a bad decision. There's no panacea.
If you call only my public methods, you can generally expect:
- if my public methods don't work properly, it's a bug
- and I, as the library author, promise to care
- if the internals of my library change, I'll keep my public
functions working if possible.
Without these annotations it would be much harder to effectively communicate and enforce the parameters within which the library is "promised" to work.
Though it won't protect you from malicious intent (though this is still the case with C / etc), and it's caught at runtime as opposed to at compile time.
It often does feel like junior developers are malicious. While I commend their ingenuity in solving the problem, code review can be a real schlep.
Const also allows the compiler to optimize better with the knowledge that the called function won't perform any writes to the object state.
Access modifiers give you a way to separate out functions that can be called externally vs those that shouldn't. The API doc generator won't know which is which unless you mark these correctly. You use access modifiers to show intended use and keep things clean, not to lock someone out. Same goes for internal classes.
But I don't think having private methods or marking variables const is about paranoia, it's about communicating intent to the people who come behind you and using the compiler to enforce that intent. Every piece of extra context you can give to someone reading your code helps them understand why it's there. Identifiers like const and private are almost like nonverbal communication for code.
TLDR sorry James, we have tried the elegance of extremely simple and open language/runtime architectures, and that was always an abject failure.
Just because it was tried in the past, doesn't mean it wasn't the right direction.
> Attempts to discipline these languages came too late,
So, as an industry, we were learning and now we know some basic principles. Discipline does a great deal for Erlang. Ironically the game sockets he describes are basically how Erlang functions at scale. To say the practice is always an abject failure is burying your head in the sand.
As someone who learned to code with javascript, ruby, and python, I'm not sure I'll ever really appreciate these "nanny languages".
I am a professional Objective-C developer and have been for several years. Type safety is not an absurd length. Eliminating an entire class of errors from your program by having the compiler infer and enforce types is not ridiculous. Using a type-safe language is a very good idea.
We are human. We have stupid unchecked nil object errors come up in our code bases all the time. Swift will ensure that does not happen again. That's like the least part of what I am looking forward to.
If you are having trouble writing a program that compiles with strong typing, I don't know what to tell you. Using types is nothing more than stating what you expect the shape of the data to be in and having the compiler make sure that is so.
I think types probably work well in situations where you don't know what kind of code you're going to have to deal with in the future. It depends on the type of organization you're in, not the type of problem you're trying to solve.
I think any respectable programmer should try to avoid having to have people hook into his code at any level other than the level he defines. To interface at the level of data, not client code. If you are needing typing to solve your own inadequacies as a programmer, you should become a better programmer rather than expect your language to do that for you.
With a dynamic language, you can get all you could have wanted from a type system without having to infect your whole codebase with it. Most of the time, you just don't need it.
However, not all type systems are equal. Haskell (for example) almost never requires explicit type annotations. It has a type inference system that is sometimes frighteningly good. You can express a huge amount of logic through the type system and enforce very non-trivial constraints.
I've been writing Ruby for 7 years, and I've loved every moment of it. It's a wonderful language. That said, I usually have to spend quite a bit of time getting my code to work correctly. In Haskell, by the time I get to the point where the type checker approves of my code, it usually works as I intended the very first time I run it. It's a wonderful feeling.
Knowing at any time I can take the class hierarchy I just built, turn each class into an instance of an object, and store those objects in a database, with a 10 minutes and a fancy bit of code, is much much more useful than having a babysitter.
But dynamic typing is terrible for large code bases. The only place I allow it is at the very edges of our system, where we are transforming the proper types of the internal system into POD-objects to be serialized and returned to the JS/HTML front-end. I'm not willing to give up the compile-time safety of breaking the build if someone inadvertently assigns a string to a number, or an object to a primitive in code that they check in, in place of unit testing that, odds are, will be incomplete or otherwise broken and won't catch the problems that static typing catches.
Yes, you can get into a lot of trouble very quickly as your codebase grows if you don't have a heavy security blanket. But the more Ruby I write, the more typing looks like Linus from Charlie Brown's blanket.
Because you don't need a monolithic code base any more. You can break it up into smaller pieces that interact with each other using POD objects.
When I start to have type problems in Ruby, I start looking around for a domain concept that I need to extract into a gem. The codebase never grows to a point to where it becomes a serious problem.
Is he talking about multiplayer or has anyone ever actually seen this?
Steam's Source games, UT do this, if you open up the console and scroll up you can see the local server initializing.
Not sure about Supreme Commander but it wouldn't surprise me if it did.
Also, Minecraft uses something vaguely similar, with a separate client and server, with the client and server communicating locally via shared memory (effectively a local socket, but not bothering to bounce through the OS). Though Minecraft's an odd case - they used to not do it this way, and switched to it, and in the process broke a lot of things.
James Lewis and Martin Fowler coined the per "Microservices" for this emerging pattern:
When you are working to refactor software written by others, things like "private methods" are a gift. If you are trying to refactor some piece of code, you only have to make sure that the callers within the said class are modified to guarantee that the codebase is not broken.
I don't think a lot of people understand that maintainers in the real world do not have the time to read and understand every line of your code. And the toughest part of doing software maintenance is figuring out how much you can safely ignore. I feel qualifiers like "private" were designed to help with this problem.
I also severely dislike the architecture proposed by the author with loosely coupled services talking over a socket. This style of code is only maintainable if you understand the entire system inside and out. For example, lets say the maintainer receives a ticket that says 'Report X has wrong data'. She will start investigation with the question : 'Why does it have wrong data ?'. She will walk back up the call tree looking for why and eventually learn that the data coming off the socket is wrong and that is where the trail ends (unless there is a document describing who is responsible for putting said data there).
I have faced this issue in real life. I can understand when this style of decoupling is necessary to improve modularity, but it does not have a positive maintenance impact.
Autocomplete is a thing. Even if there is a big comment explicitly saying not to use a function, if it doesn't show up in the autocomplete window someone will inevitable use the function anyway (and sometimes even if it does).
Why?
Separation of responsibility/decoupling on the service level are good principles to begin with, but in an OOP paradigm where you are defining/exposing classes and methods, these keywords are really useful for grouping functionality when you adhere to "contract-driven development."
When used correctly they help you abstract the interface for your class (ie, the public implementation-agnostic methods that provide interoperability between the service and the program as a whole) and separate it from methods only designed for internal use (within the class itself).
Often times those abstractions are essential (DRY principles) and the class is the proper place to encapsulate them, however you wish to clearly designate that "this method is self-modifying, limited in scope, and does not interact with or is required by any other class in any meaningful way," and for that it's quite useful.
When I started estimating the complexity of my code - just using rule of thumb and line counts - I found that most uses of classes were unjustified. The "right size" of a class was quite large, especially so in the top level of an application.
First I should note that I follow a few basic principles/sets of principles when programming loosely-typed imperative languages.
1) Single-responsibility for classes and methods, with each method coming in around <= 20 lines. In total the majority of my classes in enterprise level applications (supporting 100's of millions of users & responding to internal events with meta/statistical analysis) rarely grow beyond 200 lines per class. Usually whenever I hit the 2-500 line range I'll find that many methods can be logically abstracted to a few core traits in order to take advantage of multiple-inheritances without overcomplicating the central registrar or DI patterns.
2) I build an interface before building a class--the interface defines what methods will be available to the application/world context (thinking as per an API interface), as well as what type(s) should be received and what type(s) should be returned. By type hinting/checking at this stage and ensuring conformance to an interface, you can swap out implementations easily later, as well as have a general map or "spec" before you really start hammering the nails--this helps in staying organize and weeding out bad architecture decisions early.
3) I keep these public methods simple, so that I can clearly detect failure points and debug based on input/return types for the 'service' as a whole, and then I use protected methods internally similarly to data pipes in functional programming; each method is clearly named, has a specific transformation it applies, and acts on a series of (n) objects by mapping transformations vs iterative loops which precludes un-terminated conditionals and other type-juggling weirdness.
This all results in software that's very concise (IMHO), runs well, and is quite simple to test. I can inherit any class from a testclass, in order to test the protected data pipes with any sample streams. I can verify the I/O types for all interfaced methods of the class (via functional tests), and tie it all together neatly in knowing that I can pull up any file and clearly differentiate between what formats/returns/transports data, and what mutates it and/or the "state."
The process is by no means perfect and I continue to learn in my pursuits as do we all, however this structure has worked well for me consistently on the types of large projects where others have failed, and in that context I feel it's worth expounding.
Man, we really need a module system.
I think the problem is not that `private` exists, but that it is taught/sold to programmers as a silver bullet for making software architectures better. People think "Decoupled systems are good. `private` hides my variables from the outside world. Therefore, if I use `private` variables, my system will be decoupled." This is false. A program can use `private` extensively and still be tightly coupled.
So, I agree with the author that `private` encourages myopic engineering, but I think education is a better way to fix the problem than removing the features.
And you might be getting one: https://isocpp.org/files/papers/n4214.pdf
It's under consideration for C++17, as I recall.
EDIT: Also, CLANG allows you to use modules right now, but IIRC those are CLANG specific extensions and the final module spec. may or may not be compatible with them.
What it does is free you from a big overhead, which is keeping track of the access levels for all of your instance variables. Moreover, documenting enforcements is a very shaky way to go about it. It requires military discipline. I've seen time and time again, documentation which didn't get changed with the code.
As other comments say, its a documentation to future self and others.