Comment by bsder | Hacker News Reader

Comment by bsder | Hacker News Reader

Parent Full thread

Python cannot be atomized effectively, and the issue is political.

The problem is that I cannot count on being able to install new software in many environments.

If I fight the battle to get centralized IT to install Python, I now have a guaranteed set of standard libraries as well. I'm never going to get permission to install anything other than default. Ever.

Consequently, the standard libraries need to be very complete and very useful.

And, while people seem to love the Rust approach to libraries, I'm not necessarily a fan. Far too many times I have pulled a library that is "obviously" something that a language should consider to be "standard library" and gotten bitten because it was broken. Only VERY core libraries in Rust are guaranteed to work across multiple architectures and OS's.

I think Rust is probably doing the right thing for Rust as "batteries included" is NOT one of its tenets. However, that doesn't make it right for everybody else.

> If I fight the battle to get centralized IT to install Python, I now have a guaranteed set of standard libraries as well. I'm never going to get permission to install anything other than default. Ever.

Can you explain this more? What kind of place do you work? I've had some experience with large, bureaucratic companies, but nothing ever so far as "you can't install any other libraries."

Maybe just me but sometimes after initial install l, I later find a weird non-standard library that I need to do something.

At which point you have to scale the IT wall all over again if you work at a Fortune 500 company.

> At which point you have to scale the IT wall all over again if you work at a Fortune 500 company.

I work at a Fortune 500 company, and the only wall I have to scale when I want to use a library that nobody at our company has ever used before, is to get someone to check and approve the license (typically takes 1-2 days), and import it into our code repos.

I mean I see your point, but not everywhere is as bad as you make it seem.

At least in some places, local install permission may be available, but cross company install permission is much more difficult to get.

As in, it is very difficult to get software installed in general images or on multiuser servers.

There are obviously good reasons for it to be conservative about this.

As far as I understand this. It’s something like “You can install whatever libraries you want, but we’ll only ever install default python on user PC’s”

>What kind of place do you work?

Not OP but same. I'm currently debating with myself whether I should attempt to install PUTTY. Given that port 22 is blocked and it's not needed for my core role it'll be dicey if I get challenged.

Pulling executable code off some repo...no way that is ever officially passing muster. People might do it anyway, but on a personal risk basis.

>Can you explain this more?

Place that are heavy on confidential financial info basically. Practically everything I touch is confidential client data. So employer is naturally jumpy about what's on my laptop software wise.

Ironically the above comes full circle...need putty to get onto a VM in cloud where there are no restrictions and crucially no client data. Nobody cares what I do there - hell they'll even pay for it thanks for MSDN enterprise

I'm glad Windows 10 has OpenSSH now: > Microsoft Windows [Version 10.0.17763.437] > (c) 2018 Microsoft Corporation. All rights reserved. > > ssh -V > OpenSSH_for_Windows_7.7p1, LibreSSL 2.6.5

Nice! Too bad our desktops are still on Windows 7 at the office.

I've found the built-in Windows 10 ssh client has trouble with tunneling, so I still use the ssh client included with git.

netsh interface portproxy add v4tov4 listenport=8001 connectport=80 connectaddress=1.1.1.1

this isn't useful for tunneling through a remote machine (which is what ssh does)

If you can figure out how to reproduce the issue, I'm sure the team would accept a bug report at https://github.com/PowerShell/Win32-OpenSSH

I've been checking but I can't add the module. Either it's slow to get to Enterprise version or it's blocked. Not sure.

Found a work-around though - Google cloud shell being *nix works fine for SSHing about the place. Gets me around the port fw too

Where I work, there is currently a push to get python on the computers that manage physical equipment operation. These computers are not allowed to connect to the internet, and have extremely limited connectivity to the rest of the business network. Installing anything new on them requires risk assessments like you wouldn't believe, since the consequences of malicious code could easily hit 10s of millions of dollars and a nonzero number of lives.

Maybe a stupid question but can you ship code on the machine? If you can, what is stopping you from including the source of the library that you're trying to 'install'?

Many do, but you can get fired for it.

If your risk assessment says that the exact same tkinter outside of Python stdlib is riskier than in Python stdlib, maybe your risk evaluation process needs reevaluation.

I hear this sentiment frequently. Come on, one software engineer cannot steer the huge ship that is BigCo Risk Assessment. Well, they couldn't do that and the original task.

It might me more helpful to think of these types of external factors as fixed points that cannot be moved and just engineer around them.

You'll burn out if you try to boil the ocean on every business process that doesn't seem "logical" from your cursory examination.

On one hand, this is true. On the other hand, this is being put forth as a reason to not make a change in the entire Python ecosystem, and it's not really Python's job to bend over backwards for shops that have bad risk assessment either.

Yes but it won't get it. And at the end if the day, people need to be able to get work done.

The corporate world is full of stupid things that will never not change, or take years to change.

Where I work the solution was to use a proxy to pypi. Basically an internal pip repo (and docker, npm, maven, everything else...). All internal apps go through the internal repository that creates a local version of the package from pypi. That gives the security / compliance folks a way to block packages with security issues, etc. and at the same time provide the developers flexibility to get most of what is needed.

In a large company this gives the compliance folks a central place to blacklist packages - along with a trail of what systems have downloaded the package to target for upgrades.

Many technical solutiins exist, but the problem is political or organisational.

Agree. At this point it was more a case of executives saying they wanted internal dev teams to use and contribute to open source and supporting orgs to come up with solutions on how that can be possible with a 0-touch approach. That’s what tipped the balance.

As long as you cannot even prove that due to a lacking python code signing infrastructure for packages (wheels can do it, but it is far from wide spread).

And setup.py is a trainwreck, e.g. some packages compile download and compile huge dependencies (e.g. a full Apache httpd...), the default compiler flags may lack all the mandatory security flags (e.g. for using ASLR on python 2.x), or ship their own copy of openssl statically and break your FIPS-140 certification that way...

And since setup.py is a Python file, you can't express build time dependencies properly. Pyproject.toml let's you do that, but it's new, nobody knows about it, and older pip clients don't support it.

I disagree. tcl/tk is written in C and C can be compiled in very very badly indeed (from a security perspective).

Not who you asked, but I work for a large international company, a big 4 professional services firm. I wanted to install anaconda and Jupyter on my machine (data science is not a part of my 'official' job description, but I wanted to see how much of my data exploration workflow I could speed up or automate). I had to go up three separate hierarchy ladders to get sign off. First, my own team, then IT, then our risk and quality team (thanks to our audit practice and a few historical issues with whistleblowers and leaks, the risk guys are pretty much the final arbiter of... Everything)

After about 9 weeks of emails, meetings, and pitches, I finally got Anaconda up and running. A week later, I tried to upgrade the 3rd party packages.. No dice. Blocked by the corporate VPN. I'd need the sign off every time I wanted to `pip upgrade` anything

Needless to say, I do not bother anymore.

Hierarchies and the systems or checks that they serve at places like this exist only to keep some people employed. That has got to suck! For anyone who has a rogue or novel idea will get shot down because it’s too much of a burden in terms of overhead to get any decision made.

I can't imagine myself working in a place like this..I understand there should be a level of checks, however this is just crazy...

I also work at a big 4, and perhaps the same one as you, and I assure you, there is a way. There is always a procedure or some way you can follow. You just need to know how to look it up.

We have our own development team, our own servers, our own freedom to deliver to clients fast without the hassle of the main corporation. How? We talked to the right persons.

Big 4?

https://en.m.wikipedia.org/wiki/Big_Four_accounting_firms

I don't think your company deserves you.

I work for a major defense contractor, and while we have vanilla python we are categorically not allowed to download software off the internet and install it without authorization, even on the unclassified internet-connected corporate network.

Theoretically there's a process for requesting new software and getting it approved, but actually pushing it through requires getting one of my program architects to care enough to file the request (As a mere level 2 engineer all I can do is write it, can't submit), then potentially weeks of followup, for 1 specific version of 1 specific package.

In the case of python packages, perl and the perl packages we need are already approved because a few senior devs got together and pushed them through 10 years ago (was before my time, but I understand it was with quite a bit of arm twisting). It's more time-efficient to just code perl than to fight for python.

It's one of the many reasons I intend to get myself another job for Christmas. :)

As for why the system exists: Cost cutting, in the sense of "the less we invest in infrastructure the more we can divert to sexy hardware for the cameras and shareholder dividends. So long as it's theoretically possible for you to do your work, we don't care how many hoops you have to jump through to do it. And our competition is even worse than us, so we don't have to worry about anyone undercutting."

As a result all our infrastructure is centralized. Programs have to jockey with each other for everything from virtual servers to physical workstations and monitors. Hell the only reason my program has our primary test server is because one of our architects literally overheard a hallway conversation about a program that was spinning down and getting rid of some servers, so he jumped on it.

> Theoretically there's a process for requesting new software and getting it approved, but actually pushing it through requires getting one of my program architects to care enough to file the request

I worked for a much smaller government contractor, but before I left they were moving to a system where you needed approval from the customer in order to get new packages. (For those who don't work in this field, that means you are actually making a request to the contracting representative from the particular government agency for each package you want.) So it wasn't just in-house bureaucracy in the way of progress, and I generally just went without or wrote my own instead of trying to deal with it.

Last June I opened a ticket for our procurement department to have an open source license reviewed. It is still open. Our purchasing process is similar to [0].

[0] https://training.kalzumeus.com/newsletters/archive/enterpris...

If you must work with one hand tied behind your back you are fucked, and your company is even more fucked. Your priority should be supporting your managers who want to get things done in the struggle against centralized IT managers who want to repress aspirations to work.

I'm afraid the standard library has to be aligned with the needs of more normal users who, as already discussed, want to allow libraries to have their own release cycles and to be more "opinionated" and specialized than the standard library would permit.

> the needs of more normal users

I'm afraid users dealing with that sort of bureaucracy are much more normal than you think, if not the norm. They're just usually not the types of folks that are hanging around HN, or they're at least less vocal.

Putting code into the standard library doesn't magically create developer resources to maintain it. Indeed, Amber Brown is saying that many libraries in the Python standard library aren't properly maintained. So it's not clear that standard library policy is relevant to the Rust issues you have.

> If I fight the battle to get centralized IT to install Python, I now have a guaranteed set of standard libraries as well. I'm never going to get permission to install anything other than default. Ever.

You could install packages with `pip install --user` to have them install under your home directory.

> > If I fight the battle to get centralized IT to install Python, I now have a guaranteed set of standard libraries as well. I'm never going to get permission to install anything other than default. Ever.

> You could install packages with `pip install --user` to have them install under your home directory.

You might find that the issue isn't necessarily always technical (ie permissions), but policy. A lot of places don't let you arbitrarily download software off the internet onto a system. Pp's point is that having a full feature stdlib let's you only need to crank the policy/approval process once, rather than once for every dependent lib.

> The problem is that I cannot count on being able to install new software in many environments.

The approach Ruby is taking with gemification and default and bundled gems for standard libraries is equivalent to the traditional standard library if you can't install updates, but superior in other cases.

As I understand, the Ruby model -- that is suggested for Python above -- is to include the libraries by default but to keep them as separate pypi packages.

In this case, even If you can't upgrade, you still get the same libraries but perhaps older versions.

Reply on news.ycombinator.com