Git: The Stupid Password Store | Hacker News Reader

Git: The Stupid Password Store

tylercipriani.com

73 pointsthcipriani53 comments

> With git over ssh, you use your keys.

Why use git over https then?

I also have a hard time understanding why would people use https to clone a repo when ssh is there and so much more convenient…

I'd generally recommend HTTPS, simply because most developers cannot be trusted to manually verify SSH public keys/fingerprints. Most will blindly approve whatever fingerprint they are shown, opening the door to man-in-the-middle attacks.

This isn't just an issue during initial set-up. If the Git host rotates their keys, it will need the client to accept the new keys. If your developers are sloppy and unthinkingly approve the new fingerprint, that means they're vulnerable to a MitM attack at any moment; presumably they'd unthinkingly approve an attacker's fingerprint too.

With HTTPS, you inherit the web's CA model, and it's pretty much secure-by-default.

PSA that Github publishes their fingerprints here: https://docs.github.com/en/authentication/keeping-your-accou...

Also, when you’re asked to verify the fingerprint, you can copy and paste the matching one from that known-good list instead of blindly typing “yes.” That way SSH will do the comparison for you.

If you do this, your usage of SSH with Github is as good as TLS (since the fingerprints are served via TLS).

If your developers are sloppy about security does that not create a lot of other problems too?

Is this a UI issue to some extent? Its easy to type "y"/click yes to accept a changed finger print, but takes effort to type "Verify=false" (if I have got that right) to accept a bad certificate.

> If your developers are sloppy about security does that not create a lot of other problems too?

Yes, but: devs can be sloppy about ssh key security and better (yet still imperfect) at other things. Also, it's worth continually making improvements in some areas even if we can't fix everything. Which is good. We'll never be able to fix everything.

I wish certificate signed host keys were easier to roll out and maintain. That's a marked improvement, even if it has its own flaws.

This seems like it could be solved easily enough. Just add a feature to ssh for rotating keys securely. Each new key issued by the server could be signed by the previous (already trusted) key. If the chain is ever broken and an unsigned key is issued (after initial setup) then ssh could simply bail out and refuse to connect with a scary message warning you to contact the administrators, as it does now every time keys change.

The problem right now is that if you rotate keys frequently your users get desensitized to the scary message and end up following the drill to delete the current key and connect with the new one. With a secure signing chain for rotating keys you avoid this problem.

> Each new key issued by the server could be signed by the previous (already trusted) key

That doesn't work if the host's private key is leaked, or may have been leaked, e.g. [0][1].

The current solution seems ok; we go through another channel to confirm the new key, likely either receiving an email about the key change, or using the web.

[0] https://github.blog/news-insights/company-news/we-updated-ou...

[1] https://bitbucket.org/blog/ssh-host-key-changes

You can set up SSH to use certs and then add the server cert to your known_hosts file instead of the individual host keys. Then when the host keys rotate, all is well as long as the new host keys are signed by the same cert.

Respectfully, it seems that your actual problem is that you work with developers who can't be trusted to manually verify SSH public keys/fingerprints, not that SSH is complicated.

Defaulting to HTTPS just because it is easier for them feels like a no-solution, but I could be generalizing. Maybe you are talking about developers in training or L1 levels.

I really think very few developers bother to check fingerprints.

In 2019 I struggled to confirm the SSH fingerprint when connecting to an Ubuntu VM on Amazon EC2. [0] My local machine was running Windows, and I was using PuTTY as my SSH client. Naturally I turned to ServerFault, the relevant sibling of StackOverflow. I was surprised to see that although a similar question had already been asked there, it wasn't clear how I should proceed.

The answer turned out to be pretty non-obvious. The root of the issue was that PuTTY was using an old fingerprint format, different from the one used by Ubuntu. This meant the fingerprint shown in the EC2 instance system log differed from the one shown by PuTTY.

Which seems more likely?

1. Most PuTTY users insist on checking SSH fingerprints, but are smart enough to figure out the proper solution

2. Few developers use PuTTY in combination with EC2

3. Almost no one bothers to check SSH fingerprints

(I believe the particular issue no longer arises as PuTTY has been updated to use the proper fingerprint format. Also, in EC2 you can now bring up a command-line session in the browser, so you can grab the instance's public key that way. I haven't checked recently to confirm, but I believe you still need to look in the instance's system log to dig out the fingerprint - Amazon don't bother to show it anywhere convenient in the web UI.)

[0] https://serverfault.com/q/996828

The majority of developers do not verify SSH fingerprints. If the majority are not following a process that makes the technology safe, then the technology is flawed.

I don’t understand the pompous hostility here.

Choosing a default that requires less effort to verify the security of is never a bad idea, especially if you know the limitations of the people at place you’re in. It isn’t worth teaching everyone everything about SSH on top of one’s normal set of daily tasks.

Let’s settle into reality for a hot minute: absolutely nobody is getting fired over “what’s the point of a SSH server’s public key”

Hey, my bad if I came off as hostile before. Didn't mean to.

My perspective comes from spending lot of time training and onboarding devs. Devs can level up their skills if you give them a chance. That's why I push for upskilling rather than dumbing things down.

I get where you're coming from with the SSH key stuff. And yeah, it's practical to go with easier options most of the time. You're right that no one's getting fired over SSH keys. But hey, learning this stuff can make the team stronger in the long run.

I understand this is a petty example as Senior developers can be flexible about these things and strong-minded about others.

Simply connect the first time from a known-secure connection. If your laptop is compromised you're screwed anyway.

And store the keys on a yubikey (OpenPGP or more modern Fido2) so they can't be exfiltrated. A huge benefit over a password.

And if you really want, SSH certificates are a thing too.

> Simply connect the first time from a known-secure connection.

No, that's not a solution. The point is to treat the Internet as an untrusted network.

Ah yes but I mean the first time I don't connect to the internet or even VPN. I just connect to my git server on the local LAN.

But good point, as GitHub Cloud is not local. In that case I would verify the key indeed.

I don't think I'd use github for stuff that needs to stay private though. At work we have a self-hosted internal GitHub enterprise server for all closed source projects.

HTTP has many advantages (and SSH has disadvantages) as detailed in https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protoco....

Briefly, HTTP is more flexible. For authentication, you could use passwords or OAuth.

From your link, literally, both HTTPS and SSH have pros and cons! You make it look like SSH only has cons and HTTPS only has pros; that’s borderline dishonest.

In corporate networks having direct access to public hosts (or even internal ones) with anything other than https is a challenge.

You can't clone a public repo over SSH without having authenticated keys set up. Somewhat annoying on servers that you forgot/don't want to use agent forwarding on.

You can set up anonymous SSH access. It is just that for some reason the big "hubs" do not do so. See for example the instructions that come with gotsh(1):

https://www.gameoftrees.org/gotsh.1.html

Obviously not a good idea for repositories that you want to keep private. But it is odd to me how rare it is for public repositories.

Why ? Openbar readonly ssh servers exists.

Also, cloning private repo via http is a pain, when MFA is involved

Also, an existing repo that has been cloned over https can be converted to use ssh:

    ~ $ git remote -v
    origin https://github.com/mycompany/myproject.git (fetch)
    origin https://github.com/mycompany/myproject.git (push)
    ~ $ git remote remove origin
    ~ $ git remote add origin git@github.com:mycompany/myproject.git
    ~ $ git remote -v
    origin git@github.com:mycompany/myproject.git (fetch)
    origin git@github.com:mycompany/myproject.git (push)

    git remote set-url origin git@github.com:...

Or using URL replacements in git config.

Or using .ssh/config since entries in there act like hostnames:

    # ~/.ssh/config
    Host github_ssh
        HostName github.com
        User git
        IdentityFile ~/.ssh/my_github_key

    > git remote add github github_ssh:Username/RepoName.git

This has worked reliably for me for quite a while now.

However, this doesn't work well with git submodules, because then everyone cloning the repo must recreate this config. (Just something to be aware of.)

It works fine if coupled with URL replacement.

The custom hostname will be replaced transparantly for you and won't need to make it into gitmodules.

    # ~/.config/git/config
    # Only for single user/org
    [url "git@github_ssh:foo"]
      insteadOf = "git@github.com:foo"

    # all the things
    [url "git@github_ssh"]
      insteadOf = "git@github.com"

Similar pattern if you want to force-use ssh instead of https or vice-versa.

Plenty of places don't support Git LFS over SSH

I also used to think this, then I realised https has far better performance, particularly for high latency connections.

A big repo clone from Github to here in Australia takes about half the time over https when compared with ssh. I'm not sure if github has region-located repos.

It's possible that's an issue with GitHub, rather than anything inherent to the protocols.

Somewhat related: SourceHut blog post on them optimising Git operations over SSH: https://sourcehut.org/blog/2019-11-22-what-happens-on-git-pu...

does it? I've used both quite a bit and ssh certainly "feels" faster to me (with multiplexing)

My network blocks outbound ssh, so I use https.

I had the same issue, but you can clone using SSH over HTTPS with GitHub. Like this:

   git clone ssh://git@ssh.github.com:443/rtyley/small-test-repo.git

instead of:

   git clone git@github.com:rtyley/small-test-repo.git

Is that using HTTPS? It looks like regular ssh running on port 443.

Also, if you're on a corporate network and start pushing ssh through 443, a competent security team should check in on you to make sure things are okay.

Yes, if you have audits, most probably they will look for protocols traffic rather than just ports open

The protocol is SSH, but the port is 443. Yes, it's not HTTPS, it's just SSH on a different port than 22.

If it’s your home ISP doing this, you just need to call and ask for them to remove the restriction. Obviously not likely to work if this is a corporate network.

Is that by design or accident?

I couldn't imagine living without ssh.

It’s by design. There was some cyber practice doc years ago calling out how outbound ssh is a security risk because it can’t be monitored. And they’re afraid people will steal data and send it out without network security knowing.

I hated the policy because I loved outbound ssh tunnels to get around blocked websites (ie, no gmail). But I kind of understand it because dumb people will do dumb things and they can’t differentiate people who will use appropriate judgement.

But I’ve found that eventually every large org I’ve worked for ends up blocking ssh.

hunter2 still gives me a smile

AWS CodeCommit (just been deprecated btw), has a credential helper that allows you to use IAM role/policies instead of SSH keys.

tar -zxvf git-2.46.0.tar.gz cd git-2.46.0 make configure ./configure --prefix=/usr/local make sudo make install

rm -rf /usr/local/libexec/git-core/git-credential-cache*

If a system relies on "passwords" (or any similar shared secret scheme) it clearly doesn't really care about security. Yes that includes your bank.

If it doesn't require shared secrets, this isn't scary, the git mechanism works just fine.

> If a system relies on "passwords" (or any similar shared secret scheme) it clearly doesn't really care about security.

that's ... one of the dumbest thing I've ever read

Security is easy if you don't care about usability.

What's funniest without any doubt is that passwords and similar shared secret systems have worse usability and worse security.

They're a classic "Worse is better" like C where their principle advantage is simplicity of implementation. Sure, they're terrible, but look easy it was!

I know right? Stupid facts, going around disagreeing with your intuitions.

How can shared secrets be a bad idea? I mean, it was considered pretty good a thousand years ago, can technology really have improved since then? [Hint: yes, drastically]

Reply on news.ycombinator.com