Show HN: A fix for the .bash_profile .bashrc .profile madness | Hacker News Reader

Show HN: A fix for the .bash_profile .bashrc .profile madness

gist.github.com

173 pointsjwecker42 comments

Ignoring .bash_profile vs .bash_login vs .profile, the bash man page essentially says:

  login shell                     .profile
  interactive non-login shell     .bashrc
  run by the remote shell daemon  .bashrc

I wasn't sure exactly what counted as a "login" shell so I got each of my .profile and .bashrc printing into a logfile. These are my observations on OS X:

  log into OS X user account      (nothing)
  new terminal                    .profile
  run "bash" from that terminal   .bashrc
  run a shell script              (nothing)
  ssh hostname                    .profile
  ssh hostname somecommand        .bashrc

The behaviour of ssh seems (to me, at least) to contradict the bash man page. Anyway this was all so complicated that I'll never remember it, so what I ended up doing was:

* .profile does nothing other than source .bashrc

* In .bashrc anything that should be specific to interactive shells goes inside:

  if [ -n "${-//[^i]}" ]; then
    ...
  fi

'login' is when called with --login flag (which login(1) does)

'interactive' is when called with -i flag (which sshd does when reached by ssh with no command passed, i.e if you do 'ssh ls' it should not call bashrc, but should call bash_profile)

The only thing needed is, in bash_profile:

    [[ $- == *i* ]] && source ~/.bashrc

because bash does not load bashrc when it is both an interactive and a login shell.

There is no madness, everything is explained in TFMs (bash and zshall, which I coincidentally went through just yesterday)

FWIW here are my dotfiles, which are designed to relieve me of the true madness of the startup scripts: the thousand-line hairy mess (bonus included: mutualization between bash and zsh configs).

[0] https://github.com/lloeki/dotfiles

  > 'interactive' is when called with -i flag (which sshd does when reached by
  > ssh with no command passed, i.e if you do 'ssh ls' it should not call
  > bashrc, but should call bash_profile)

Do you mean that an interactive shell should load .bash_profile? Because the bash man page says: "When an interactive shell that is not a login shell is started, bash reads and executes commands from ~/.bashrc".

Re. ssh, the bash man page says (at least on my system, with bash 4.2.37): "Bash attempts to determine when it is being run with its standard input connected to a network connection, as when executed by the remote shell daemon, usually rshd, or the secure shell daemon sshd. If bash determines it is being run in this fashion, it reads and executes commands from ~/.bashrc".

That last quote from the bash man page contradict my own experiments. Logging into my OS X system with ssh loads .profile, not .bashrc. The same is true logging into my Fedora 17 system with ssh. Perhaps sshd uses --login instead of -i (or, on my systems, happens to be configured in that way).

Running a command via ssh ("ssh hostname somecommand") loads nothing when the remote system is OS X, and loads .bashrc when the remote system is Fedora 17.

All in all, I will respectfully disagree that "there is no madness". Normally I completely relate to the RTFM sentiment, and I have spent plenty of time in the bash manual myself.

I remain unconvinced that there is a better solution than putting everything into .bashrc (with the $- guard around stuff specific to interactive shells) and doing nothing in .profile other than unconditionally sourcing .bashrc.

  >  [[ $- == *i* ]]

Thanks for the sane test syntax: It's much clearer than my ugly mess of almost every symbol character I could find. :-)

> Do you mean that an interactive shell should load .bash_profile?

No, ssh logs you in, so it is a login shell (and thus calls {bash_,z}profile), but if you call 'ssh ls', it is not an interactive shell since you just ask ssh to launch the 'ls' command remotely.

> Logging into my OS X system with ssh loads .profile

sshd starts login(1) (which shows the 'Last login:' line), which calls your shell as login

Ah, I understand now: The sentence I quoted from the bash man page refers to ssh only when run with a command ("ssh hostname command"). Thanks!

I've always thought of it as

login/profile file(s)

- Run on a "new" shell when there's not any environment present. It should contain things that are "carried over" from one process to child process (ie, exported variables). May also contain things that are run "only once" from the user's point of view (check mail, yadda yadda)

- You don't want these commands run for every subprocess. Imagine that bash sources /etc/profile on each "new" shell to set PATH. You append to PATH in your user profile to add some additional directories. If the profile was sourced for each child process, your path would grow and grow, containing multiple copies of the same directories.

rc file(s)

- Run for each subshell (with caveats[1]) so that you can define things that aren't carried over to subprocesses (define aliases, functions, etc)

[1] caveats being things like remote shell command execution. I honestly don't know why it's not sourced then, but my guess would be that, if you're running a remote command, you're expected to not need things like aliases and functions defined.

Thinking about it all this way, the way the files works just seems... natural to me.

Alternatively, just ensure sourcing .bash_profile is idempotent[1], then .bashrc can simply be

    if [ -n "$PS1" -a -r "$HOME/.bash_profile" ]; then
        . "$HOME/.bash_profile"
    fi

because, on all platforms, PS1 is non-empty on startup iff bash is interactive.

[1] E.g., assuming PATH starts out non-empty, this

    export PATH_DEFAULT PATH="<stuff>:${PATH_DEFAULT:=$PATH}:<other stuff>"

should work for PATH in bash much as

    if [ -z "$PATH_DEFAULT" ]; then
        export PATH_DEFAULT="$PATH"
    fi
    export PATH="<stuff>:$PATH_DEFAULT:<other stuff>"

works in most any Bourne-derived shell. Having the _DEFAULT variables around comes in handy for clean build environments, as well (e.g., I build Emacs.app once and distribute it to several Macs, so I don't want configure to pull in random dependencies that just happen to be installed in MacPorts on my build machine).

.profile is used by other shells, so if you ever used another shell it could be a problem to source .bashrc (unless you write .bashrc in portable way)

Why did someone even decide that the program (the shell executable) needed to behave differently depending on "login" vs. "interactive"? If I'm executing a remote command (e.g. via ssh) I expect it to be working in the same environment that I'd be working in interactively. Maybe I need some environment variable to tell my script what "mode" we're in so that it can make some decision, but the shell not providing the same environment (aside from interactive vs. non-interactive behavior) bugs the hell out of me.

Historically, there were a collection of commands that ran on login (e.g., checking for mail, giving a security report, etc.) that you wouldn't want popping up when you were just firing up additional sessions with su or the like. I'm not saying it's perfect, but there is a logic to it.

Why did someone even decide that the program (the shell executable) needed to behave differently depending on "login" vs. "interactive"?

Just try this little experiment: disable the bash-completions package, and see how much more quickly you can open a new shell. While it might seem miniscule on your multicore multigigabyte desktop machine, on a NAS with limited resources, it can make a significant difference. And when you are scripting something (say, a nightly rsync backup), do you really need to have tab completion for git? This also applies to things you want to run once (on login) versus every time you open a new shell.

Just because you can't think of a use for something doesn't mean no one else can. Usually there is a reason for a feature, and the nice part is that nine times out of ten, when I'm looking for a way to do something I hadn't thought of before, it's already there in UNIX/Linux.

A login shell parses .login (on login), and .logout (on logout), where an interactive shell does not, IIRC. This allows you to things like display certain information to the screen on login that you might not care about on a non-login interactive shell. E.g. running 'fortune' when you log in.

I'm replying to the answers to my post here.

Let me put it another way: don't try to outthink the users' intentions- leave it to me and my scripts to decide what to run and when. My script should make the decision whether it's appropriate to load certain things. Passing --interactive, --login, --remote, --wtfe to the shell should cause nothing more than a flag to be set for my script to query. Then my script can make the decision what to load and when.

I do understand the original intent. I believe it was the wrong was to go about it.

I forked this file into a proper repo [1] and made some adjustments. Now the script does not spawn external processes any longer, making the shell lighter and faster to came up.

Enjoy. Patches are welcome.

[1] https://github.com/gioele/bashrc_dispatch

Some time ago I had to deal with Bash initialisation, so I wrote this:

http://news.ycombinator.com/item?id=4369934

This is great! I made a little tweak where you can change the locations of the scripts: https://gist.github.com/3323840

I set PREFIX to ~/Dropbox/rcfiles/, and the scripts will follow wherever I go :)

Good call. Merged into the repo https://github.com/josephwecker/bashrc_dispatch . Thanks!

The other thing not being mentioned is that sometimes logging into GNOME will source .profile or .bash_profile (I forget which). This can be used to make sure that environment variables are set at the top-level (to be inherited by all sub-processes of the GNOME environment). Mostly this is useful for things like $PATH (which programs like gmrun make use of).

Yes, this is standard and useful behaviour. When I started using OS X, it really confused me that 1) it didn't source .profile on login 2) it sourced .profile (instead of .bashrc) whenever I opened a new terminal.

Yeah, many terminal emulators (like Terminal.app) are configured to run login shells.

bash is the problem, the solution to this madness is to not use bash.

Don't other shells have similar issues? ZSH sources .zshenv, .zprofile, .zshrc and .zlogin (and corresponding /etc files) depending on the exact situation.

Only difference is that it does not source .profile unless it's in compatibility mode (invoked as `sh`)

The other difference is that zsh has a .zshenv (and /etc/zshenv), while bash only has $BASH_ENV, which only happen if it's set.

/etc/zshenv is always sourced, and there's no way to stop that. On the other hand, you can control if /etc/zshrc (etc) are sourced with:

  unsetopt GLOBAL_RCS
  IGNORE_SYSTEM_ZSH=1

But you would have to put it in .zshenv for it to get sourced at the right time.

Yeah, my point was simply that other advanced shells have as many files they may source as bash (if not more), with similar naming conventions. So holding that against bash and bash alone didn't make much sense.

The details and decision tree of zsh's initialization are rather well documented already (if complex) and beyond the scope of making that point.

Looks awesome!

Please use a github project (also called a repository) for this sort of thing rather than a gist. There isn't a minimum size for a github project's codebase and it's much better for code that is intended to be used by more than a handful of people.

Indeed. https://github.com/josephwecker/bashrc_dispatch - thanks to gioele for already doing some cleanups and speedups.

Solution looking for a problem. Not a particularly good one at that.

If I do a survey of 10 engineers who use Linux, and OS X every day - I'm willing to wager that 9/10 of them wouldn't be able to explain what should be in .bash_profile, .profile, and .bashrc. And I can almost guarantee you they couldn't explain the difference between OS X and Linux. I know I couldn't. I like the solution, actually:

  - .bashrc_all - always gets executed 
  - .bashrc_scripts - non-interactive bash execution
  - .bashrc_interactive - duh.
  - .bashrc_login - motd and friends (when you "login")

I'd have no problem remembering that. Why don't you think there is a problem, and, why do you believe this is a poor solution to that (nonexistent) problem?

Masochistic geeks are a dime a dozen. Once they've memorized an unintuitive design, there "is no problem."

I think most engineers can hack their way to .bashrc even if they don't know the exact differences between those files. And those that can't probably don't need to or can get someone to help them. I just don't see a huge problem out there that needs to be solved.

Why don't i like the solution? Well it breaks a well known standard which will confuse others: show me your bash_profile and bashrc...wait what they are the same?? Secondly it invokes the terror of symlinks and the spaghetti mess that often results.

Sort of a nice idea in theory but i would not do it in practice.

Am I correct in thinking that instead of symlinking .bashrc, etc. to new_bashrc.sh, you could simply have them 'source new_bashrc.sh'? This would also mean you could keep all your dotfiles in a git repository without worrying about the symlinks.

You would never need to know the difference - because you can always read the man page.

Just because I "can" do something doesn't mean I "want" to do something or that it's the best way to use my time. That's like saying that we don't need expressive function names in code because we can just refer to the documentation.

$ man bash | wc 5375 41026 312785

"From the Earth to the Moon" by Jules Verne has 39,985 words

You don't need to read the whole manual. It's hard when you're looking for something very specific, but in this case it's just the FILES section, so I'll try man bash and then /^FILES.

It's like saying /etc/services has 11033 lines in it. Well, use grep or search in the document.

Which man-page exactly?

The fun part about .bashrc, .bash_profile, .profile, /etc/bash*, /etc/profile etc. is that their semantics vary wildly between linux-distros. Not even Ubuntu and Debian agree here.

You never quite know where, when and if .bashrc is sourced by the distro-templates. A .bash_profile that unconditionally sources .bashrc will work fine on one distro, on the next you will see it run multiple times, on yet another you run into an infinite recursion.

His approach is the correct one and you'll find a similar script in just about any sizable, heterogenous deployment.

man bash

The INVOCATION part. You find this part by searching for the word "profile".

"Let's replace `cd` with a macro that knows some things about my workflow!". Runs script KABOOM. Oh crap, guess I need to make that macro only run for interactive shells. How do I do that? 2 hours later [expletives deleted].

"I want ssh-agent to start if it isn't running, and only for my login shell. How do I do that?" 2 hours later [expletives deleted]

Not so sure about .bashrc_script, but the above two are more than justification enough.

So I admit it's a 10 minute hack done on my way home on the train, but the issue is real even if you understand the man page. Here's roughly the way these work normally.

Starting with "login" bash invocation- which means:

* On linux, usually only when you first ssh into a machine or boot up into a terminal- and then _never again_ as you open various terminals or su etc. If you boot into gdm or something like most desktops/laptops do nowdays, you may hardly ever see your local machine invoke bash as a login shell.

* On a mac, "login" mode is invoked on pretty much every terminal you open.

* The login mode of a shell is _orthogonal_ to the shell being interactive (in theory. In practice, and in the gist, a login-shell is almost always a small subset of interactive shell invocations [or larger subset on mac]).

The rules for the login shell invocation are to load:

('/etc/profile' THEN (~/.bash_profile OR ~/.bash_login OR ~/.profile - one and only one, the first one it finds)) (and nothing else)

UNLESS bash is invoked as a login shell with the 'sh' name, in which case it does:

('/etc/profile' THEN ~/.profile)

IF the shell is NOT a login shell but IS interactive, don't load any of the profile stuff but load ~/.bashrc (ignored if bash is invoked as 'sh')

Finally, a shell in posix mode and a shell invoked by a script- i.e., non-interactive, will first load any file specified in the BASH_ENV environment variable and nothing else.

On a mac, the default for new home directories is to have a ~/.profile sitting there. Should you unwittingly drop a ~/.bash_profile or ~/.bash_login in there one day, you would find (and many have found) that the ~/.profile suddenly stops working (more likely you suddenly notice your PATH isn't set anymore). Except sometimes, when bash is being invoked as sh, in which case ~/.profile is alive again.

And, note that .bashrc isn't loaded at all for a login invocation, even if it is interactive (true the vast majority of the time). On a mac, .bashrc seems useless, then on linux, .bash_profile seems useless. So many on linux end up filling up their .bashrc with the good stuff and then realize that when they log in remotely nothing gets loaded so they troubleshoot, read the manpage, and end up sourcing .bash_profile from their .bashrc. On mac, people tend to do the reverse. Oh, and when you are trying to remember what is what and why you should care-- the manpage is almost 5,000 lines long. AFAIK, it's the only manpage the size of a book.

Anyway, I was putting my dotfiles into github and spend a lot of time on OSX and Linux so was merging the two sets. I also have seen (and written) so many rc scripts that liberally mix stuff that is relevant for interactive-mode with stuff that should really only be done at login, etc. That said, it _is_ a 10 minute hack so forks + fixes appreciated.

Could you achieve the same outcome by spreading the logic though the three standard bash dotfiles? Standard names plus no symlinks, i could live with that.

Kind of... That's pretty much what mine had evolved into, a bunch of them sourcing each other and making sure they weren't recursively sourced for when they're used on a different OS/machine, with complex conditionals to re-detect if it was interactive or not and logic to try to not re-run things too often that was only supposed to be run on login... with no certainty that they were really doing what I intended in weird circumstances like `ssh ... command`.

But those wouldn't have been "shareable" or standard either- unless they were all shared as a group. When someone says "here are some lines from my .bash_profile"- they are not implying something by having them in bash_profile and not bashrc. No one in their right mind would think "must be something they only want to run when they log in." Instead, you take those lines and put them into your .bashrc or .profile or whatever you happen to have working well enough. Or, worse, someone says "here's my .bash_profile" and someone new to osx drops it in their home directory and all of a sudden nothing works because bash stopped looking at their .profile that already existed, while someone on linux drops it into their home directory and it looks like it's not doing anything at all.

etc. (: obviously people get it to work, and obviously "madness" was a euphemism, but I was tired of all the ambiguity.

You could also replace each of the dotfiles with a script to source the dispatcher. No redundant logic, and you don't mess with system files in a way that might screw up install scripts, etc. (But unless I'm missing something it doesn't seem like you'd break that much by using symlinks.)

Reply on news.ycombinator.com