Squash | Hacker News Reader

Squash

squash.io

229 pointsvital10181 comments

This product reinforces a mindset that I find harmful.

Where I work, teams of developers are responsible for products -- we all own the quality and robustness of our product's code. If the application is broken, why would you only email the person or pair who wrote the particular line of code that excepted? Am I supposed to pretend broken code is not my problem unless I personally wrote it?

Development is a collaborative activity. If a product is broken, the whole team needs to know, so the whole team can fix it.

I don't disagree. However, it seems like the idea with Squash is to give the person who is most familiar with that code the chance to fix it quickly.

Fixing a bug early in development is my problem. Fixing a bug in production is all of our problem. I think Squash encourages the former in order to mitigate the latter.

There are a couple assumptions here that seem like they could be pretty faulty:

1. Whoever touched a line of code last is most familiar with that line of code. I just moved a method to a different module, so following this assumption I am now the person most familiar with every line contained in that method.

2. The line of code triggering the exception is actually at fault. I just changed a value in a configuration file and it triggered an exception off in some code that creates a network connection - code I've never touched.

How faulty these assumptions are probably varies pretty wildly from organization to organization.

In your first example: you are not necessarily "the person most familiar", and I don't think that's the assumption. You are, however, likely (to some degree) to know who should be contacted to resolve the real problem.

In your second: If you changed a value in a configuration file and it triggered an exception somewhere, many people would say you are at fault for not testing your change.

Most places I've worked there's a simple rule: whoever last touched the code that broke is responsible for getting it fixed. That isn't meant to be taken literally in that I must actually make the code change myself. It means that it can be assumed that if I'm touching some code, I'd better be familiar with the code I'm touching. If that means I delegate the fix to the original author, then great: It's still getting fixed. If I can delegate to someone who's better suited to make the code change, that's OK too. If not, I make the fix. In any case, If I'm changing a line of code, you can assume that I'll know what action to take if that change breaks something. If I don't, I shouldn't be allowed to make the change in the first place.

In your second: If you changed a value in a configuration file and it triggered an exception somewhere, many people would say you are at fault for not testing your change.

Exactly. However, the exception will likely be thrown from somewhere in the network stack, not from a config file, so whoever last modified the error checking code in the network stack will get an email instead.

I do not disagree with the sentiment; it is useful to immediately know if there's been a bug introduced that might slow down your work and you may be in a good position to offer suggestions to fix. But also consider:

- If the bug is a regression it may stem from the author's misunderstanding of the codebase. By being the one to fix the bug they have a chance to learn correctly how it works and reduce future mistakes.

- The original author of the bug was just in the code and can likely most efficiently fix it since they are recently familiar.

- It would be wasteful to have two people working on the bug at the same time (though bug policy can also help with this).

- On a large project there can be lots of noise which does not affect you, especially if the project is well compartmentalized.

- Taking responsibility is always a good thing!

It is only toxic when people start pointing fingers and shifting blame instead of helping out. As a "component owner," I personally like to fix my own mistakes so I can learn from them.

There's power in the concept of chain of custody. I agree that you don't want to associate negativity with this stuff, but identifying an owner who understands the code and whose job it is to fix or find the right person to fix has huge merit. I've worked on too many projects where the issue tracker balloons due to diffusion of responsibility.

Perhaps a way to configure it to work in that environment may be: Bug email gets sent to the dev it determines responsible, and if they 'sit on it' it gets sent to the whole team. With an appropriate sit-on-the-email period of a couple of hours, or a day. If it was really critical the dev in question could easily escalate it anyway in that kind of environment.

This should give you the contextual advantage first, and then collaborative advantage very soon after.

This has a big (but easy to fix) security flaw. Don't deploy this publicly without changing the secret token or else you are effectively publishing your app code for analysis by attackers.

https://github.com/SquareSquash/web/blob/master/config/initi...

More broadly, if you're writing an open source rails app please don't commit a hard-coded secret_token into the repo or session fixation attacks are trivial.

Why has the Rails team not fixed this?

Seems like a sane default would be to have a special token (used in the auto-gened config) that generates a new random key, and then writes it to a 2nd config file (which is in the default gitignore)

Because it's not a problem for private applications. You actually do want the key in your VCS in most cases since it should be stable across deploys. Rails will log out every user and throw an error for each active session when the key changes because it suspects that the session has been tampered with.

The handling for OS applications is a little more difficult and I must admit that I know a couple of mediocre and no good solution to it.

In this app's case the setup.rb is an ideal candidate for generating a new secret_token (and raising an exception on startup until you've generated a token).

I actually went through the setup.rb and was surprised they don't do it. Rails itself will raise the exception if the initializer does not exist IIRC, so just removing the file and .gitignoring it would do.

.gitignoring will hurt anybody deploying via git (e.g. heroku) in the absence of some ENV magic. It'd be fine to have the regenerated file checked into each user's repo as long as they have the good sense not to push that repo up to a public fork on github.

There is talk about more general solutions here:

https://groups.google.com/d/msg/rubyonrails-core/N2EFnf6X_i4...

Neither of the solutions is actually beautiful, they all require some sort of deployment magic - env variables set, cap symlinking files or similar. What works for me probably doesn't work for you.

Btw: you can check the regenerated file into the repo, adding it to .gitignore just prevents you from accidentally adding it:

  Last login: Tue Jan 15 17:07:14 on ttys005
  Voice-of-Evening:~ fgilcher$ cd /tmp
  Voice-of-Evening:tmp fgilcher$ git init test
  Initialized empty Git repository in /private/tmp/test/.git/
  Voice-of-Evening:tmp fgilcher$ cd test/
  Voice-of-Evening:test fgilcher$ echo "README" >> .gitignore
  Voice-of-Evening:test fgilcher$ touch README
  Voice-of-Evening:test fgilcher$ git status
  # On branch master
  #
  # Initial commit
  #
  # Untracked files:
  #   (use "git add <file>..." to include in what will be committed)
  #
  #	.gitignore
  nothing added to commit but untracked files present (use "git add" to track)
  Voice-of-Evening:test fgilcher$ git add -f README
  Voice-of-Evening:test fgilcher$ git status
  # On branch master
  #
  # Initial commit
  #
  # Changes to be committed:
  #   (use "git rm --cached <file>..." to unstage)
  #
  #	new file:   README
  #
  # Untracked files:
  #   (use "git add <file>..." to include in what will be committed)
  #
  #	.gitignore
  Voice-of-Evening:test fgilcher$

That doesn't stop you from accidentally pushing it to GitHub though.

No, quite to the contrary: Once you do this, you'll push it in the next commit. But in the case of an OS app you'd probably want to create a private repo for the app since all your configuration would be public otherwise. If you deploy to heroku, you'll have at least one private git repo on heroku.

I have to disagree. An application connected to a network is never "private" unless you configured a strict firewall and are sure that other running services don't contain exploits. Besides, if an employee leaves the company, this "private" app is now known to an outsider.

Also, just because something needs to be "stable across deploys" doesn't mean it needs to be in VCS. Are your application's third party passwords and API keys all stored in its version history? We picked a solution where the deployment tool configures the sensitive pieces of the application.

Private was meant as "the source is kept private". In this case it's less of a problem since only people with access to the source get to see the source. In any mid-size team those probably have access to the servers anyways, so they have access to the secret in any case, so they're implicitly trusted. You're certainly right that there are other ways to store the secret and if you read my posts in this thread you'll see that I'm aware of that, however, keeping the secret in the VCS is a simple solution that works without any further magic. If your requirements dictate that you can't do that, chance it, but that doesn't mean it's not a working solution for many other people. And that's the reason things are as they currently are and I don't expect them to change soon.

To answer your second question: depending on the case, I store 3rd party passwords and API-keys in the repo. If an employee leaves the company I'll have to change those anyways since he probably had access if he had access to the project at all.

The setup script has been updated to generate a session token. Thanks!

I can't watch the video right now, and as a result I have no idea what this actually is (despite being able to read about all the features).

Fewer e-mails sounds good, but it doesn't mean much to me if I don't know what the product is.

I came here with the same problem, hoping these comments might give me insight. Please add a summary of what this is please! :)

Did you guys read the same thing I did? AFAICT it's a service for logging bugs, analysing who is most recently responsible for the bit of code at the point where the exception/segfault happened, and emailing them.

To be fair to everyone, I was initially confused, but I was able to piece together what it is. However, I'm not sure about it's scope - like if it's a general bug tracking system (as opposed to just bugs that generate exceptions or stack traces), or if it is even for features, not bugs. I think it would help the project to say these things up front.

It's self evidently an exception logger with built in static analysis. If you can't tell that from the lengthy description then the product probably isn't aimed at you.

It's definitely one of those simple "why didn't I think of that?!?" type ideas. Well done to the team responsible.

Not so evident; "exception" is mentioned once deep down. Zero mentions of "log", "logger", I thought at first it was a debugger ("bug" is mentioned 9 times, the first header is "Say goodbye to bugs").

They should have a once sentence description involving exceptions under "Say hello to Squash."

You're right, from a marketing point of view it could be clearer. It's not as opaque as a lot of landing pages you see though. All it needs is that descriptive strapline, the rest of the info is there.

What's not clear to me is if it's suited for bugs that dont' result in an exception and/or stack trace.

I had the same problem, don't want to watch the video as I am at work and I read quickly the headline... didn't get what it was so I closed the page. I got back to HN and read your comment and only got back to the website at that point to finally figure out what it is. So a little headline wouldn't be too much below the title (and I'm part of the target audience).

As I understand it, it acts as a reporting tool for errors that occur in your web app.

For example, when a user encounters an exception you get notified about it while at the same time squash gives you a nice interface to view how many times the error occurred, where in the code it occurred, etc.

The opening statement in the video is,

"Hello, I'm Tim Morgan and I'm going to help you get started using Squash, the open source exception reporting tool that's better than what you're currently using. Plus, it's free."

It looks like a tool that can automatically track bugs and notify specific developers whenever exceptions in your project come up.

From the github README:

"Squash is a collection of tools that help engineers find and kill bugs in their code."

https://github.com/SquareSquash/web

Are you sure you actually read through the webpage? I haven't watched the video either, and I was able to figure it out pretty easily.

> ...analyzes the stack trace of every exception, and determines which line in the backtrace is the source of the bug.... uses git to figure out who might have caused the bug... shows you where in the code the problem was, so you can quickly get to fixing it...

So, it's an exception logger that integrates with your version control system and `git blame`.

> ...sends an email to the engineer at fault...

It emails someone when it figures out they were responsible for an exception.

> ...If the engineer sits on the email, eventually it escalates...

It messages the rest of the team if the bug doesn't get fixed.

> ...Dig inside the values of environment variables, instance variables, request parameters, and more... analyze information about a bug to determine its root cause... symbolicates iOS crashes, un-minifies JavaScript code, and de-obfuscates Java code...

It tracks data about your application, and captures state useful in digging into the cause of a bug.

> ...full-featured commenting system, ticket-management system similar to JIRA, and a news feed... PagerDuty and JIRA integration...

It has collaboration features and integration with various existing development tools.

But it's for only rails applications? Only for web applications? Only for ruby applications?

It currently has clients for Java, Objective-C, Ruby and Javascript. So it most likely supports any application you're working on as long as that application has a git repository. Feel free to contribute a Python client.

If you make me watch a video to get what your product is about and offer no succinct text explanation, I bounce 100% of the time.

This isn't a commercial product, it's an open-source project. It solves a real need and people who need it tend to be more interested in the quality of the product than that the splash page communicated maximum clarity of information in 5 seconds.

Also see: https://github.com/SquareSquash/web

Interesting FAQ full of the developer preemptively defending design decisions.

Thanks for pointing this out. Definitely worth reading.

Another open-source alternative is Sentry (https://getsentry.com/), which supports multiple languages and frameworks. A hosted version is also available.

The Ruby support in Sentry currently isn't quite as good as the Python support, but it's not bad at all, and constantly improving.

I second using Sentry, and it has UDP support. Maybe a few Ruby devs can just improve the Sentry Ruby client?

I third using Sentry. I love it. David has been making huge improvements in the Ruby client in recent weeks.

Interesting: With the whole 'marketing' page I expected a hosted service (and was looking for the pricing page), not an opensource project. Looks great!

>Squash uses git blame to figure out whose fault it was

Questionnable assumption, to say the least.

Still, not a bad idea. You could also send a file to the people who contributed most of the culprit file.

Agreed - the cause of a bug isn't always a line present in the stacktrace.

You need to start somewhere and with someone and the assumption that the person who last changed the line has at least some knowledge about the problem is better than spamming everyone.

I'm surprised at all the criticism over the blame mechanism. It's clear the goal here is to avoid the diffusion of responsibility that occurs when dozens of people are getting the same exception emails. And in any case, what kind of developer worth his salt is offended by blamed for a bug? I mean sure if someone is being an asshole about it, or if they jumped to the conclusion that it was you when you had nothing to do with it, but if git blame algorithm flags you then what is there to be offended about? Surely we all accept that good software can not be created by tiptoeing around the facts and occurrences of the software development process?

This looks like fantastic software, it's the most exciting thing I've come across in several weeks or maybe even months, and I'm looking forward to trying it out.

agreed. Squash is finely tuned with a set of features to fill in gaps found in another very popular commercial tool in the same vein.

tl;dr: don't knock it till you try it!

This is all fine and dandy, but there's too much emphasis on finding the "engineer at fault" and blaming.

The emphasis is on finding someone who can either fix the problem or knows someone who can. The idea is that when you get an email from Squash, you should pay attention and respond immediately, rather than getting thousands of emails that you don't really care about.

In practice the tool doesn't have any blame feel to it. When I commit code that causes problems Squash correctly identifies me and lets me know what to fix. And when I was mistakenly chosen I'm still a good candidate for triaging the bug and sending it off to the appropriate developer.

The only thing that instantly tells me what squash actually is is in the <title> tag. Which chrome truncates to ~30 characters.

Is it a pest control system that doesn't send me emails?

The focus on finding "who's at fault" is going to immediately put people on the defensive, I'd imagine.

It's more about finding one person to give responsibility to (or who can delegate responsibility). Emailing a whole team means a whole team ignores the emails.

If this lives up to even half of its potential, it will be fantastic.

Also, routing exceptions using git blame isn't about placing blame on individuals for defects, it's about automatically managing and routing the massive quantity of exceptions that can occur in a large rails app with many users and many developers.

First you have a bug in your javascript somewhere. I use a real mouse and if I scroll to the bottom of the page via the scroll wheel it jerks me back up a few pixels.

Second, as others have mentioned, I loaded this page, scanned for a few seconds, and still really had no idea wtf Squash was.

Finally, this phrase: "Squash uses git blame to figure out whose fault it was" is really terrible. This assumes that someone is at "fault" for introducing a bug. Not all software organizations embrace this concept and instead consider bugs being introduced as a team-wide or process failure, and do not make a point to assign blame to defects to individuals. Be careful not to shoehorn your own views of methodology into your marketing.

Fixed up the marketing page.

It isn't about blame, it's about finding one person (and only one person) who is likely to be able to fix the bug or at least know who to delegate the bug to. The workflow is around reducing signal to noise, so that if you get an email from Squash, it's something you need to take action on. Either fix the bug or find someone who can. That's why it always tries to find one person to give responsibility to.

The name, seeing a word like Symbolication and having just watched an Onion video made me think this was a joke at first, even while watching the video.

All the breath & pops in the microphone is really annoying. You should re-record the audio using a pop filter or something?

Have you considered using a professional voice over for the screencast? It will help you look more trustworthy.

Yes, and bringing on a non-technical cofounder who can do marketing for you. What's better, 50% interest with a big market share or 100% of none? How are you ever going to be on the front page of TechCrunch?

Seriously, this is an open-source project. Why in the world would he use a professional voice over? That's just weird. If anything, the home page is too polished. People seem to be mistaking it for a well-funded commercial endeavor.

Nice app - a great alternative to some of the popular hosted services. Great work guys.

Could you please name a few of those hosted services? Because Squash seems currently to be written for Ruby specific apps, however PHP, Django, Nodejs apps could use such a service too.

We use Exceptional (exceptional.io) and Airbrake (airbrake.io). Both are fairly good but occasionally have discrepancies.

(self plug) Check out http://ratchet.io - we have libraries for PHP, Django/python, and Node.js, as well as Rails, client-side JS, and Flash.

Bugsnag (https://bugsnag.com) supports Rails, PHP, Python/Django, Nodejs, Android, iOS and Unity. Caveat: I'm a founder there.

Squash has client libraries for Ruby, Objective-C, JavaScript, and preliminary Java support.

Agreed - the PHP world seems really short of such tools, which either means a) they're there and I've missed them, b) no one in PHP-land is interested in writing them, or c) Integrating w PHP's core is much harder than other languages.

For example, NewRelic's PHP support is still not on a par with their Rails support...

Just as a heads up, I cannot scroll to the bottom of your page. I keep bouncing as I hit the bottom and end up with 'Start using Squash' being cut in half. Screen res: 2560x1440

Looks good, I just wished it integrated with Github instead of having it's own UI. Github already has all the functionality the squash UI webserver has.

Nice, going to give this a try, along with errbit! (https://github.com/errbit/errbit)

Yes, very curious why the effort didn't go into an existing solution like errbit? Off the top of my head, errbit is rails/mongodb, which isn't a great combo for scaling out to the size of, say, Airbrake, but then Squash itself is written in rails.

The code in Errbit is quite reasonable, and the team is great as well - would love to hear the rationale!

The origins of Squash predate Errbit. In addition, Squash is laser-focused on solving a specific problem, the signal-to-noise ratio in dealing with exceptions. The idea is get the bugs assigned or fixed as soon as possible, and bother as few people as possible.

We have been using Errbit for about six months. We are reasonably happy with it. On the plus side, it does the job; that is, it allows me to do my job. The UI is reasonably clean, and it reports every exception to our Campfire room.

On the minus side, Errbit is somewhat buggy, and very slow. If it gets any load at all (say, above 2-3 exceptions per second), it becomes near-unresponsive. A casual look at what it's doing seems to indicate MongoDB traffic. We are running Errbit on a small VM, but the slowness is way beyond what can be explained by the VM. So on the rare occasion we get an exception storm it's pretty much impossible to access Errbit.

It's also terrible at merging identical exceptions. Not sure why, haven't look at the code, not bothered enough to do so.

We had similar problems as well, but I do miss the user experience. Performance-wise it's just not there though.

So this is open sourced by Tim at Square. We're hiring (-: https://squareup.com

In that video, what extras does he have installed on his Terminal? Anyone?

zsh with a custom theme probably

Looks like it. A good starting point for those that feel like trying zsh out should check out oh-my-zsh (https://github.com/robbyrussell/oh-my-zsh).

Symbolication is the ugliest word I've heard this year. It sounds like disembowel and embolism put together.

Also it looks like they found a font to abuse in exactly the same way that Comic Sans kills aesthetics, but with a different name and a slight update.

To be fair, it's still early 2013 ;-)

Reply on news.ycombinator.com