PopSQL – Modern, collaborative SQL editor for your team | Hacker News Reader

PopSQL – Modern, collaborative SQL editor for your team

popsql.io

170 pointsfedericoponzi49 comments

There doesn't seem to be any PopSQL users around here so I'll share my experience with it.

Currently, we're a team of 5 on a paid plan and we're loving the tool. We mainly use it to:

- Share queries to extract some kind of data from our databases - Quickly run queries to answer a quick question - My favorite use case: We create queries around bugs we've noticed in our data. We add "TODO:" in front of the query's name. We'll then move the query to a "Done" folder once we get the expected result.

Other solutions in the `write sql and share the graphs with your team`-space:

- ChartIO

- 'WagonHQ, Modern SQL Editor' [1] (now aquired by Box)

- MetaBase [2,3]

- Redash [4]

MetaBase is the only one that I know of that is

- fully open source,

- mature (graphing options, permission model),

- still actively maintained, and

- both friendly to non-technical users and expert sql'ers alike

[1] https://news.ycombinator.com/item?id=9792464

[2] http://www.metabase.com [3] https://news.ycombinator.com/item?id=10425959

[4] https://redash.io

Metabase it’s pretty nice, although it’s a bit slow and resource intensive.

It also seems to have some serious bugs. For example, none of their SQL Parameters examples[1] work with a Postgres connection. I just downloaded it this morning to give it a try after reading this topic.

[1]http://www.metabase.com/docs/v0.21.1/users-guide/12-sql-para...

As far as I know there is no way to use or download Wagon now that Box has acquired them

This is true an also what gives me pause before trying any more tools like this.

Would people pay for a desktop tool like this? How important is sharing? I had built something a year ago (on top of PyQt) but shelved it for lack of interest.

I might if it wasn't tied to a cloud service. Sharing not that important for me unless it can be done without the need for getting a 3rd party involved.

To add on to your list:

https://www.holistics.io (I work here).

We use and like redash, because we can have it trigger zapier tasks when there are new results on a query. Makes cross-integration when testing new things great, and saves time in development being able to hack together a stop-gap solution this way.

Can you go into more detail on this?

I'd add Blazer to your list too. Used it with great success and it meets your 4 criteria.

https://github.com/ankane/blazer

Thanks for the links! Here's a recent related discussion of another product:

Franchise – An Open-Source SQL Notebook | https://news.ycombinator.com/item?id=15303833 (Sep 2017, 63 comments)

https://github.com/hvf/franchise

Also mentioned there:

https://github.com/apache/incubator-superset

Redash creator here.

Just to make sure someone doesn't get the wrong impressions from your comment: Redash is 99% open source (and I'm going to close this gap this month[1]), mature and actively maintained. The friendliness is subjective, but we're not trying to please everyone :-)

[1] Sometimes it's easier to prototype new things in the SaaS version, but everything reaches open source eventually. There is practically one feature that wasn't open sourced until now, and I'm going to add it to the open source version now.

is the feature joining multiple data sources? i'm a fan of redash btw, people should check it out. i use it as a docker container for self hosting

It is :-)

Redash is more that just a SQL editor. It's a full-featured dashboarding system which works particularly well with relational data sources.

In case you're interested in more tools in that area, I'm actively maintaining & updating a list of BI related tools: https://github.com/thenaturalist/awesome-business-intelligen...

Or, we could just go all meta, in a Postgres shell:

  CREATE TABLE queries IF NOT EXISTS (
    name varchar(64),
    version varchar(64),
    query text,
    description text
  );

  NOTIFY chat 'Alice: Bob, please insert that query into the new queries table, and then NOTIFY "chat" with the name and version of it';
  LISTEN chat;

  Asynchronous notification "chat" with payload "Bob: See update-rank, v. borked-1" received from server process with PID 8448.
  Asynchronous notification "chat" with payload "Bob: I have it set to update in a temp table, so we don't have to reset the real table" received from server process with PID 8448.

  SELECT queries.query FROM queries WHERE name="update-rank" AND version="borked-1"
  INTO query;

  \echo query

  -- "Looks sane, let's run it"

  DO $$ EXECUTE query INTO result $$; -- Something like that, I'm guessing.

  \echo result

PL/PgSQL's EXECUTE is not to be confused with the EXECUTE that is used with stored procedures.

Bonus points if you edit the query by shelling out to sed or butterflies.

Marketing Electron app as native and requiring sign in with Google account only is not an option, just for a try...

How are credentials and result sets transfered? Is it directly from my computer to the server or is PopSQL a middle man in the communication?

This is a key question! Also want to know as I’m considering signup and data security is a factor.

It seems to store the database URL on the server (popsql) but the username and password are stored in the Mac Keychain, any "team members" must enter the database username and password against when attempting to use the connection.

I am unsure of what happens for other OS's.

PopSQL co-founder here, if you choose to share your database host, port, and name with your team, then we'll store that on our servers, but never your database username and password. If you choose to get a link to share your results, the results are uploaded from your computer directly to S3. Hope that helps!

I like the GUI, I like the landing page, I like the fact that it has native versions for the 3 main platforms... Congratulations for the team or person behind this! Great work

Question: is the native app actually native or JavaScript in a web frame?

Web frame (Electron), at least on Linux.

Electron app on Mac

Postgres user here. pgadmin4 lead me to try alternative clients. Recently, I've found DBeaver useful. I'd give popsql a try if it didn't require google account information.

I want so badly to like DBeaver. I'm actively using it, but on Linux the rendering always gets screwy with me. Almost every time I start the program I've got to change themes just to get it usable.

It's fine to require people to pay for your software. I still think it's a good idea to release at least a basic demo version as open source though. With close binary blobs for all I know it could upload all my personal files or delete them for good, drop all database tables, install a backdoor, etc. I don't trust random people on the Internet to run their code on my computer.

This is what permissions are for, we use limited read-only access for all 3rd party tools.

Please add Oracle support when you have an opportunity (I was going to give it a quick try, but unfortunately it doesn't appear to have support for Oracle just yet and that's the primary database our internal systems run off of).

Being an electron app, can anyone tell me how it behaves when viewing 200, 400, records at a time? I attempted to open PopSQL but I don't really want to sign up for an account just to try the thing out.

I tried out TeamSQL a while back and it choked pretty hard at about 250.

Not quite directly SQL, but you can try CSVExplorer.com to work with huge CSV dumps from your db.

(disclaimer: I built CSV Explorer)

Just tried this out

- Chart is greyed out on my data. Why is that?

- Can I provide parameters (such as a date) in the query that can be changed by whoever wants to run a report? I can't see how to do this.

- Can I publish this as a dashboard that can be used by non-technical team members?

PopSQL co-founder here. Regarding the greyed out chart, we rely on your results being in a specific format to render a chart. Check out this help article for more details on that: https://intercom.help/popsql/how-to-create-charts. We have plans for advanced charting, query parameterization, and dashboarding!

I love the name!

>Free for individuals, or teams of less than 3 people

That's a convoluted way to say "1 or 2 person teams"

"Stop emailing SQL queries or pasting them in chat"

Hunh you don't you put your queries in sprocs and store that in the data base and your source control system of choice.

Soory I don't see what problem this product is for apart from maybe enabling sub optimal coding practices

> Hunh you don't you put your queries in sprocs and store that in the data base and your source control system of choice.

Not everybody enjoys sprocs-- having 3 sources of truth (repository, dev db, prod db) gives me headaches.

so you have the same problem with code dev test and live you just need to be professional about it.

Are you really arguing that "you already have 1 deployment headache, why not add more?" One of the signs of professionalism is simplicity-- reducing moving parts and things that can go wrong.

so is your not your sql code in the same way the non sql code is ? Ignoring source control for you database is highly unprofessional and in my opinion just lazy.

Nobody's "ignoring source control." Statements are written as .sql files in git repositories.

Story time;

I've recently started working with a SME (<100 users) who took this approach after abandoning the vendor's reporting recommendations for their Line of Business application (apparently it was just crap).

Other areas of the business cannot be without these reports under any circumstances. Negotiation is not an option.

Since about 2012 they've had various employees generating reports, alerts and data warehouses as objects in the database. Naturally some of these have been superseded. Very often the old objects were left behind "just incase". Almost all of the original authors have left and documentation has been lost or just didn't exist in the first place.

Many of these objects are dense and difficult (because SQL was the wrong tool for the job, or because they go many layers deep like matryoshka dolls).

Demands for changes to reports are frequent (weekly), and some reports functionally overlap, but produce wildly different results for different areas of the business based on various "rules".

Honestly, the database is a fucking mess (approximately 1900 objects relating to reports, alerts and data warehouses - some of which are dolls going many many levels deep., There's a huge sense of shame and fear over trying to regain control.

I totaly understand that this was entirely a human problem. With more restraint this wouldn't have happened. Both inside and out side of the IT team. However, a tool like PopSQL, metabase, etc. I believe would've helped with;

1. Ad-hoc queries that didn't really need to be full objects

2. Discoverability through prettier annotations, etc. - reduction of reports being duplicated

3. Auditing - last access/run times, etc. would be much easier to find to reduce the cruft

4. If the tool is more appropriate it could off load some of the things that SQL is not good at

There are undeniably good reasons for storing reports, alerts and data warehouses (developer maintained objects) in version control. If the report creation/alteration requests don't come often, then I'd argue it's ideal.

However living in the real world, I personally feel it needs to be balanced in conjunction with tools like PopSQL, metabase, etc. Right tool for the right job.

I've seen this same story at every job ive ever had or contracted for. Fortune 100's down to mom and pops.

I think metabase/redash/etc is a great fit for the development of reports and the kinds of things that dont need to be anything more than a sql query. IME fast turnaround can be huge for lots of businesses.

I agree with all of your points. A lot of the problem is organizational and political and historical - but the right tool can make a big difference.

Any tips on prying people away from Excel, and into something like metabase/Tableau/whatever?

That seems to be one of the biggest hurdles I think I'm going to have to figure out, rather than the technical side.

a decade ago I would have had a lot of ideas for you but as time goes on the more I have learned to embrace excel.

Theres two directions you could be coming from though - if its a problem of the users producing data and storing it in excel instead of your database thats more of an application problem.

But if its a matter of excel being where your users want to look at and work with the data - I say embrace that. Use your metabase/tableau/redash to provide them with that data and let them get it out in excel and do whatever they want. Give them a little time and then let them show you what theyre doing with it.

IME about half the time theyre actually doing things with it that you cant really help them with unless you spent a ton of time. The other half you can take what theyre doing and either apply those changes into the query itself, saving them time and sharing that work with others - or teach them a better/easier/faster way of achieving whatever it is theyre after. I use it as a long-tail way of gathering requirements a lot of times - let me give you all the data you might need for this and you figure out how it can be useful for you.

Reply on news.ycombinator.com