Hacker News Reader

Z-mongo-admin, the Meteor equivalent of phpMyAdmin, was a project that won a prize at the Meteor hackathon earlier this year. [1] Here's one of its authors, Geoffrey Vedernikoff, speaking at HackMIT. [2] Perhaps you could compare notes with the team to see what's going wrong in your version.

[1] http://www.meteor.com/blog/2013/07/09/congratulations-to-the...

[2] http://www.meteor.com/blog/2013/10/11/meteor-at-hackmit-onet...

I'm sure his project worked, too, it just didn't scale.

This is true. In the Q&A of this video [1], the creator of Z Mongo Admin mentions that the browser will hang if the collections are huge.

[1]: https://www.youtube.com/watch?v=yeF_b8EQcK0

Speaking as the author of Z Mongo Admin, it's mainly because we haven't implemented pagination yet. Can't be too hard to add {limit: 20} to the queries though.

So do you do you think Meteor can scale? I don't know if I'm following, but I'm assuming limiting the queries(?) means limiting the size.

Your claim that the client has a full copy of the server's data is incorrect. By default that is the case, but removing such functionality takes only a few lines of javascript and removing the "autopublish" package (one command).

Seriously, Meteor has grown up a lot recently. This stuff moves quick. I encourage you to take a look at it again, because I imagine many of your initial problems with the platform have been settled - there are still people out there who feel any user can clear out the whole DB from the client .

No stopping autopublish doesn't solve anything in this case. In my leaderboard example, the clients view of the highscore is dependent on all data in the collection which has to be published and the client has to subscribe to it.

Even if you only want to show the best 10 scores of your 500k population, Meteor doesn't allow you to subscribe to the Mongo equivalent of "select player, score from results order by score desc limit 10" (last I checked anyway) And even if it did, well what if you want to see result 11-20 or the 10 best scores among your friends?

You or someone else is very welcome to prove me wrong here and I'll admit I'm an idiot. All that is needed is 1 grid, 500k items, allow me to sort up and down on the headers and let me paginate through it as I please and support 5 simultaneous users. Trivial to write using Rails or Django. I claim impossible to write in Meteor.

> Meteor doesn't allow you to subscribe to the Mongo equivalent of "select player, score from results order by score desc limit 10" (last I checked anyway) And even if it did, well what if you want to see result 11-20 or the 10 best scores among your friends?

That's actually all possible. Just write a good publisher which takes the number of results as an argument. All the sorting can happen on the backend and then you push down the x results.

Sure, not as effortless as autopublish, but also no harder than in traditional apps.

It's quite easy, and you still get reactivity. You publish the collection as before, but you write a server-side filter (a well-documented core feature) that only sends a small piece of that to the client. Even though the client only sends a small piece, the changes to those items are still reflected back to the server and other clients live.

It's supported this from the day it was released - I remember hooking up something quite similar at the time. It was never purely autopublish.

In addition to that to make robust pagination you need aggregation. Printing the message "you are looking at page X of Y pages" requires you to know the count of the collection or else you can't say how many pages there are. I.e You need to reactively monitor MongoDB's equivalent of "select count(*) from blaha". Doing that (without cheating and caching the count) is very expensive in Meteor.

How is caching the count "cheating"? Grabbing the count for every request would be ridiculously expensive in any framework, not just Meteor.

One count(*) query per request is not expensive at all. Suppose you cache the count and someone else adds rows to the collection. Your cached count is not getting updated and the whole reactivity concept breaks down since your app now thinks the row count is different from what it really is.

5 minutes of my time, code to do what you're saying: https://gist.github.com/MaerF0x0/7242600

You could have asked for help, but that wouldn't work very well here, it's not what this site is for after all.

Far more effective to announce something incorrect and let the forces of truth and justice solve your problem for you. Well played.

Yep, if only the "forces of thruth" actually knew what they were talking about it would have worked great. :) Btw, I've asked for help in other places (which I won't link to as I like to keep my internet identities somewhat separated) but no one has an efficient way to deal with large collections.

This is very easy to do in Meteor. You just tell Meteor to only publish the information you need, just as you would in any other web application.

I am currently running an enterprise software project on meteor with thousands of pieces of data in the database. If I published all of the data at once, it would crash the browser. I know because I actually did it when developing. Once you write the correct publish functions, it runs extremely quickly and only has the information you need in memory. All of this data is reactive too. And yes, you can paginate the information.

Wow, thx for sharing! I would love to talk to you more about what you are building - josh AT differential.io

The most hilarious thing about this is how uncannily your claims mirror those that Rails had to deal with when it was new - people saw the easy-to-get-rolling-with scaffolds and assumed Rails couldn't make your app look like anything else.

It took the better part of two years for people to stop saying so. The "smart, sensible people" condescendingly repeating "you need more than a _scaffold._" It was just as amazing then.

I was thinking the same thing. There were literally the same arguments against rails in 2006-2008:

- it won't scale

- too much magic

- it's is too opinionated. Only good to do things one way

- it's ok. but only for toy apps

- I can do that easily in X framework, language (php, java, and now rails)

- these are just toy example. can you do anything "complex"

The author says this is significantly cutting down on his dev time. He also mentions how meteor reduces context switching when developing. IMO those are signs of a very promising technology. I don't know if meteor will become practical or popular. All I know is if works and it's stupid, it's not stupid.

FWIW Rails did not scale well during that time period. (One word: Twitter.) I don't know what the situation is like now but performance was a serious issue with rails in the early days.

What sites are going to have Twitter's scaling problems, though?

And more importantly, I am pretty sure most startups would gladly have twitter's scaling problems in exchange for being the next twitter.

Admittedly very few, although I did run into performance issues on Rails sites handling orders of magnitude less traffic. It's an interesting use case though -- tough to imagine a simpler web app than circa-2006 Twitter. Tellingly, they ultimately threw up their hands and moved it all to Scala.

I would say that your argument is one that can be fixed by a number of means:

1. Removing the local copy of the DB.

2. Removing the need to use MongoDB.

#1 will let you have your responsive clients while #2 will let your application scale. We've got 2.5TB in MongoDB and I can't wait to get away from it for our use case.

(MDG-er here) A few meteorites actually did write an equivalent of phpMyAdmin for the Meteor Summer Hackathon, in case it's of interest to you: http://www.youtube.com/watch?v=ixJyB8Z-tU8 :-)

this is so wrong, you should really read the documentation first.. turn off autopublish and add server-side filters, should take about 2 minutes

You should really read up on autopublish and turn it off for your app.

Don't make bold claims unless you try a little harder!

Meteor has publication/subscription system that ships only data you specify w/ very fast communication w/ server.

I hear ya... from what I can see it looks like it would be great for rapid prototyping, even big systems... however why anyone would go with a JS-backed framework for a serious application is beyond me - it's just not ready as a language (and a framework)

Most frameworks aim squarely at programming in the small/medium scope.

The Big Lie is that you should outsource your design to them completely, because the people who designed them are smart. Nobody ever calls out the implicit appeal to authority there.