61 karma · joined January 5, 2010
I think we'd have to add some sort of client side caching on top of it so that we're not fetching the same objects over and over. Tend to saturate our network if we don't do that.
The other thing is that I think we'd have to add some sort of object migration so that when redis servers come up or go down we could re-balance where things are stored.
But I was curious if other people had similar problems and how they were solving them.
At least for my particular applications, though, there's either 1) a steep learning curve for programmers 2) language support issues 3) they're designed for batch processing.
Personally, I find shared memory interfaces the easiest to program when there're complicated data access patterns. But that just might be personal preference.
Though in practice I find that if performance really matters using numpy or writing a C extension module usually works out the best.
MySQL has a lot of very specific limitations about when it will and will not use the available indices. It also matters how you created the indices (one index on multiple columns vs multiple indices on individual columns).
For example, if you created a single index with the columns (date_added, device_id) MySQL would not be able to use the index since device_id is the second part of the index, not the first and thus not available for use in the WHERE clause.
See http://dev.mysql.com/doc/refman/5.1/en/mysql-indexes.html for more limitations.
And the other sources don't seem to provide offer letters, NDA/invention rights docs, board motions, option grants and all the other boiler plate stuff that's needed.
That all said, I suppose making it too turn-key risks entrepreneurs not really understanding the legal issues involved and getting nasty surprises later on.
I think this is a really interesting observation. Many of the interesting things in computers such as advertising bidding systems, social engineering security attacks, social networks, collective knowledge systems, games etc are at the intersection of technology and people.
Specifically, what I loose with transactions is fault tolerance and predictable performance while I find I don't really need atomicity in most places.
Fundamentally fault-tolerance is achieved by having few interdependencies among computer systems. Transactions makes that really hard to do for replicated databases and you need replication to be fault tolerant. So usually people cheat and settle for async replication, but then you really don't have transaction anymore since different parts of your system are seeing different states.
Transactions also lead to difficult to debug blocking behavior either due to deadlocks (btw, where else in computer systems is the advice that you have to destroy all abstraction and understand the entire locking order of your program in one place?) or locks getting held too long due to programming errors etc. I hate when my entire site hangs because a key table has locks on it due to some transaction getting accidentally left open.
Understanding database consistency levels and their nuances is extremely difficult and error prone as well. So you may not really be operating in the pristine transactional environment you thought you were.
Finally, programming to a non-transactional model is actually pretty easy in most cases. You just have to go into your project thinking about this upfront. And the decisions you make building a non-transactional site will probably increase how decoupled all the parts are which will make it scale better anyway.
1) performance matters immensely to user experience and I can hand write much better SQL than any ORM I have seen can generate. Additionally I find it easier to predict how well my SQL will perform than to predict how the SQL generated by my ORM will perform.
2) I will likely never switch databases during the lifetime of my company and so switching SQL dialects is not a problem I face.
3) There is an impedance mismatch between tabular data stored in databases and object graphs that most ORMs represent.
4) The specifics of every ORM I've tried always left something to be desired. Hibernate was wicked complicated. Django didn't support multiple databases, SQLAlchemy was error-prone. I'm sure these specifics will get fixed over time, but for me they've just provided further disincentive.