It took Israel security services around 3 years, to stop most bombing and rifle attacks. They are better funded and staffed (forced conscription larger in scope than in France) and that didn't help either . Eventually a massive wall was built (ideologically neither party supported, which gave rise to opportunistic and most corrupt politicians) and as enough of the voters didnt approve with the results the army got approval for countless ground and air raids. The end result was a young generation filled with hatered torwards Israel, now finding other ways to attack.
My point is that to expect security forces to stop such attacks is reasonable up to a point. Escelation on one side builds escelation on the other and eventually its becomes an established norm. The problem Europe has is real, and difficult, tragic.
My heart is with those in Paris, France and Europe .
I cannot sleep.
Do you support only transitive aggregation operations? If so why not push the entire aggregation to elasticsearch/cadsandra?
How do you plan to scale cpu wise? ES and streaming engines (dont know cassandra) are cpu hogs (compared to map reduce).
I heard at devopsdays Tel Aviv that bigpanda decided to provide different sla to paying and non paying customers to balance the costs.
I recently read the BuronOut wikipedia page https://en.wikipedia.org/wiki/Occupational_burnout because I was having some thoughts about it myself. Burnout is a real problem in the software industry. With the exception of 2 months hiking leave (about five years ago), I never stopped working since after University (actually also during). I found that multiple short intensive vacations (long weekends 4-5 days), in times of stress, eases the tension a lot. It also depends how much support do you get at work (from you peers/manager, can you openly talk about it)? And how much support do you get at home from your immediate family.
A few things to remember - once you got a new job it is ok to negotiate the start date. Buy yourself enough time to recover, knowing that you are going back to work, but not now. Depending on your negotiation skills, if you can't get the salary you wanted, try to trade it with vacation days (so you can take more long weekends). Some burnouts could take a few months to recover from. Take your time.... Looking back at your career 10 years from now, these few months are very short, and who knows - you might get more offers during that time... going to meetups, messaging friends for better job offers, asking them about the stress levels in the place they work.
Well actually ... I have one. I worked at a big corporate that decided to create a link between SMS and IM for emerging markets, that was a few years after the iPhone was introduced. At some point we were 100 people, 5 dev groups (4 services and 1 infrastructure), another PM and QA group and a few more Ops. After 9 months the project was live, and had a problem with market acceptance (although there were focus groups showing it would catch with teens that dont have access to their IM before they go to sleep, or at school). The funny thing that a "pilot" version was ready after a month, written by 2 developers, that was almost feature complete. The joke was we should have shipped the pilot after a month.
I wonder if the SQL hold the insert traffic, because if it did... I would guess one could argue file->logstash->redis->ES would be overarchitecting it :)
SNS is a push notification service, that focuses on various consumers outside the cloud. It can be used with web hooks (http endpoints) to publish the same message to various web servers. In that case it would guarantees at least once semantics (which means the message is delivered, but sometimes twice), but it does not guarantee FIFO.
Our first architecture was indeed peer-to-peer where all components discovered each other. It is difficult to isolate failures on this kind of architecture. A reliable queue makes a big difference in zooming in on the root cause of the problem (a queue producer or a queue consumer problem).
Regarding latency - we are aiming for a few ms tops. preferably less. So bulk events using S3 was not considered
I always wondered if Amazon would really be open about and go public when a split brain in their network caused one of the hosted data services to loose data. Maybe they just throw money at the hardware and hope for the best.
Thanks! I'll update the post. A quick search found this RabbitMQ plugin https://github.com/rabbitmq/rabbitmq-priority-queue , which specifies: "In contrast to the AMQP spec, RabbitMQ queues by default do not support priorities. When creating priority queues using this plugin, you can specify as many priority levels as you like."
You are right about IronMQ, they are said to have a very strong product. For the record, we haven't actually checked redislabs and cloudamqp either, but since they are based on opensource that is mentioned in the blog post, I added them to the notes section. I couldn't say anything intelligent about IronMQ since it is not based on an opensource offering, and so other than actually trying it, I cannot tell its pros/cons.
Sure we have. Kinesis was considered because it was inspired by Kafka, and Kafka is known to work well. The thing is that any distributed stateful service, by design, requires DevOps experience and takes time to manage. We try to stay lean at this stage, and chose Kinesis over Kafka becuase it took very little time to setup, and has no "maintenance costs". Had we have chosen Kafka, we would have enjoyed much shorter latency times.
thanks. The Pros/Cons were supposed to refer to the complete architecture setup, and not to the Q component by itself. I can see how that could have been read that way...
We are currently not using nodejs streams. The nodejs components are the event dispatchers placing the events in a queue, they do not include much even processing. Most of the event processing is done in Apache Storm. We have also contributed the Storm-Nodejs integration, but the nodejs Storm components are not using streams either. I have tried using streams to compress the information between the Java parent process in Storm, and the nodejs child process, but compression wasn't a good tradeoff, so even that little bit didn't get into production.
We haven't done comparative throughput performance. SaaSiness, Low Latency and use case matching was the main concern of choosing the queues. As for throughput, the main concern was for the event stream processing pipeline. Redis and Kinesis are (somewhat) SaaSed on AWS, throughput can scale out horizontally with sharding, and we haven't had any problems since :)
The event stream processing is probably worthy of another blog post, but to be brief... we try to align all of our API servers (or event dispatchers) to be nodejs and have all of the processing in Storm (java). But reality is a bit more complicated. Specifically the API server pushing events into Kinesis is in python. It is a refactoring of some code that was written in the first days of the company. At the time it was based on multiple processes communicating through files... which didn't scale (as one would expect). The Kinesis implementation was done by a researcher which is a very talented coder but did not have experience with Kinesis. He was up and running after a short whiteboard session and a few hours of coding. We had to tweak a few things in the following days (such as adding more shards for higher read throughput, abandoning Kinesis shards altogether in favor of multiple stream with one shard)
Your are correct. The sentence you referenced is in the section that describes RabbitMQ without clustering. The next section talks about the tradeoffs of RabbitMQ clustering.
At Forter (www.Forter.com) a big part is the screening of CVs. HR and RnD work closely to contact only relevant candidates, and when we a CV could be relevant the RnD manager or the lead developer calls the candidate for 15 minutes focusing on a key strength that makes the candidate relevant and why working at Forter is so great. The first interview is about the candidate's comfort zone. You would be amazed how many people don't know what they were working on about the past few years... plus a hands on algorithmic coding task that does not require experience (we hire also coders before college). The rest of the process (usually three more interviews) are mixed. Include technical, personal discussion. We try to fit all interviews into one day. Even though Israel is small and travel is non issue , there is a strong psychological effect if the candidate gets an offer after one visit. This might require in some cases talking with the candidates recommenders while she is interviewed in the next room.