itaifrenkel

Amazon is forcing elastic to make money from their hosted offering, instead of their proprietary x-pack modules. Professional Services don’t scale as software do (compare AWS with Rackspace) - so I’m ignoring that income stream for now. Basically, they are trying to force Elastic into the AWS red ocean. Elastic, in response, would have to make a technology leap in order to best solve their customers problems. Examples include: Integrated data prediction, anomaly detections, clustering (online machine learning). Seamless multi region cluster deployment (with some consistency guarantee)...

OP here. AMA

Forter | Infrastructure Engineering Manager | Tel Aviv, Israel | Onsite, Full-Time | https://www.forter.com

Forter's fraud prevention decision as a service technology is way ahead of the competition. And we are investing in automation, infrastructure and security to keep it that way. For example, Forter requires data to be synced in multiple AWS regions in real-time while staying compliant with SOC2 and GDPR.

We are looking for an Infrastructure Engineering Manager that will lead a team of 3 additional Infrastructure Engineers (1 AWS expert, 1 security/infosec expert, a new hire and yourself). Your job would be to make sure we keep winning new customers by delivering our new cross-region solution, chaos monkey automation, negotiate technical RFPs and present our strong technical capabilities to customers, deliver secure microservice and data pipeline infrastructure. We expect this job to be 50% hands-on.

To learn more about our strong engineering team, check out our engineering blog: https://tech.forter.com

To learn more about your new manager, here is Oren's readme: https://managerreadme.com/readme/orenellenbogen

Our Tech Stack: AWS, Docker, Chef, Java, Python, Nodejs, ElasticSearch, Couchbase, MySQL, Redis ...

For any questions you might have contact itai(at)forter.com

This job requires an Israeli job permit.

The problem is that OLAP is run by analysts/business/data.science people and they may mess up with their queries/workloads. without isolation the production performance could impact the user expirience

How would you run spark on top of the production db without affecting its performance?

What I'm referring to is changing the partitioning dynamically by splitting streams, not just redis nodes. Here is one implementation example http://docs.aws.amazon.com/streams/latest/dev/kinesis-using-...

Doing it without server support is tricky.

The use case I'm referring to is when the client must be sharded to avoid CPU or network or disk bottlenecks.

The consumer groups proposal breaks the FIFO abstraction of a stream by allowing multiple clients to process a single stream.

Have you considered adding a semantic layer inside streams that allows each client to consume a substream? In effect the stream becomes multiplexed substreams.

If substreams makes the design too complex... have you considered server side stream 403 semantics? When a stream is manually deprecated it enters an immutable state and provides a redirect response with a link to another stream. This would allow multiplexing and demultiplexing streams without changing the client implementations too much.

For completeness I would state the obvious when fifo grouping is needed: 1. Scaling stateful event processing by splitting streams and adding more clients (CPU limit) 2. Scaling cross region replication by splitting streams and adding more tcp connections (network limit) 3. Handling more throughput by splitting a stream into two redis nodes (disk I/O limit)

Two comments on effectively once stream processing.

1. Consider adding an example for a stateful event stream processor client that saves the last read stream offset in redis, together with its current state and continues reading from that offset as an atomic operation. For example, a client that sums a stream of numbers, in order to have effectively once semantics would need to persist to redis the sum and offset together.

2. Consider adding a stream read deduplication example to mitigate clients that reinserted the same event twice. It is not clear how the client should behave if it didn't get an ack and it resents an event. What is the correct resending semantics so the reader would effectively dedup? What is the right data structure used to dedup message ids without consuming too much memory, etc...?

hi. Here is my attempt to clarify the domains section. Could you please comment on PR https://github.com/forter/security-101-for-saas-startups/pul... ?

hi. Here is my attempt to clarify this section. Could you please comment on PR https://github.com/forter/security-101-for-saas-startups/pul... ?

Please comment on https://github.com/forter/security-101-for-saas-startups/pul...

There is another issue, that US citizens ussually do not encounter. When you setup a VPN on amazon, for example, you would like only some of the traffic (intranet) to go through that VPN (or SSH socks proxy). The rest of the traffic should go directly to the internet. The reason being is that the roundtrip is too costly.

I am not sure configuring these settings are trivial, and VPN clients provide that out-of-the-box.

SSO is not enough. In secure systems you are ussually required to provide both network access, and applicative access restrictions

And do you find that non-techies can also handle this? Also, I think this won't cover DNS changes that a vpn client does

Same question. Do you refer to using SSH and socks proxy. If so, is that a viable solution, in your experience, with less techy employees?

Do you mean that you use it as bastion host?

After sleeping on it... I am not sure it would work with managers. They work with outside council on a day2day basis and they use attachments for that. I wonder why gmail hasn't made phishing attachments obsolete.

Do you use ssh as a socks proxy?

SSH does not cover all use cases.

One example if you have an internal web service, how would you restrict access only to employees (without having it open to the internet?). SSO is not enough since you want the ports closed to non employees.

Another example is accessing a database that is not configured with SSL. You don't want your info travelling in plaintext on the internet.

Are there any best practices you would recommend?

I am not sure that binaries are the use case for g-docs.

I would have used s3 for that.

My understanding was that SPF and DKIM are only for sending emails.

I need to rearrange that paragraph and add the 2 other reasons you mentioned, and you are right that outgoing emails are better sent from subdomains.

I do wonder though if a spam filter blocks x.d.com would it also block emails from d.com?

USB flash drives is the term used in the US, right?

us-west - perhaps I should change it to don't use us-east-1, since it fails much more often and is more crowded.

I'll rephrase the git and 2fa.

Endpoint Security - you gotta have it. I understand the natural objections, but there is no certification that doesn't ask about it. There are the more expensive ones like cyberreason or carbonblack.

I need to research more the domains issue. I suppose it's more prevalent in Israel since some devops in Israel worked for gaming (gambling) companies where they definitely use multiple domains for multiple purposes. But I think the main reason is allowing devs more management access to internal subdomains and disallowing management access to the API endpoint domains that customers use, to reduce attack surface.

Thanks for your comments. Keep them coming

And... Apparently after enough time, people come to expect slack to be asynchronous too.

Ahhh - now I get it. Well our marketing staff are mostly native English speakers. I'm not :)

I might need to find a better example.... Do you have an example of a process that is not needed in the beginning, and needed one year down the road that is less obvious?

I know. We use emails for some meeting summary. But you can't ignore the fact that many attacks start with phishing, and slack is free from that problem

Older employees, and some managers, I found prefer email in many cases. I still request them to use slack

Can it be done with hosted mail solutions like most startups use? (Gmail/office365)