ktamura

>In particular, the ability to live-stream ("tail") logs seems to be a feature generally missing from logging aggregators.

If you are talking about tailing log files live, Fluentd has supported it from Day 1: http://docs.fluentd.org/articles/in_tail

Also, as other sibling comments mention, there are tools, both SaaS and open source, that you can use as a destination of the logs Fluentd tails/listens/collects.

* Elasticsearch: https://www.digitalocean.com/community/tutorials/elasticsear...

* Graylog: http://www.fluentd.org/guides/recipes/graylog2

* Scalyr: https://github.com/scalyr/scalyr-fluentd

* Loggly: https://www.loggly.com/blog/stream-filtering-loggly-fluentd/

* SumoLogic: https://gist.github.com/d-smith/8d3e7d53db772c6a7845

* Papertrail: https://github.com/docebo/fluent-plugin-remote-syslog

(and literally hundreds of others)

>though I rather wished Heka had taken off; it's much more flexible and in theory leaner and faster since it's Go

Heka was a great project, and a drop-in binary (as opposed to requiring a VM like Ruby) approach was interesting if not compelling in certain situations. That said, I never saw any benchmark that showed Heka was materially faster than Fluentd (or Logstash, for that matter). A lot of speed in this type of complex software comes from data structures/algorithms, an appropriate use of low-level language bindings, etc.

While language plays a role in the speed of software, it's hardly the only factor. As you said, it's only in theory, not in practice =)

I agree on the near-impossibility of causal ordering at scale. I don't see the need for a lot of "post processing". Adding filters in Fluentd is fairly simple: http://docs.fluentd.org/articles/filter-plugin-overview

Now that I understand the parent comment, I don't think this quite solves the problem he's asking unless all of your services are perfectly synchronized. No?

Ah, I completely misunderstood. Yes, you are right: Threading data together across multiple containers belonging to a single logical request is hard (and AFAIK, none of the container orchestration tools deal with it well).

While the article does not touch on this explicitly, Docker's Log Driver for Fluentd supports tags, which can be used to pass on container name/ID and other metadata to wherever logs are sent. Since Docker Machine itself allows these metadata to be passed on via API ({{.Name}}, etc.) one can create a fairly robust system to track log lineage.

There's been some comparisons in the past.

http://logz.io/blog/fluentd-logstash/

https://www.pandastrike.com/posts/20150807-fluentd-vs-logsta...

http://jasonwilder.com/blog/2013/11/19/fluentd-vs-logstash/

Some folks might like this: https://gist.github.com/kiyoto/4dbe676711e8e91659b02d72eced5...

It's definitely a good time to join or start one: Verticalized Consumer Packaged Goods startups with solid software infrastructure (hence the ability to power logistics/distribution/etc. with software) can undercut incumbents while maintaining a healthy margin.

I think the key phrase here is _verticalized_. Go-to-market as well as scaling operations is much easier when you can focus on a single product category.

This buyout is based on FUD, and FUD-based investment is a recipe for disaster.

From the OP:

>But for both Jet and Wal-Mart, Amazon’s frenzy of warehouse construction and fast delivery—as quickly as one-hour—have proved formidable. The retailer has logged three straight quarters of record profit while locking in an estimated 60 million members to its $99-per-year Prime service, cultivating a loyal customer base and giving consumers fewer reasons to shop at traditional stores.

The answer is not buying a unicorn startup with seemingly no fundamental advantage over Amazon. If I were Wal-Mart, I would take the $3B and try to build a great technology organization and/or fund Wal-Mart Labs more. So much of what makes Amazon better is the data accrued over the last 18 years and the insight mined from them by their super-talented team.

And Jet.com is definitely not going to give that to Wal-Mart overnight. If I were them, I'd focus on long-term viability, not a short-term hack to placate shareholders.

This makes sense.

1. Talent: As others say, an all star team. I've interacted with some of them as a vendor (Quip is our customer) and know a couple of them as friends of friends, and they have a rockstar team.

2. Integration: This is going to add collaborative document sharing to Salesforce's product line. Much like the Chatter acquisition to beef up their "collaborative, connected customers" story.

3. DATA: Essentially, Quip's metadata is the intra-organization graph of who collaborates with who. While Salesforce needs to be careful how they are going to use it (and definitely need to create the technical infrastructure to take advantage of this data), this is a deep well of customer knowledge that they can parlay into selling other Salesforce products.

The suggested approach most likely requires a lot of recursive backtracking. Of course, there's an efficient way to implement this, and that's what most commercial databases' path analytics features do. Here's one example by Oracle: https://docs.oracle.com/database/121/DWHSG/pattern.htm

I've always found it befuddling why so many developers want to use Solr/Elasticsearch for analytics heavylifting. It's probably because

1. SQL is not the most intuitive (although most pervasive) API for data analysis

2. Much of the data is already in Solr/Elasticsearch to make your data searchable/perform simple roll-ups and filtering, etc., so it'd be great if you can do more complex analytics against them as well

AS to why Solr/Elasticsearch is not ideal: the existence of superior alternatives that is OLAP databases.

>For funnel analysis, it’s not feasible to use this data model for getting back a summary of the funnel steps and the sessions matching it, since there’s no option in Solr to run a recursive query, which would allow to go over each session and check if it’s a match for the funnel.

I don't think this approach scales, even in an environment that supports recursive queries like PostgreSQL.

The more scalable approach would be to use either a commercial database systems with explicit support for pattern matching or encode conversion path as a string (ex: "top page -> product page with SKU=1337 -> Purchase" becomes "T_SKU1337_P") and use REGEX/GROUP BY.

In all cases, this sounds like a suboptimal use case for either Solr or Elasticsearch.

(I studied math and computer science. Walked out of a math Ph.D. program before starting one)

>I'm not a mathematician and was miserable at math in school, but I apply these habits in the business world every day. They help me cut through a lot of crap that comes from other people's sloppy/lazy thinking.

That's possible. The OP does not claim that the traits he listed apply if and only if you are a mathematician. It's certainly possible that you hone these skills without being mathematically inclined.

I do contest, however, that these habits apply to all "highly" rational people. I know plenty of people who are rational (or so are they deemed to be by their friends and colleagues) yet decidedly do not possess some of the listed habits, especially "scaling the ladder of abstraction" and "Being wrong often and admitting it".

On "being wrong often and admitting it," based on my experience, it's not that you become modest as a result of studying math. By studying math, you realize how prone your mind is to arriving at a faulty, unproven conclusion, and you learn to be more skeptical of your own thinking and become more at ease with being "wrong", allowing yourself to let your intellectual guard down.

>Side note, I'm surprised that Oracle is still running with Safra Catz and Mark Hurd as co-CEO's. I figured that that setup along with a very strong willed founder in Ellison, would cause some spectacular implosion at some point:)

It's because Safra Catz is a master operator while Hurd is a good IR and market-facing leader. Of course, Ellison is the visionary.

I read this as Oracle realizing that cloud is real and that they need to buy as many sources of cloud data silos as possible to build up their fortress of customer data.

>But I wonder what their clickthrough-rate is.

It's shockingly high, especially on Facebook Audience Network (FAN), Facebook's network of 3rd party publishers.

It was so high that I initially thought I was being defrauded. But upon closer inspection, it turns out that if you specify your industry/persona well in your ads, Facebook is incredibly efficient at matching demand with supply. I've never seen such good performance on any other mobile network, Google included.

FWIW, I've only done B2B marketing in my life. I have no idea for B2C folks.

>This article is more probably to convince ppl to use Google Cloud rather than how to run multicloud strategy.

Yes. For public cloud, it's AWS v. everyone else right now. This is why Docker is popular among IT mega-vendors because container portability gives IBM/Microsoft Azure/GCP/Red Hat a fighting chance to stop AWS from gobbling up the entire cloud infrastructure.

Container portability => less AWS lock-in => They'll come to my platform, perchance.

Some relevant articles: http://redmonk.com/jgovernor/2016/07/15/what-kubernetes-and-... and http://redmonk.com/jgovernor/2016/03/08/rise-of-the-docker-p...

I highly recommend Ellenberg's "How Not To Be Wrong". Ellenberg is a first-class mathematician and general expositor, and his book illustrates the power (and the limitation, in an endearing way) of mathematics as a tool of thought.

As a maintainer of the projec,t it's cool to see the Fluentd plugin in the repo: https://github.com/eventql/fluent-plugin-eventql

This is the opportunistic genius of Masayoshi "Masa" Son as one of the most successful technology investors of our time. He's taking advantage of the weak GBP thanks to the whole Brexit fiasco. As a major carrier in Japan and an owner of Sprint, this move makes total sense.

Previously on HN: https://news.ycombinator.com/item?id=8972066

As a marketer, I am all for launches, re-launches and marketing, but it's unclear to me what's different now compared to a year ago.

It's actually as amazing that Korea beat China. This is only the 3rd time it's happened since both started competing in 1987.

https://cl.ly/1M2O2t1h0p1n

Wow. To be fair though, USA has been a top contender at IMO for decades, including the legendary 1994 team (all Americans, including a future Googler and the creator of "Did you mean...?" search suggestion Noam Shazeer) that got perfect scores [1]

That said, it's true that many of USAMO/IMO top contenders are fairly recent immigrants, born overseas and moved to the US at a young age, but that just embodies the spirit of the United States: it's founded on a constant influx of immigrants and their desire to prove themselves.

[1] https://www.imo-official.org/country_individual_r.aspx?code=...

As an Acme user, I find the extensibility feature very interesting: Once you are used to Acme's contextual (and programmable) handling of text, it's hard to go back to a normal terminal.

The advice here is sage. If you run a platform-type SaaS business (not necessarily PaaS but more like SaaS with multiple functional use cases, which the OP calls the API business), make sure to read the OP.

To me, this one resonated best:

"APIs have a particularly tough marketing challenge in that the product can have a ton of varying use cases. You need different messaging to multiple customer segments each with their own use cases and needs."

In terms of "doing things that don't scale early on", this is _the_ thing API-as-a-Service companies ought to do. Your users won't build reference apps for you. You need to build them for your customers and make them very easy to use. This amounts to what enterprise software marketers call "solution selling."

Disclosure: I lead marketing at Treasure Data, an early Clearbit customer.

Missing from this article entirely is AWS's shrewd marketing and ability to define and grow a market.

As other commenters and the article noted, when AWS was launched, it just provided simple, primitive building blocks like compute (EC2) and object storage (S3). At the time, AWS's value proposition was entirely around elasticity of infrastructure and its archenemy was data centers. In the process, they created a new market that came to be called Infrastructure as a Service.

Since then, AWS has gone up the stack steadily, adding infrastructure application services like RDS, ELB, etc. These infrastructure applications have made it easier and faster to deploy applications on AWS, but unlike pure infrastructure, they come with a degree of vendor lock-in. But developers didn't blink at all because it was so much more convenient than upgrading your own MySQL or setting up an HAProxy.

And now, AWS is squarely in the applications business. Last year, they announced Quicksight, signaling to the world that they are going to squeeze out business intelligence vendors. Few know that Amazon has Microsoft Outlook/Gmail for Work competitor called WorkMail. Slowly but surely, AWS is trying to swallow the entire enterprise software stack. I noticed that most people no longer describe AWS as "IaaS" but as "cloud infrastructure." I.e., their scope and market is expanding.

Key to this success is their strategic prescience and marketing execution, both for the developers and the suits. They knew what made developers happy (free/cheap to start, great API and documentation, scales well) and took the time to build solid infrastructure. For the suits, they created a clear vision of "old v. new" and used customers like Netflix to convey how the new world is going to run on AWS.

As a keen observer and practitioner of enterprise software marketing, AWS's rise to dominance is a textbook example.

Hi there,

This was my source: http://www.recode.net/2015/10/22/11619964/google-has-its-hea...

The # is from Oct 2015, so it's entirely possible that my number is very out of date. Couldn't figure out how to amend my original comment, so adding this one as a substitute.

This is probably some kind of GCP PR's placed article, but regardless, Greene's impact on GCP has been incredible and highly visible.

Previously, the standard arguments that I've seen in sales conversations were that GCP wasn't enterprise ready or serious enough. All of that disappeared in the last 6 months.

Yes, GCP is still small compared to Google (or AWS) at $400M/yr revenue. Yes, AWS still "gets" enterprise better than GCP.

And yes, GCP is back in the cloud infrastructure game and will give AWS and Microsoft a run for their money.

>But if you ask the city's protesters, high school students and favela residents, “safety, peace, celebration and joy” are the last thing on their minds. There is little appetite for the Olympics, as seen in signs at occupied schools and community organizing in favelas in response to forced removals to make way for Olympics facilities. Police have increased violent armed raids in the favelas. Anti-impeachment activists have been met with pepper spray and arrests.

Olympic games don't bring sustainable prosperity and economic benefits, especially for the poor -and the poor knows it. As someone born in Tokyo, I am somewhat concerned about 2020 as well. Sure, Japan is not Brazil and Tokyo has no favelas. However, economic inequality is creeping up in Japan, the anticipation for the Olympics is driving rent and property prices, and most construction projects for the games will probably prove to be a waste of time and resources.

I have nothing against the Olympic Games. I think it's one of the most inspiring, unifying events that we have in a rapidly divided world. That said, the pros must be weighed carefully against the cons, and I am not very sure if it was a good idea for Rio in 2016 or Tokyo in 2020.

Developer marketing: marketing to developers. Marketing gets a bad reputation among developers, but it's vital for any technology to be marketed: keeping documentation up to date, answering questions for newcomers, explaining strengths/weaknesses, etc.

>Don't they have anything to gain from using Rust?

Who's "they"?