Palantir acquires Kimono
kimonolabs.com
kimonolabs.com
As a team, we’re very proud of the product we built, now used by over 125k developers, data scientists and businesses" ... from FAQ: "service will shut down Feb 29, 2016"
In fact, they are so proud, they're giving all but ~2 weeks notice to said 125K developers, data scientists, and businesses to migrate off the platform? That is an abysmally small amount of time and if I was a paying customer, i'd be furious.
On the other hand, congrats to Kimono team!
Palantir is the poster child for black box software company that keeps trying to sell you more expensive licenses for their mystery algorithm. I know they've had some success in the government sector but I've seen them make a real mess of things in the private sector. At my last company their system just became a giant paracite where data went in and stuff came out the other end but it was entirely unclear what happened in between. The more we looked into the mystery middle ground the uglier it got.
Modern tools like spark and mesos in theory let you do what palantir does and then some but it assumes your internal IT team is semi-competent which more often than not is not a correct assumption.
We at CloudScrape.com are happy to welcome you onboard our platform - you get 20 hours of scraping runtime for free and the ability to navigate much more complex websites, opening up tons more opportunities then you've had previously. And we're here to stay :)
Kind regards, Henrik Hofmeister, CTO, Co-Founder @ CloudScrape.com
My guess, they weren't commercially successful, and aquihire was the best way out.
Why? Money.
The tone of the announcement is in bad taste given the bad news for their customers.
When we evaluated options a year ago, kimono was the best fit. We'd happily have paid, and did ask. I don't understand what happened... Did they run out of funding?
I wish people picked their jobs to match their ethics instead of the other way around. It makes me sad when I hear about my friends going to intern for this place.
The pay actually isn't actually particularly high for SV standards. I think Facebook outpays them, for example - and Facebook is less selective than Palantir. But their reputation is that they're full of top engineers and you'll learn a ton there, and that's why the people I know work/worked there.
The downsides I've heard about are no work/life balance and fratty culture. Those are real downside, not the FUD about ethics.
Well, more generally, anyone who has a nonstandard set of ethical beliefs, and is basing their company around that set of beliefs, can't really go public without their belief-set being thrown under the bus of their shareholders' belief-set.
For example, I've often considered starting a company in the game industry that actually hires experienced adults and treats them well, rather than hiring fresh college students and burning them out. I imagine, though, that if my hypothetical company became publicly-traded, I'd be forced to relinquish this policy in the name of short-sighted market competitiveness.
Capitalism wins, eventually.
Just rail on the hypocrisy if confronted.
Apple and Uber are hiring like crazy and so it's quite a lot easier to get in there as an intern than at Palantir (which is more selective).
Interning as a prison ward in Auschwitz is a pretty big resume boost for anyone who served in a prison system, and Nazis have one of the highest paying internships abroad. There's some serious benefits of becoming a prison ward in Auschwitz.
That's how I see this parasite company.
https://en.m.wikipedia.org/wiki/Palantir_Technologies#Contro...
Also:
> When intruders from the hacker group Anonymous gained access to thousands of emails stored on the servers of the security firm HB Gary Federal, the emails revealed that Palantir had worked with HB Gary Federal to develop proposals for attacking WikiLeaks’ infrastructure, blackmailing its supporters and identifying donors.
http://www.forbes.com/sites/andygreenberg/2011/02/09/did-sec...
There was also the sex scandal with their founder and a Stanford student: http://www.nytimes.com/2015/02/15/magazine/the-stanford-unde...
This thread on HN also has some discussion about their government and military contracts:
Also, your quote doesn't appear in that article at all any more.
http://www.forbes.com/sites/andygreenberg/2013/06/07/startup...
Second to last paragraph.
As for the sex accusations, who knows what really happened. Though the article doesn't exactly paint him- edit- either of them- in a flattering light. It does show poor judgement for the dude to get involved with a student he is mentoring. Then again, he also stumped on the campaign trail for Rand Paul of all people and also pals around with Mitt Romney.
However, data mining for target leads (e.g. through cell phone, financial, social network + other data) would be much more up their alley.
I won't share specifics (as they aren't mine to share), but I have a friend working for Palantir overseas. Once essential benefits (e.g. accommodation) are factored in, his salary is north of six figures (GBP). For someone only a few years past university, I imagine that sort of offer is difficult to refuse.
http://apifier.com (favs)
https://github.com/scrapinghub/portia/ (our open source equivalent of Kimonolabs)
http://scrapy.org (allows to scrape more complex sites; also OSS)
http://scrapinghub.com (for a cloud-based platform and a smart proxy rotator)
(Disclaimer: working there.)
More seriously though, it's basically a bargain. You get:
- Access to the largest team of Scrapy, Portia, and Frontera experts around at your fingertip.
- AWS functionality for actually less than what it would cost you to set things up yourself (think VBR IP over ATM).
- Smart proxy rotating tech at your fingertip with tens of thousands of IP addresses spread across a slew of different IP networks.
- Built-in integrations with a growing list of third parties.
The list could go on and on... It's definitely not pricey. :-)
For folks looking for an alternative, check out http://www.parsehub.com
ParseHub is especially good at dealing with dynamic websites and multidimensional data (arbitrary relationships instead of just rows and columns).
Our team would be happy to help you migrate from Kimono. We're bootstrapped and profitable, so we don't have pressure to sell when an offer comes along. We also pledged to release the back-end code under a liberal open source license if we were ever in a position where we could no longer provide service.
https://www.quora.com/Which-are-some-of-the-best-web-data-sc...
The target demographic was developers who were too lazy to write simple scrapers for a given predefined use case. However, scraping most modern websites is not simple, and requires a lot of code to ensure the correct HTML elements are scraped and the data is transformed correctly...which defeats the purpose of using Kimono.
Kimono joining Palantir was likely an acquihire, especially since the core service was permissively free.
An actual parser for the complex use-cases requires some custom code and isn't covered by what Kimono does.
What Kimono does can trivially be achieved by writing 15 lines of python (after importing httplib2 + BeautifulSoup). Which is even reusable.
The only real use-case they had was the visual interface, so that "non-programmers can do it. No code required, etc. etc." Heh. As if that ever did, or ever could work.
translating for the Palantir's business - "a DHS/FBI/CIA analyst can dot it. No code required" Easy scraping websites for terms like "subversion", "Arab", "airplane" (cue starting scenes from Harold & Kumar 2) ...
You know what else has a visual interface? Excel. It works great! For "trivial problems." It gets stretched all the time into places where programming would have been faster, but for the "trivial problems", just being able to have a domain-expert non-programmer use Excel to implement the solution directly, is a lot faster and simpler than calling in a programmer and attempting to fully specify the problem to them.
For example, I hit an issue when using Kimono where it refused to accept a given regular expression as valid, even after trying testing many variations which passed a regex checker. If I had just started with Python/BeautifulSoup, I could have completed the entire scraper in that time.
I put the HN front page through their Discussion API Test Drive...and it identifies submission elements except submission score and plain submission URL. Which is a problem.
All of our automatic APIs aren't meant to be used on directories or listings pages (for that, we provide a crawler which gathers those links and then feeds them into our automatic APIs). The discussion API is intended to be used on discussion pages, for instance: http://www.diffbot.com/testdrive/?api=discussion&url=https%3...
If you wanted just a list of submissions and score, you could use our custom API toolkit(similar to Kimono): http://www.diffbot.com/products/custom/
http://www.stockhouse.com/companies/bullboard/t.y/yellow-med...
Is this the type of task DiffBot could theoretically accomplish? (Running it through the API test drive suggests no.) Are there any other tools that might work, or is this a problem that requires hand-coding?
If you'd like me to set you up with a demo, feel free to email me: dru@diffbot.com
Does anyone have any suggestions on how to run something similar on my own server?
[1] https://github.com/scrapinghub/portia [2] https://scrapinghub.com
I'm not sure how to put this, but I've used Palantir's products in a couple different warzones. I can't imagine they're not pointing those tools at everyday Americans. In fact I'm certain of it (just read FedBizOpps postings from the DHS and DoD). I'm sure most of the people working there have no idea, but I'm also sure there are some truly bad people working for that company. Man, we really need to regulate this a lot better.
http://www.dhs.gov/state-and-major-urban-area-fusion-centers
Powered by Palantir, basically it is real Big Brother brain network where all the information like lic. plate readers, video cam feeds, cell phone data, Internet data and everything else is "fused" together. One of the immediate resulting products of it is generation of Suspicious Activity Reports. Hoover (and Beria) would die of envy.
Could someone give an example?
Why is access to these most important problems limited to Palantir, or otherwise not accessible to most people (as implied by this shutdown notice)?
* shameless plug * For longer than most such tools, we've been offering what our customers call the "easiest scraper of 'em all": https://feedity.com Try it out and see if it works for your needs.
https://www.kimonolabs.com/faq#faq-303
> Will Palantir (or anyone else) have access to my personal data?
No. All of your user data including names, email addresses, passwords and APIs will not be shared with anyone and will be securely purged from our servers on March 31, 2016.
Uh...but what about the scraped data? I don't think I've used Kimono since it was first announced on HN as a side project...but I'm assuming that you got to save scraped data semi-privately with Kimono? Is that considered "user data"? Kind of a vague term, and one I usually associate with data of the personal information variety. I'm sure someone at Palantir can find some interesting meta-insights from analyzing the data collected by the kind of people (data scientists, etc) who would use Kimono heavily.
Here's a tutorial all about running a scraper on morph.io https://www.openaustraliafoundation.org.au/2015/10/13/ruby-w...
Technically talented employees receive a pittance and are sold into the Palantir machine, a morally unscrupulous high-tech consulting shop that takes in talented engineers and churns out one-off software black boxes designed to lock in customers and extract a long-term tax.
Will Palantir (or anyone else) have access to my personal data? No. All of your user data including names, email addresses, passwords and APIs will not be shared with anyone and will be securely purged from our servers on March 31, 2016.
Thanks for nothing guys!
I don't quite get why startups are not trying to generate revenue from day one.
There is not a download function on the desktop page, what is the direct download link? https://www.kimonolabs.com/desktop
basically its like hiring a bunch of regex gods because you have regex problems on large scale
edit: why the downvotes seriously.
The place is likely to be very shitty to work for but by suggesting that they might be doing something that is James Bond equivalent in software world might make it sound too cool for some developers.