HNHacker News
TopNewBestAskShowJobs

mekarpeles

1,082 karma · joined March 10, 2011

Hi, I'm Mek (https://mek.fyi). I run OpenLibrary.org over at the Internet Archive.

I'm an entrepreneur, an advocate of non-profits, 2022 & 2023 Berkman Klein Center affiliate, and a PhD dropout. I helped run tech for Hyperink (YC W11) and Ark (YC W11) and co-founded Hackerlist.net + Baybo.

I am passionate about non-profits, open source software, GNU/Linux, teaching and education, and spreading knowledge across the lands.

My mission is to enable others and to curate a living map of the world's knowledge.

submissionscomments
mekarpeles··on Internet Archive raises National Emergency Library: 1M free books no waitlist
Thanks @sayhar! specially given so many libraries across the country are currently shut down.
mekarpeles··on Better World Books and the Internet Archive Unite to Preserve Millions of Books
This question appears in https://controlleddigitallending.org/faq. Specifically for our sponsorship program at this stage, we decided to limit eligible books to those we had no copies of (in the interest of maximizing preservation -- we're as much an archive as we are a library).

In general we're not thrilled about books being destroyed. There's always someone who can benefit from a book. Also, one never knows when a book may have to be scanned again because a digital copy could theoretically become lost, corrupt, or benefit from new tech (e.g. a palimpsest). These are all big reasons our digitization process is explicitly non-destructive.

http://openlibraries.online describes how we are working with other Library partners and including them in our model to help make works more broadly available.

None of this directly answers your question which is probably better answered by a lawyer. I'm just a fake librarian.

mekarpeles··on Better World Books and the Internet Archive Unite to Preserve Millions of Books
Mek here from Internet Archive's OpenLibrary.org. We've thought carefully about this as well.

We feel it's critical that Internet Archive and Better World Books aren't the only organizations involved in determining which books should be preserved.

In late October, we launched a new program which allows anyone in the world to take control of our Library's shelves and contribute/sponsor the books which reflect their values: https://openlibrary.org/sponsorship

So far we've seen an influx of high quality reference books come in (everything from African antiquity to Microbial biology) that we almost certainly wouldn't have procured ourselves based solely on the data available from e.g. library holdings, wikipedia citations, etc.

Having Better World Books as a partner has been incredibly helpful because we're able to calculate sponsorship prices quite accuracy (because shipping prices are included) which was a big challenge with other APIs we previously used. Also importantly, Better World Books helps our patrons sponsor older books which are often more at risk of disappearing, which is one (of many) useful heuristics when considering preservation.

mekarpeles··on Better World Books and the Internet Archive Unite to Preserve Millions of Books
mek here from Internet Archive's openlibrary.org project. We've been in broad talks w/ folks like OpenAI about how the contents of texts may be used to power better discovery and to increase usefulness of books. Open Library is pretty far from GPT-2, but we do have fulltext search across ~3.5M books: http://openlibrary.org/search/inside

We're also an open source project [https://github.com/internetarchive/openlibrary] and happy to collaborate w/ folks on such projects. I'm personally very inspired by the https://techcrunch.com/2014/07/25/apple-booklamp/ Booklamp project; building a genome for every book and surfacing as much content as we can to help patrons discover citations, quotes, and other useful content which would inform their reading choices and otherwise be completely inaccessible behind a borrow.

If anyone is interested in helping us move the needle on such an effort, please do get in touch and we'd be glad to invite you to Open Library's slack channel.

mekarpeles··on Wayback Machine director outlines the scale of everyone's favorite archive
- foundations + government grants https://archive.org/post/2776/who-funds-this

- donations (in-cash / in-kind) https://archive.org/about/credits.php

- digitization services https://archive.org/scanning

- web archiving services https://archive-it.org

src: I work @ Internet Archive on https://openlibrary.org

mekarpeles··on July 12th: Internet-Wide Day of Action to Save Net Neutrality
Why did you lose interest? Both SOPA and Docket 17-108 fundamentally address freedom of speech on the Internet.

The outcome of the SOPA fight was the 2015 Open Internet Order (which ostensibly treats the Internet as a public utility and protects against both government and corporate attempts at censorship). Docket 17-108 (what this is about) reverses the 2015 outcome and gives ISPs discretion to regulate the web and slow down or block companies with competing services, which is what they did in 2011 and a reason why the Open Internet Order was crafted[1].

[1] http://www.businessinsider.com/verizon-blocking-google-walle...

mekarpeles··on Science Isn’t Broken (2015)
If I understand, the author is suggesting something like pagerank for axiomatic correctness where a paper's score can be interpreted per paper (in terms of some binary promise of "correctness") even if the score is computed by the weighted correctness of the papers it represents.

Really, the information should be abstracted away from the paper, a la freebase or wikidata, and there should be strict rules (e.g. the protocols stackoverflow enforces) such that no cycles can occur, such that propositions can be resolved to axioms, and such that it is practical to compute scores in a finite period of time with reasonable resource constraints...

See also gitxiv.org (at attempt at literate research for computer science and physics papers that can be paired with code). It doesn't address the problem described, but if statements of a paper are codified (e.g. curry howard isomorphism) in a strict way, it makes it easier for "linked statements" to have a quantifiable value.

mekarpeles··on Science Isn’t Broken (2015)
P.S. One way you might describe this project is "an axiomatic Euclid's Elements for all scholarly knowledge".

You might also like metacademy.org (one of my favourite projects -- it's a package manager for knowledge built on DAGs)

mekarpeles··on Science Isn’t Broken (2015)
@averagewall, check out openknowledgemaps.org

Also, citeseerx and crossref are your friends.

Pubmed central similarly has the ability to return much of this citation data programatically (for certain scientific subdomains)

If you're passionate about prototyping such a service, let me know and we can get you an invite to the Archive Labs slack (we're a non-profit, volunteer-run community which incubates open access and public good projects). Literate, reproducible research and knowledge + citation graphs are two of our emphases.

Good luck

mekarpeles··on Internet Archive is suffering from a DDoS attack
In the same vein, the Internet Archive recently hostly the Decentralized Web Summit in SF (http://decentralizedweb.net).
mekarpeles··on Ask HN: What should we fund at YC Research?
+1 for YC Research's basic income (and standard for quality of life)

Meta-Research. +1 for @mietek's Meta-research; creating a distributed, open ecosystem (not a website, but a protocol) for publishing and navigating interdisciplinary research.

Medicine. +1 for Bacteriophages @mietek & open-medicine (e.g. Counter Culture Labs - Open Insulin) https://experiment.com/projects/open-insulin. Additionally, cancer and heart disease research.

A distributed, persistent, interoperable world wide web. (e.g. IPFS)

A sensible distributed DOI and identity system for people.

A meta exploration to determine the world's hard problems and impending catastrophies (e.g. www.metaculus.com)

Alternative energy and renewable energy.

mekarpeles··on A website for testing your app ideas
There should be a way to suggest/tag an existing solution (to generate conversation -- why is their solution better?)

This seems like a good way to help people collaborate and not re-invent the wheel, unless there's a compelling reason / competitive advantage / unique value proposition.

mekarpeles··on Link rot and redirect madness
For anyone interested, the Internet Archive has a small but tight open source community called Archive Labs <archivelab.org> (all experimental services)

We're working on several initiatives, like bringing OpenAnnotations to the wayback machine, reducing link rot (see: vinay goel & http://vinay.blog.archive.org/2014/12/17/9/), universalizing access to open-access publications, and supporting efforts to distribute and decentralize the web.

Send me an email if you're interested in checking out our slack channel and meeting fellow contributors. Also, if you have a public-good or open-access project you would like help with or resources for, we'd love to hear about it --

mekarpeles@gmail.com

mekarpeles··on The Lack of Open Tooling for FPGAs
@diamondman responded to this in another thread: http://www.reddit.com/r/programming/comments/338t25/the_lack...

"I actually want to hook OpenOCD into my Daemon to bring some sanity to how it manages devices. Several of my IRC friends prefer using closed source tools to open OCD calling open Obsessive Compulsive Disorder, frustrated the license is so strict (mine will be LGPL), and annoyed that everything has to be exhaustively specified in TCL files for it to do ANYTHING."

The TL;DR is OpenOCD's driver model itself is fixed/limited/standardized to a degree which prevented him from, in many cases, optimizing underlying controllers (or getting them to work at all -- e.g. OpenJTag, I believe). Also, see licensing disagreements above + unreasonable configuration requirements outlined in @diamondman's answer above.

mekarpeles··on The Lack of Open Tooling for FPGAs
About @diamondman's Adapt Framework (github.com/diamondman/adapt)

Adapt is an open, modular framework which offers a streamlined approach for JTAG controllers to speak to target devices (currently CPLDs and soon fpgas).

Adapt is built to be extensible and currently includes open/reversed drivers for the following Controllers and target devices:

- Digilent & Xilinx PC1 (JTag Controllers) & XC2C-256 (CPLD)*

*Note: Map files are required for CPLDs to avoid legal recourse

FEATURE ROADMAP:

1. Support for Controllers: Currently Adapt only supports JTag but @diamondman intends to support dbw & spi. Patches for other serial protocols are welcome.

2. Support for Target Devices: The next milestone is to support Spartan3 & Spartan6 fpgas

3. Extending Adapt / Contributing Drivers: Adapt can be extended to additional controllers and target devices -- @diamondman will write a tutorial on how this is done + what the limitations are, if there is enough interest.

Please express interest in driver support by replying to this comment thread.

mekarpeles··on Ask HN: How do you find good developers on oDesk?
MortenK, these are great pieces of feedback. It's disappointing that job marketplaces like oDesk/elance don't incorporate these heuristics into their process in attempt to raise the quality bar. In terms of better developers, there are a few companies like toptal (high quality temp placement) grouptalent, and ooomf, etc.

My experience (co-managing ~$300k on oDesk) is communication / expectation alignment is one of the biggest reasons a project fails. Project managers, for example, don't know how to vet computer vision experts. Screen sharing is good for seeing that someone is typing something or "appearing" to do work, but most PMs have no idea if the researcher is programming a canny line detector versus AAM. More importantly, just because they know what they want to build doesn't mean they have experience or knowledge to manage an application lifecycle.

Really, the PM should be able to tap into a software pipeline which compliments/augments the expert's natural workflow. Source code management should be a requirement, enforced by the platform, not something one has to vet for. Roadmapping and proposals should be a paid deliverable and a discrete step within the process. And I don't think continuous integration + test cases should be the exception, they should be the norm. Using such a pipeline is self-selecting as programmers who don't understand these concepts will easily be discovered.

Sorry if this is a shameless plug, this is something we've been addressing for a while at Hackerlist.net (we're in private beta). We've open sourced a good potion of our stack in case others want to use these tools to manage their own software development projects.

To support to your list of tips, MortenK, we've also found github to be a reasonable selector. If a SWE has a github account with several projects, this at least demonstrates rudimentary source code management understanding and an interest in open source. It's also an opportunity to review code. We ended up designing an internal search engine for handling discovery / evaluation as vetting great hackers at scale is a big challenge.

mekarpeles··on Freelance or job?
Consider creative ways of finding fellow freelancers and partner with them to share relevant opportunities. Ideally, partner with people you respect as being equally smart and driven. We are often judged by our friends.
mekarpeles··on Freelance or job?
3 weeks without work is a nice feature so long as one has options/opportunities to work.

It is possible to earn a professional livable wage on Elance, however the professions for which freelancers are being employed via Elance are frequently menial / not complex software projects (and thus the wages are closer to $5-50/h as opposed to $100+). I am unable to find more than 11 jobs on Elance in IT that pay > $150/h. I receive about that many requests from companies by email weekly.

I don't believe Elance, as a platform, is conducive/setup to address involved problems which require rigorous testing & best practices to begin with. What Elance does well is aggregates supply and demand and provides a method of finding work.

I believe the best option is to build a network / partner with other agencies who have different skill sets and refer opportunities within your network. This is how we've been addressing the problem of continuous opportunities with Hackerlist. We partner with tons of independent consultancies with domain specific expertise and forward them work.

Also, freelancing picks up over time. Almost every client we've worked with has turned into a return customer (which we're really grateful for).

mekarpeles··on Freelance or job?
There are a lot of premium / hybrid solutions out there which may offer you the balance and higher price points you're looking for. Toptal (a16z) does temporary placements and is one such hybrid model.

There's also gun.io (actually, think they pivoted slightly), ooomf, hackerlist.net, 10x, etc.

Disclaimer: I am a founder @ Hackerlist.

Having run and advised consultancies for a while, and having been an employed engineer, I'm happy to discuss the tradeoffs of each in detail, assuming there's interest. In terms of getting clients, I do think it helps working with another organization to increase your intake, even (especially) if you run a small consultancy/agency.

Alternatively, happy to answer questions or help make intros around SF if people prefer personal interactions <m@hackerlist.net>

mekarpeles··on Bram Moolenaar responds to Neovim
To a degree, I think these distinctions both support and are a logical consequence of haberman's argument (please correct me if I'm wrong). As computing systems evolves, we learn more about how to address their constraints and how these constraints impact how users choose/desire to interface with their programs. Vim is publicly ~22 years old. During it's maturation, many computing milestones have been met which overcome many of the original assumptions or challenges which prevented us from using some of the interfaces we prefer today. Async (re: MarcWebber) being one such example.

I believe tmux / screen is a beautiful example, given Screen has a lot of black magic and has a very challenging codebase to work with. A lot of the philosophy built into tmux are cumulative learnings deduced from what was done correctly or incorrectly within screen. This materializes as features missing (or implemented with different intent) from tmux that exist in screen -- the philosophy behind sessions is one such big underlying principle change.

I like to think of this as similar to using a genetic algorithm to evolve an overly complex/convoluted solution over a period of time (based on some assumptions which may no longer even be applicable), and then explicitly defining/reconstructing a comparable solution using more deterministic and intentional approaches. The focus of these two development cycles are fundamentally different and perhaps neither wrong (the former attempts to discover novel ways of addressing a problem whose solution is unknown, the latter prunes this known search space).

mekarpeles··on Rejected from YC? Here is what you're missing out on
Definitely need it if you're living there... Fellow connectican
mekarpeles··on What Long Hours Really Mean
During my first startup, I worked 6 or 7 day weeks, 13+ hours a day. At Hyperink, in my first year, I (on my own accord) slept in the office over 30 days. During this entire stretch, I was perhaps more productive and happier than any other time I can remember. The main reason why? The teams and the comradery.

I wish this article at least had relevant studies + statistics on this subject as it is not representative of any of the 4 startup experiences I've been involved with. Actually, in diametric opposition to this author's stance, our teams' cultures were united and strenghtened around the concept of working hard (which, in our case, happened to at least partly equate to long hours).

I should be fair and admit, it may be the case that this article is more relevant to later stage companies -- I don't have a great amount of experience at 100+ employee companies.

From my experience, when one loves what one works on, one often works on said thing a lot. Some of the hardest working people I know work (addictively) towards projects they're not (necessarily) paid to work on. For instance, one of my teammates, Mischief, spends about 5 hours a day porting packages over to Plan9 and writing terminal emulators in go (why not).

Im not so enlightened that I can draw conclusions about people who _don't_ work long hours, but it seems intuitive that people who do work long hours fall into one of two binary classes: (1) they are forced to (in order to achieve some level of compensation) / fear layoff, (2) they like the product, team, or like working hard. That is, intrinsically one either wants to work hard or they don't want to.

You want people in the latter category, and you achieve this by being fair with equity, providing people opportunities to have an impact or work on what they love, by providing a supporting environment, and by having good hiring practices.

There will always be great people who love your team / vision / culture / management (un)structure and who physically cannot commit to 70 hour work weeks (family, etc). You and your team have to decide whether it makes sense for this person to be part of the team and be prepared to make adjustments to make it work.

What I wish this article was about:

* How teams can (and should learn to) effectively prioritize teammates' health

* Why pressure to work hard can be destructive

* How to programmatically or systematically eliminate or amortize stressful tech/scaling/hr/bd problems to avoid disasters.

[edit: fixing typos]

mekarpeles··on Choc: Bret Victor's 'Learnable Programming' implemented in Javascript
Nate and Ari know what they're doing, great guys and great programmers.
mekarpeles··on Hackerlist: 1st Batch (last call)
Ha, thanks Jim! Probably need a: "from hashlib import sha256" in there too!
mekarpeles··on Hackerlist: 1st Batch (last call)
We appreciate your kind words, Ozzie -- Happy hacking!
mekarpeles··on ODesk vs Elance vs Freelancer vs Guru
That's a really important skill to have -- the ability to assess social skills as well as technical prowess.

I find trello pretty effective for daily goals. In terms of a mature bug tracking system, I'm a fan of phabricator. For invoicing, we use envois (https://github.com/lambdal/envois).

Agreed, communication and expectation alignment are critical components of avoiding "never-ending contract syndrome"

mekarpeles··on ODesk vs Elance vs Freelancer vs Guru
Thank you for your questions mjhea0. Long term, as you guessed, we're building something different from a job board -- we're architecting an ecosystem where talented engineers can make a living completing missions online, with version control, test cases, and continuous integration. We imagine this looking like a hybrid between github and odesk but it's ultimately the hackers we work with that will shape this experience.

One of our objectives is to bring the certainty and assurance that top companies enjoy with full time staff, to the talented, nomadic hackers of the world, while preserving hackers' needs for flexibility, getting paid what they deserve, and supporting best practices (unless you're a fan of submitting that .zip file on odesk -- nothing hurts me more).

To answer your question about retaining talent: One of the reasons I seek ephemeral opportunities (and I suspect there are others like me) is I appreciate flexibility; working on various projects which interest or challenge me, with different teams, on a schedule which works for me. I don't anticipate 'solving' the problem of attrition directly as much as I do finding creative, safe ways of working around hackers' preferences to achieve the most beneficial outcome for all parties involved. This includes experimenting with strategies like structuring missions around teams of hackers to mitigate risk.

That said, there are several things we're doing for our hackers to be deserving of their loyalty. First and foremost, we don't charge our hackers any fees -- all of our fees are client facing. Why? I complete missions on behalf of Hackerlist, I don't want fees and I don't think it's right to hold more talented technologists to a different standard. Second, we're giving equity in Hackerlist to the top 4 performing hackers in our first batch. Finally, we have a team of mentors available for advice and guidance.

In terms of finding and vetting experts: We have a board of mentors with specialties in different areas who are assisting in the screening process. I also bring my own expertise, being a 2-time founder, having a partial PhD in AI/NLP and having led technology (and a good deal of technical recruiting) efforts for two YC companies (Hyperink, Ark).

It's worth nothing, we're only accepting 2^4 applicants for our first official batch (so far there's 700+ applications) specifically so we can address and tackle these quality issues head-on and provide our hackers the best experience possible.

TL;DR - no fees to hackers, they keep all their earnings - high price points - we handle invoicing + vetting (clients & hackers) - we have a strong network of mentors to control quality - we're good to our hackers and aim to build a custom experience around their needs.

best wishes + happy hacking

mekarpeles··on ODesk vs Elance vs Freelancer vs Guru
Understood, thank you. We're in the process of modifying this now.
mekarpeles··on ODesk vs Elance vs Freelancer vs Guru
elbear, no, there's no specific reason. We're using an arbitrary regex, 4-32 chars, alphanumerics and a set of specials including: !@#$%^&+=_ Let me know which characters you feel are appropriate and we'll happily consider implementing your recommendation. If you have any additional feedback, feel free to reach out to me personally, m<at>hackerlist.net, tel: 415.690.8033. Ultimately, we're building hackerlist for hackers, so we take your feedback very seriously.

best wishes + happy hacking

mekarpeles··on ODesk vs Elance vs Freelancer vs Guru
I agree and finding great developers is a big part of the challenge. However, there's more to arranging a project then finding the right talent, especially for clients who have expectations for security, domain expertise, and quality assurance. Most platforms simply solve the matchmaking component (eharmony style). The solution is a platform which supports engineers' workflows (git, hg, etc) -- or at least this continues to be our philosophy and mission as we grow Hackerlist.net.

mjhea0, how do you typically structure projects after you've found someone who is a good fit? Do you use any additional services like invoicing, github/bitbucket, bug tracking, etc?

← PreviousPage 3 of 6Next →