HNHacker News
TopNewBestAskShowJobs

danpat

599 karma · joined February 25, 2011

contact me at danpat@danpat.net

[ my public key: https://keybase.io/danpat; my proof: https://keybase.io/danpat/sigs/s-kP3xrOGFw_4IURoo3qLHJOo0fWeJVMnGiU6vsgzpM ]

submissionscomments
danpat··on Equifax statement regarding extent of security incident announced Sep 7 2017
I'm typically not particularly paranoid, but if a state actor wanted to destabilize the US economy, increasing the rate of fraud via identity theft sounds like a great way to do it. Crank it up just enough to hurt the economy, but not quite enough to break the identity status quo....

The leak of this data basically enables a denial of service attack against parts of the economy dependent on personal identity.

I know this is somewhat off point, but there's a https://en.wikipedia.org/wiki/Fallacy_of_division happening here when considering risk - individually, my chance of impact is low (I think?), but as a society, the risk of impact is very high.

I think that makes it worth replacing everyone's SSN, at a minimum.

danpat··on Bitcoin Is Creeping into Real Estate Deals
That's apples and oranges. A house and land doesn't become inaccessible if a key is lost, and the gatekeeper of "ownership of land" isn't the mathematics of the blockchain, it's the government.
danpat··on SoftBank Leads $164M Bet on Mapbox
The `{name}` field that's the default for most place labels in styles is the "local name", in the local symbol system.

Change it to `{name_en}` in your style for place labels, and you'll get English-script names where available.

danpat··on Fast geodesic measurements in Go
The basic answer is speed and accuracy. The approximations implemented here are within 0.1% of the Vincenty method for distances under 500km, which is useful in all kinds of situations. The Haversine formula works for spheres, so it's not as accurate as the Vincenty method for the Earth (a flattened sphere), and it uses more trig functions than this approximation, so it's slower and less accurate for lon/lat distance calculations.

I work with Vlad (the author of the original Javascript https://github.com/mapbox/cheap-ruler). At the time it was created, we were basically just looking for the fastest method to get accurate distance calculations on lon/lat values within "short distances". Vlad found that this approximation fitted our needs well (i.e. the types of distances we typically needed to calculate distances for), and performed the best.

For really accurate measurements, you typically need to use a projection system that's targeted at the area you're measuring in. The earth isn't even a perfect flattened sphere, it's covered in lumps and bumps, so if you need really accurate measurements, you need to use one designed for the bump you're measuring on. Some great illustrations here: http://www.icsm.gov.au/mapping/datums1.html

danpat··on Don't use Hadoop when your data isn't that big (2013)
Postgresql has base64 encode/decode functions:

SELECT convert_from(decode(encode('Hello world','base64'),'base64'),'LATIN1');

danpat··on “MP3 is dead” missed the real, much better story
> since Licensors have agreed to license the unexpired Licensed Copyrights and Licensed Patents of the G.729 Consortium Patent License Agreement under the existing terms on a royalty-free basis starting January 1, 2017.

Am I reading that right? The consortium has graciously begun issuing royalty-free licenses now that the patent has expired?

How nice of them.

danpat··on Velodyne Announces a Solid-State Lidar
Multi-frequency phase interference can be used to get sub-cm accuracy without needing ridiculous oscillators. The only problem is that you need to sample your target several times with different frequencies to remove the ambiguity.

Units like this:

https://www.amazon.com/Bosch-GLM-35-Measure-120-Feet/dp/B00V...

that go for $75 (or less) do this. They're super accurate, the only downside is that it takes a second to cycle through all the necessary frequencies.

danpat··on Ask HN: How to actually “talk to your customers”?
Absolutely this. This is also one of the reasons why there seem to be many successful business where the founder "scratched an itch" - they built a business on something they knew a lot about.
danpat··on Long-Term Thinking and Nuclear Waste
As a practical example, the aboriginal people of Australia (particularly in the north) refer to a thing called "sickness country":

  http://www.artistwd.com/joyzine/australia/abr_culture/sickness_country.php
These have generally been found to be regions with a lot of near surface uranium ore.

Given enough time, people will figure out that certain areas are dangerous and should be avoided. As long as it's not everywhere, it probably won't threaten the whole of humanity.

danpat··on Unplanned Freefall? Some Survival Tips
Pretty much what you'd expect - massive injuries, sometimes death. Search Youtube for "speed skiing crash" for examples. Sometimes people get lucky and just slide to a stop, depends on how they fall.
danpat··on JPS+: Over 100x Faster than A* (2015) [video]
Hello from OSRM :-) I would've quoted exactly the same paper, it's the best overview that I know of.
danpat··on Ask HN: Who is hiring? (February 2017)
Mapbox | ONSITE in Washington D.C. or Berlin, Germany | Systems Engineer - Directions | Full-Time | http://www.mapbox.com/

The Directions team at Mapbox is looking for someone to help grow our navigation platform infrastructure. We have a core group working on routing algorithms and traffic data analysis, and we need help growing the infrastructure that runs that code (we develop and make heavy use of http://project-osrm.org/).

We use nodejs and AWS services extensively for our infrastructure, so familiarity with those tools is a plus, but by no means a requirement. We like adaptable people who aren't afraid to learn new skills, and bring new perspectives to the table.

A bunch more details at: https://www.mapbox.com/jobs/553439/ or hit me up with any questions.

danpat··on Ask HN: What are your profitable side projects?
This approach, while good for setting a lower bounds for what's sustainable, is rarely how you should first price things.

I run a small side business. My initial customers I priced like this. After the first year, I realized I was leaving a lot of money on the table - my business was niche and not commoditized, and my customers were willing to pay quite a lot more for the service.

I grandfathered in the first year customers and re-structured my pricing to be more "value based" - how much were my customers actually willing to pay for the service, i.e. how much was it worth to them.

It took a little while to figure out good price points, and there was a short period of time when I'm sure I lost some customers because prices were too high, but in general, my margins are significantly higher by charging customers based on their percieved value, as opposed to just covering my costs with a fixed margin. The customers don't even mind, because in their minds, the price is good value.

It's important to do this not just for its money-making potential - it creates a better impression of "value" for your product, and people value things that they pay more money for. Underpricing can cheapen people's perspective of your product, and de-value your time.

Minimum margins are for commodity businesses - where there are lots of competitors and customers can switch easily. For anything else, you can earn more money by considering "value pricing" based on talking to your customers and finding out their pricing tolerance.

danpat··on Ask HN: Who is hiring? (January 2017)
Hmm, that's no good. I'll let our operations folks know right away.

In the meantime, you can email me at daniel@mapbox.com and I can manually add you to our applicant tracking system.

danpat··on Ask HN: Who is hiring? (January 2017)
Mapbox | ONSITE in Washington D.C. or Berlin, Germany | Systems Engineer - Directions | Full-Time | http://www.mapbox.com/

The Directions team at Mapbox is looking for someone to help grow our navigation platform infrastructure. We have a core group working on routing algorithms and traffic data analysis, and we need help growing the infrastructure that runs that code (we develop and make heavy use of http://project-osrm.org/).

We use nodejs and AWS services extensively for our infrastructure, so familiarity with those tools is a plus, but by no means a requirement. We like adaptable people who aren't afraid to learn new skills, and bring new perspectives to the table.

A bunch more details at: https://www.mapbox.com/jobs/553439/ or hit me up with any questions.

danpat··on The surprising benefits of a mid-career break
I went through this - it definitely affected me, but I wouldn't call it a negative.

It wasn't so much the living away from home (we were overseas for 2 years when I was 10 years old) it was the return - the shocking disparity between how I now saw the world, and how my peers (who were frozen in time in my memory) saw things. It forever separated me from them in ways they could not understand.

In hindsight, it was one of the best things my parents did for me - compared to most of my home-grown peers, I adapted faster to new situations and had little difficulty adapting to life as an independent adult - in my opinion, thanks to the expanded perspective I gained when I was younger.

YMMV, but IMO, calling it "challenging for a child's social development" implies a negative, but it really depends on what your goals for that child's development are. It's certainly an experience I'd like to repeat for my own children if I can make it happen.

danpat··on Most drivers who own cars with built-in GPS systems use phones for directions
Just for kicks, I used OSRM and added a large time penalty for taking left-hand turns.

Here's the kind of route you get:

http://imgur.com/VW63fq1

In reality, the metric probably needs to be a lot more complex than what I hacked together here. It's actually more like "don't turn left across traffic, left turns are ok onto one-way streets, etc", my metric was simply based on turn angle, and there are a bunch of scenarios where that falls down.

The rule probably works well in travelling salesman-like scenarios - you want to optimize the route to deliver multiple packages.

Having played with adjusting these kinds of metrics quite a bit, it's never as easy as you want :-(

danpat··on Reflections of an “Old” Programmer
I went looking. After years of cruft work, I took some time off to pursue other interests. When I was ready to come back to software development, I took a good hard look at the things I actually enjoyed doing, then went looking for organizations that did that stuff. I was fortunate to have the breathing room to be deliberate in my search.
danpat··on Most drivers who own cars with built-in GPS systems use phones for directions
Interestingly, according to this summary:

http://drivinglaws.aaa.com/tag/telematics/

there are exemptions of some kind for navigation systems in many places.

I've heard "it's not allowed by law" often, but I have yet to track down the actual law. Hopefully automakers aren't doing this based on hearsay....

danpat··on Most drivers who own cars with built-in GPS systems use phones for directions
Check out the http://maps.me/en/home app - it gives you offline routing using OpenStreetmap data on your phone.

Google Maps also supports offline downloads, but from what I remember, they expire every 30 days, so you need to keep re-downloading.

danpat··on Most drivers who own cars with built-in GPS systems use phones for directions
Believe it or not, there exists a Nash equilibrium for this. There are several papers discussing various approaches to optimally distributing traffic so everyone gets the best possible route given all the cars on the network:

http://dl.acm.org/citation.cfm?id=2008646

http://www.sciencedirect.com/science/article/pii/S0191261504...

http://ieeexplore.ieee.org/document/1181966/

I have read a paper (which I can't find now) that claimed that cooperative optimization had visible effects on overall routing with as little as 7% of the traffic population participating in the cooperative routing scheme. So it doesn't even require that everyone use the same scheme to get some benefits, which is nice.

danpat··on Most drivers who own cars with built-in GPS systems use phones for directions
OSRM is looking at improving this exact situation right now: https://github.com/Project-OSRM/osrm-backend/pull/2912
danpat··on Most drivers who own cars with built-in GPS systems use phones for directions
I'm paid to work on navigation software: http://www.project-osrm.org/

The answer to your question is: because it's hard to make some of these adjustments on-the-fly and still have reasonably fast queries.

On embedded devices, space and processing power are limited - so you need to perform optimizations to speed things up so it's usable. These optimizations usually limit the flexibility of features like you're requesting. That's a basic trade-off.

For server-side routing solutions, you need response time to be fast so you can scale - again, you need to optimize your data and you lose flexibility. Route calculations are CPU heavy, it gets very expensive to serve lots of users if your route calculations aren't fast.

There's a lot of algorithm research being done in this space - the last 5 years have seen some really smart new approaches. However these haven't been deployed commercially in very many places.

If you'd be happy with 10s of seconds to calculate a route on your phone (offline), then you can have all the flexibility you want :-) This is rarely desirable, so very few implementations are as flexible as you want.

danpat··on Reflections of an “Old” Programmer
I work on navigation software (http://project-osrm.org/).

Many of the algorithms we're implementing (or at least considering) only exist in recently published papers, or sit behind unpublished APIs. There have been huge improvements in graph route-finding algorithms in the last decade, so much of it is new, interesting and it's far from run-of-the-mill implementation.

I'm 38 - I spent the first many years of my career doing CRUD development, first in Perl (late 90's), then Java/PHP (2000's). I skipped the JS craze, and now I'm enjoying my work more than ever improving my C++ skills (last time I touched C++ was 98, modern C++14 is a huge improvement) and working on backend, specialized algorithm implementation. It's great!

Experience is the best teacher. Kids don't listen to their parents, new developers don't listen to the greybeards until it's too late. This is the way things are :-)

danpat··on Show HN: Dijkstra’s algorithm in the web browser with OpenStreetMap
The difficulty is that the performance of queries on the final graph is dependent on it's shape. As lorenzhs said, you want the shortcuts to be as long as possible.

The final shape of the graph is highly dependent on the order you contract the nodes in - small changes in contraction order have large effects on the final shape.

One of the very expensive parts of the pre-processing step is determining the best order to perform contraction. Sure, you could just iterate over all nodes, contracting as you go (and parallelize), but you'd end up with a contracted graph that's not a whole lot better for queries than the original. Order matters.

The original CH paper covers lots of the details:

http://algo2.iti.kit.edu/documents/routeplanning/geisberger_...

There is a general group of approaches that do what you're describing - partition the graph recursively, and produce optimized overlays in various forms. This can be done in parallel, and recursively:

http://www.dis.uniroma1.it/challenge9/papers/holzer.pdf

Query performance is generally not quite as fast as a well-optimizied CH graph, but the overlays can be generated much faster and that work can be highly parallelized. We hope one day to get a chance to implement this approach in OSRM.

The difficult problem with the second approach is partitioning the graph well :-)

danpat··on Show HN: Dijkstra’s algorithm in the web browser with OpenStreetMap
I'm one of the OSRM devs.

To expand on this comment a bit - OSRM still uses Dijkstra, so if you understand that, you already basically understand what OSRM does.

What OSRM does in order to speed things up is optimize the graph structure - we still use Dijkstra, but the search completes in a handful of steps, rather than hundreds of thousands.

There are quite a few techniques like this. OSRM implements an approach called Contraction Hierarchies. We scan over the graph, inserting "shortcuts" that skip over nodes. As long as you follow a few basic rules, you can repeatably insert shortcuts all over the graph. This gives you a routing graph that is equivalent, but a Dijkstra search will typically complete in a handful of iterations.

We hope one day to implement several other speedup techniques - each has advantages/disadvantages, depending on what you want to do. Contraction Hierarchies lead to very fast queries (~5ms for a cross-the-US route), but the pre-processing time is very long (~6hrs on a beefy machine for the OSM planet). Any updates to the graph require complete re-processing (new/removed roads, adjusted road speeds, etc). Other techniques compromise search performance for a bit more flexibility - faster update times, query customization (i.e. "avoid highways").

It's a really fascinating corner of CS theory to work in, I really enjoy it :-)

This paper:

https://arxiv.org/pdf/1504.05140.pdf

"Route Planning in Transportation Networks" gives an excellent overview of current search speedup techniques. It's a bit hefty, but if you're interested in knowing what's the state of the art, this is a good place to start.

danpat··on Show HN: Dijkstra’s algorithm in the web browser with OpenStreetMap
You can type place names into the entry boxes on the front-end - they get converted to lon/lat using a geocoding database.

OR you can drop pins on the map.

danpat··on Python 3.6 dict becomes compact and keywords become ordered
For once, C++ got some usability right and created std::map and std::unordered_map which have explicit key ordering behaviour.

Now, back to waiting for my project to compile.....

danpat··on Show HN: Eppstein’s k shortest paths in the web browser with OpenStreetMap
A plug for one of my co-workers who recently finished his PhD on alternative route finding:

http://algo2.iti.kit.edu/documents/Dissertation_Kobitzsch_Mo...

Choice quote from page 50:

k-Shortest Path. A, at the first glance reasonable, approach to alternative routes is the k-shortest path problem [Yen71]. The basic notion behind the problem, which has been studied quite extensively (e.g. [Shi79, Epp94, Epp98, Rup97]), is that next to the shortest path itself, slightly suboptimal paths will offer good alternatives. While this idea seems valid for specific networks2, it has been described as less effective [BDGS11] in the context of alternative routes in road networks. It is rather unlikely to find a good alternative route among the first few hundred paths. Jumping off the highway at a ramp and directly returning back onto it does not take much time compared to the full journey. Doing this at every possible combination of ramps might already contribute a large number of possible paths that are only slightly worse than the original shortest path. This directly implies that we might need a very large value for k before we can report a reasonable alternative route.

danpat··on The Day I Got My Green Card
It's probably an E-3, which was created as part of a free-trade deal between Australia and the US.

https://en.wikipedia.org/wiki/E-3_visa

Source: I've had one.

← PreviousPage 2 of 7Next →