nicksdjohnson

Enough parts for 20 boards cost me about 100 pounds; the board fabrication cost about $60, but a lot of that was because I wanted it in a rush. All up, that works out to about 8 pounds, or 13 USD per board. It'd be somewhat cheaper - around $10-$11 - if you're not in a rush for the boards, and of course cheaper again in larger quantities.

I probably overdid the solder a bit, but there's no way most of the pads had enough solder on them for SOICs without my adding any. I've soldered SOICs before, but this is by far the largest volume I've done in a sitting.

As I mentioned briefly in the post, the autorouter was something I used largely as a result of time constraints - with 200-odd wires to route and very little time to do them in if I wanted the PCBs back in time, I decided to give it a go. That said, I was surprised with the quality of the results - I'd be surprised if I can find any easy-to-remove vias left in its solution. The tool is also very good for assisted hand routing, since it lets you nudge traces around without ripping them up every time.

Have you read the papers I linked in detail? Some of them, such as HyperLogLog, provide corrections to give better estimates for small sets, and although I can't follow the proof in its entirety, they claim to be more efficient than the alternatives, including the one you propose.

A friend mentioned the existence of this, but I couldn't find it myself. Thanks for pointing it out.

All the algorithms can process data in a streaming fashion, though - they only require a single pass.

I actually cover that in the post (well, more or less - I talk about hashing to remove bias, after talking about the 'min element' algorithm. According to the papers cited, though, taking the count of leading zeroes is more space efficient, allowing you to have more buckets in the same amount of space.

Very nice, thanks! It might be too similar for a post all of its own, but I think it'd be a worthy followup to this post.

I can't express how awesome it is to hear that.

Actually, it's even worse than that - with a rateless code like Fountain codes, you can generate an effectively unlimited number of encoded blocks, making pregeneration of hashes for them all completely impractical.

In all other particulars you're dead right.

Absolutely. It's still pretty expensive to take a 257-bit power of a 1024-bit number with modular exponentiation, though.

You should write this up somewhere and post it to HN and the App Engine Reddit. Getting geospatial right is hard, and it sounds like you've done a good job.

That's a good question. I can't point to published figures, since the 2.7 runtime is still fairly new, but I can say that based on both my personal experience and based on fairly basic reasoning, the per-thread memory overhead is definitely a lot less than what's required by the whole instance. The entire of the Python standard library, along with your framework and other libraries, are shared overhead between all the threads.

The issue with charging by CPU hour was that you could occupy memory-seconds as much as you wanted without charge; that's no longer the case - by charging for instances, we're implicitly charging for the memory they use.

As far as determining how many instances you run - you can do this to a large degree, both by setting budget limits, and by setting scheduler parameters.

One gets from 31 cpu hours to 879 instance hours if your average CPU utilization is about 3.5% - see my other post for details.

Hi folks,

I'm on the App Engine team, and I just wanted to clarify one thing: The main difference between CPU hours and Instance hours is that CPU hours are charged based on CPU usage, while instance hours are based on wallclock time. The high ratio between the two you can see with PlusFeed is because it's spending a lot of time to serve each request, most of which is spent doing nothing - likely because it's doing outgoing HTTP requests.

Previously, we had no way to account for apps like this, that take a lot of wallclock time but very little CPU time, and as a result we couldn't scale them well. Under the new model, the charges reflect the real cost here - memory pressure. Every second an instance sits around waiting is a second that the memory occupied by that instance can't be used to serve other requests.

As others have pointed out, we're in the process of launching Python 2.7 support - it's currently in Trusted Tester phase - which will support multiple concurrent requests, and services like PlusFeed are likely to be able to take great advantage of that, reducing their instance hours by a large factor. Likewise, doing asynchronous URLFetches (where that's practical) can cut a huge amount off instance time.