twtw

These two statements seem to conflict a bit. Anybody else think so?

> are flooding their home, driving up rents, installing yoga studios, polluting the local vibe with new technology, and generally making everything suck while sipping kale juice.

> These people want to be artists, teachers, blacksmiths, therapists, mechanics, and musicians. They want to have children, open bakeries, own a house ...they aren't techies; they had the audacity to want something besides tech.

Are the people who left California and who apparently are driving up rents and sipping kale juice in wherever they ended up artists, mechanics, musicians, and blacksmiths? California rents have gone up apparently because technology companies pay workers enough they can afford $3000/mo rents for an apartment. Is the effect of an influx of bakers, musicians, artists (who I'm assuming probably can't afford high rents) sufficient to inflate rents enough to complain about? I would have said the "californians" people complain about are employed by technology companies (or other high paying companies).

Yup. I side loaded an app yesterday onto my phone running iOS 8.4.1.

FWIW, automatic differentiation (as used in popular machine learning frameworks) is not the same thing as the symbolic differentiation described in that section.

I do it on iOS 8. Can't recall the Xcode version.

Does sideload still work? I recall they enabled the ability for anybody to load apps via Xcode around iOS 7, but haven't kept up to date with latest versions. Did this stop working on newer iOS versions?

I sideload stuff on my phone quite frequently.

I'm not sure I agree with your analogy between current "autonomous" vehicles and an inefficient lightbulb. I'd say it's more like a lightbulb that flickers and burns out in 5 seconds. In which case it probably makes more sense to make the lightbulb not burn out first.

I don't think Edison (and his employees) got a momentary flash of light from a lightbulb and then said, "alright, it's almost working, now let's see if we can do it without the fragile glass bulb."

The radar in tesla vehicles is the type intended for use in adaptive cruise control. Extremely low angular resolution with a filter that ignores objects that are close to stationary.

This is why Teslas keep running into stationary cars. The radar in them is not designed for this, and can't tell the difference between a fire truck in the lane and the barrier to the side of the lane.

Good discussion. The arguments that because humans can do it with two eyes any approach using more than cameras is wrong frustrate me to no end. Technology and engineering isn't about imagining the minimum theoretical requirements, it's about building stuff that works with what is available. People can navigate cross country without gps, but it's not wrong to put gps in cars.

It seems like the teams working on autonomous vehicles need all the crutches they can get, and it seems like a good idea to lean on superhuman sensors to make up for subhuman "cognition."

I do wish, though, that people would stop equating all radar with the adaptive cruise control style radar - imaging radars are a thing that exist, and can be competitive with lidar.

> and therefore the complete chip.

Most semiconductor companies bin chips based on how broken they are. GPUs, for example, frequently have different skus with the same chip differentiated by how many processors are broken (and thus disabled).

Why?

On my system, I see the following:

  $ /usr/bin/time -f "%U\n" ./exp_volatile_a_b
  0.13
  $ /usr/bin/time -f "%U\n" ./exp_lapack
  0.52
  $ /usr/bin/time -f "%U\n" ./exp_openblas
  0.32

I got LAPACK and BLAS via apt-get on Ubuntu 18.04, so whatever that means. I installed OpenBLAS from source, and it looks like the target detection decided to enable AVX2 (but not AVX512).

So the fully-unrolled version is still faster on the 5x5 matrix, which doesn't seem super surprising to me. I would expect a LAPACK implementation to have some overhead compared to a straight line solution to a problem this small.

Some more interesting points:

- Switching to -O3 for gcc (instead of author's default -O2), the C version (exp_volatile_a_b) gets ~30% faster, bringing it down to ~50% of the python-generated llvm - it appears to still be doing the full computation at runtime.

- Switching to -O3 for llc doesn't make any difference for the python-generated llvm version.

- With gcc 4.8 -O2 (instead of 7.3 -O2), I get ~0.06s - it looks like gcc 4.8 decides to inline everything, and gcc 7.3 doesn't.

Has anyone reproduced these results?

FWIW, I can't.

  $ cd wordsandbuttons/exp/python_to_llvm/exp_c
  $ make
  $ /usr/bin/time -f "%U\n" ./exp_volatile_a_b
  0.13

  $ cd wordsandbuttons/exp/python_to_llvm/exp_embed_on_call
  $ make
  $ /usr/bin/time -f "%U\n" ./benchmark
  0.16

Intel(R) Core(TM) i7-7800X CPU @ 3.50GHz, gcc (Ubuntu 7.3.0-27ubuntu1~18.04), clang version 6.0.0-1ubuntu2, LLVM version 6.0.0

I would think MIT might be a bit special, given that both EE and CS are Course 6 - some other schools have a similar setup (i.e. one EECS department), but many do not.

From a cursory look, it appears there were around ~55,000 CS graduates in 2015 the US, ~11,000 EE, and ~5,000 CE.

I don't agree that a "generation have skipped EE," but I do think it is probably true that CS graduates in general now probably have much less hardware experience/education than has been the case historically (no comment from me on whether that is good/bad).

I wouldn't call this Harvard. It has ICCM/DCCM (closely-coupled memories) that appear to be strict (i.e. core cannot ld/st to ICCM), but everything is in the same (physical) address space, and the CCMs are just mapped into that space. In that sense, the ICCM is not much different from having a ROM in the memory map, which isn't usually seen as making something a harvard architecture.

In my view, RISC-V pretty much precludes a hard Harvard architecture. The fact that RISCV has a FENCE.I instruction that "synchronizes the instruction and data streams" means that a RISC-V implementation can't really be strictly Harvard, since if it was it wouldn't make much sense to invalidate the icache.

This core has an icache, but that also doesn't make it a Harvard architecture. As long as the backing store of both the icache and the dcache is the same, it won't be any more difficult to work with than any other modern modified Harvard architecture (read: pretty much every computer system in every desktop, laptop, phone, etc in the last decades).

> a bus turned upside down is perceived as a plowshare

If I'm thinking of the same picture you are, it's a schoolbus on its side on a snowy road that is misidentified as a snowplow.

FWIW, that picture does look a lot like a snowplow to my eye. Big yellow rectangle against road cleared of snow? If I glanced at it, I might have said it's a snowplow.

One interesting note re: 8b/10b encoding. The motivation in the article is accurate, but not the whole story (never is, in the analog world).

Nowadays, 8b/10b (or more realistically 128b/130b) is critical to enable clock recovery by making sure the signal transitions frequently enough.

> Computers can't count past 1

Reminds me of this excellent quote: "Every idiot can count to one" - Bob Widlar

> We're talking about an API

I don't think we are. We are talking about languages.

In CUDA, compute kernels are written in a language called "CUDA C++." In OpenCL 1.2, compute kernels are written in a language called "OpenCL C." Somebody else probably could have (likely did) implement a compiler for their own version of C++ for OpenCL kernels, but the point 'pjmlp was making is that the standard platform did not enable C++ to be used for kernels until long after it was available in CUDA.

Your rhetoric is a bit out of date. I have no problem with AMD (their hardware is good!), but I don't think your presentation is accurate.

> CUDA

Implemented by AMD too, under the moniker "hip."

> their RTX crap

Vendor-independent via DXR, soon to be available through Vulkan as well.

> Gsync

Nvidia GPUs now work with freesync monitors

> Physx

Open source, as of december.

I want to feel good about AMD, but they have thus far failed to build a stable platform around their GPGPU stack(s). Nvidia has done a pretty good job with making CUDA a stable platform for SW development, however anti-competitive you might think it is.

Thanks for your response.

I'm interested particularly in preemptible pods - I see here https://cloud.google.com/tpu/docs/pricing pricing for preemptible single units, but there is no indication that preemptible pods are available.

Are there plans for preemptible TPU pods?

As is, it looks like p3 16xlarge spot instance (or probably preemptible gcp 8xV100) are still by far the most cost effective option. You have to do a bit extra to tolerate preemption, but it's worth the ~80% savings.

Also, the "TPU pod is 200x faster than single v100" comparison is a little goofy. Might as well say Summit or Titan is faster than my desktop.

> in the end made to happen by an unlikely pair

Who spent years studying birds...

No, the wings of the wright flyer don't flap, but that doesn't mean it wasn't designed based on birds. One key piece (controllability) came as a result of the wright brothers studying how birds adjusted the shape of their wings.

> They also don't post their code so I can't check that their problems with ADAM aren't due to using L2 regularization,

The article indicated they used the same implementation on all device types, so I don't think L2 regularization would explain why the results are fine on CPU+GPU and only bad on TPU.

Looks like riseml has shut down and taken their comparison post down. I was hoping to compare the results.

This is pretty similar to Peter Shirley's "Ray Tracing in a Weekend" series, which gives some more detail and discusses the ideas more, and which recently was put online as "pay what you want": https://twitter.com/peter_shirley/status/984947257035243520?...

The bible for raytracing techniques is PBRT, which, funny enough, is also now available online: http://www.pbr-book.org/

It is a pretty fantastic time for easily available, high quality resources on rendering in general and ray tracing in particular.

This is still my goto diversion whenever I end up in an Apple Store - trying to get Star Wars going on as many computers as possible.

I would add

5. All those new developers entering the web development community want to make a name for themselves, and there isn't as much opportunity for that if you are just using a framework created by someone else.

> You gotta admit the situation is kinda similar.

I disagree.

1. Everybody knows Python is a scripting language, and trying to do heavy computation without handing it off to a library is going to be slow.

2. A vector is the de facto data structure for this in c++. In Python, numpy is effectively the de facto approach for math.

> advocating the use of __builtin functions vs standard functions

It wouldn't say it "advocates" for the use of builtins. The author demonstrates a problem they had (including math.h made their compile times 2x slower in c++17 mode) and shows their solution, and advocates changing libc++ so that c headers don't pull in c++ stuff.

In any case, the snippet you object to uses the builtins if they are available, or falls back to math.h. It's a hack to avoid math.h when it's slow to compile, but stills seems pretty portable?

1/2 second for this single file, not for the whole project. The bit you quoted about removing includes was about the changes throughout the project.