akssri

isync and cron

Gandi may also be an option here - a domain registered with them comes with two free mailboxes, each with a few GBs of space and calendar support.

(Yes, yes, this is the same company that lost the data on their cloud servers.)

- Function in Cupy takes 29.4ms, Numpy takes 427 ms. Happy ?

- Broadcasting semantics + division takes care of the outer-product normalization. This is 2 L1 ops in size of the matrix & the input (~ xSCAL).

Pedantry is still not an argument.

> And, yet, you've taken it pretty seriously to diss Clojure/Neanderthal and my blog post

I have not - all I've said so far is that your benchmark is flawed.

The fact that the code fragment above assumes zero mean data (thus using 2 fewer L1 ops) doesn't change a single thing in anything that has been written; to wit, the timings change to 28.6ms (GPU) and 333 ms (CPU). Pedantry is not an argument.

Common Lispers have had the ability to write CUDA kernels (with shaders) on-the-fly with a DSL in cl-cuda for many years now. See,

https://github.com/takagi/cl-cuda

Cupy (a project in which takagi is/was actively involved) also has the ability to compile kernels in the Python REPL, but the kernel code needs to be written in C and passed in as a string (pyOpencl can do the same for OpenCL). In fact, not too many years ago, the kernels for gradient descent, max pooling etc. were all entirely in Python in Chainer. Projects like JAX take this to another level by having the ability to transform the bytecode of a restricted class of Python functions straight into CUDA kernels.

I can't speak much about Clojure/Neanderthal, but I'd advise the people to stop dissing projects they are not familiar with, least of all using silly benchmarks like the above.

> user of NumPy/CuPy to perform the float32 computation

This is just getting tiring.

Numpy and Cupy are perfectly capable of doing float32 computation - their only "fault" is that they coerce data to float64 in this one fairly unimportant function (which you can implement to your liking in 3-4 LOC). Hell, PFNs entire deep learning system, Chainer (which mind you, both predated and inspired PyTorch, and is still quite competitive with it), is built entirely on top of Cupy!

Benchmarks make sense only when the outputs are the same - in this case, they certainly are not. It's the responsibility of the author to make sure that the outputs are the same (a real benchmark), or to argue that Numpy/Cupy are wasting time by using float64 in xp.cov (an issue of implementation). He does neither.

Numpy/Cupy do explicit coercing to float64. There is no documentation for why this is done, but since GEMM is used to compute the covariates (summation over data points), it makes sense to increase the precision.

You could get away using a double to only accumulate the sum, but it's a pain to write such mixed-precision (slow) function in C and then wrap it in Python, esp. for something like this.

(See, https://github.com/numpy/numpy/blob/v1.17.0/numpy/lib/functi...)

> What is /flawed/ there?

You're comparing float32 vs float64 computation. I don't need to tell you how much slower DGEMM is vs SGEMM esp. on the GPU (you mention this in the post yourself!).

Numpy does this for precision reasons, and CuPy simply follows its behavior. This is precisely why I noted that the float32 version runs 3x faster on the CPU.

> The reason for that is that CuPy is poorly implemented.

It's a cheap shot to call something 'poorly implemented' when you don't understand what you're benchmarking.

The PyT version above and the both the CPU/GPU versions on Neantherdal work completely in float32. I confirm that the performance of Neantherdal GPU is similar to that in the above PyT version. Yes.

The part that is flawed is that you're comparing this to Numpy (CPU)/ Cupy (GPU) both of which coerce the input array to float64 (for precision reasons) before computing the covariance and correlation. You only need to check the output type of the result to verify this (if the pointer to the code is not sufficient).

The benchmark is flawed. Cupy follows Numpy in doing a type-promotion of the data to float64 before computing the covariance.

https://github.com/cupy/cupy/blob/b19577199d3f75c76e23c75194...

The following function in PyTorch completes the float32 computation in 27 ms (1080 Ti). I'm fairly sure that a similar function in Cupy would achieve very similar results.

  def corr(dat):
      cov = (dat @ dat.T).div_(dat.shape[-1]-1)
      sigma = torch.diag(cov).sqrt_()    
      return cov.div_(sigma[None]).div_(sigma[None].T)

The heart of the computation SGEMM is in anycase provided by CUBLAS, and the only areas in which PyT and Cupy differ are in the tricks used to unroll the loop for the broadcasting & L1 op. The performance difference b/w PyT, Cupy and others should be negligible IMO.

Note: np.corrcoef takes about 1.12 s and the above float32 version takes 325ms on the CPU.

Sorry, but how is this flamebait ? I've given sources, and academic citations (from publishers like OUP and SUNY press). I even mention how such false knowledge affects not just Europeans view of India, but the Indian view of India.

I'm a string in X^*, and the universe is a real number.

Shrug all you want, but it most certainly is not and neither will it ever be. The definitions have been laid out in the Ashtadhyayi ~2000 years back.

It's not my concern if certain folk have no education in Panini, and are arrogant enough to ignore his work.

It is indeed the सम्बन्ध; it's not a कारक technically. The canonical defn. of कारक is that which relates a noun to the verb. सम्बन्ध relates relations between nouns (you can't have sentences with only a verb and a noun in genitive).

It is for the most part (see answer above).

That's the story that's told in introductory texts, but the genitive generally forbids such permutations (depending on the flavor of usage in the geographical area).

There are actually only 7 kinds of सुप्s (x 3 numbers); there are about 10 तिङ्s, but only about 3-4 are in general usage. Then there are other sentence transformations outside these.

The relational modifiers "stick" to the roots, so moving words around the sentence is quite easy - this is particularly useful to poets who need to satisfy certain mathematical rules in their compositions. Much of this is also true for languages like Kannada/Hindi. There was also a paper about a KR scheme inspired by something of this kind.

Sanskrit is generally easy to pick up with the right teachers, in an immersive environment; the grammar-technique, esp. in a "sterile" environment, is rather terrible IMO.

Language too has nothing to do with the Universe - that doesn't prevent sentences in it from "mean"-ing something.

Mathematical truths have nothing to do with the universe.

Interestingly, many Indic [2] languages follow similar verb-centric grammars. In the canonical Vyakarana tradition (of Panini), sentences are seen as revolving around the verb [1].

The "noun-cases" or कारक (karaka) are generally equivalent to the "particles" in Japanese. The genitive (eqv. の) is not a karaka, since it has no relation to the verb. Of course, since there is technically no syntactic difference between adjectives and nouns in Sanskrit, the semantics of the genitive in particular can be very undeterministic. This is not the case in others though.

I wish there were more studies on how Indic traditions affected East/SE Asia [3]. Sadly, most academics/people here don't believe there exists a world outside N. America & W.Europe (often no India either!).

[1] There is a competing tradition of semantics called "Nyaya" where sentences are seen to be Noun-centric. These discourses are generally not easily accessible.

[2] Dividing the languages based on presence/absence of noun inflections would appear not to have much discriminative power to claim anything about historical origins. Historical Linguistics, I believe, is mostly a politicized pseudoscience.

[3] This documentary highlights the kind of things I mean.

https://www.youtube.com/watch?v=8WaenzbSJwk

It is also fascinating to look at the Thai/Khmer scripts and realize these are related to current day Telugu/Kannada scripts.

Graphical models might also be something folks might want to consider. Ideas from PGMs are often behind many advances in ML.

The canonical text is by Daphne Koller; a course I took used Martin Wainwright's monograph though - the book is briefer and dives into the math quicker.

https://people.eecs.berkeley.edu/~wainwrig/Papers/WaiJor08_F...

This is just too sad. Iranians at grad school often struck me as incredibly smart - be it students or faculty.

The recent breakthrough in TSP approximation was due to an Iranian; the woman who won the Field's prize earlier was one too! They are a tiny nation but are very well represented.

They were already at the receiving end of the stick under Obama, having to deal with the sanction and single entry student visas.

These policies make no sense considering their distinctions as immigrants; it's an affront to one's humanity TBH.

The language used can be found in introductory chapters on Differential Geometry.

NES clones are still going strong in India :) Summers spent playing pirated games on 8-bit cartridges - what joy!

SNES sadly was a rarity, even if it was much better (I'd personally rate SNES higher than Wii). Frankly, if there were nice touch interfaces for SNES emulators, I'd gladly pay $10. Carrying a bluetooth gamepad is definitely an alternative.

Mark Levi has written about these things on SIAM and has a nice video up on,

http://www.personal.psu.edu/mxl48/Inverted_Pendulum.html

I think they've also shown that this is robustly stable for a particular family.

Implementation in Lisp,

https://github.com/matlisp/matlisp/blob/master/src/graph/gra...

Ah, that's too bad :( CL is very very underrated as a language.

Any insider info on why cleartrip moved away from Lisp ?

Neat. I use it for things like linesearch and regularization (in optimization),

https://github.com/matlisp/matlisp-optimization/blob/master/...

As a fellow Indian lisper, are you by any chance using CL for work ?

Last I heard, the only big CL shop, cleartrip, moved all their codebase to Ocaml.

For me, error handling has a major flaw: stack unwinding - extremely annoying thing to happen when the program state took many many hours to achieve. I don't think there is any language other than CL that allows restarts etc. to be defined; slime-repl too is invaluable when debugging.

http://www.gigamonkeys.com/book/beyond-exception-handling-co...

How general/efficient are these AD systems ?

New ops in tensorflow seem to be oriented towards forward-mode AD rather than reverse-mode (for which one needs a pull back op on a dual).

Do you know if CompNeuro models have been trained to do things interesting to CS folk ?