pdeffebach

Perfect! So glad this exists!

Thanks for the feedback! Just to re-state my case in clearer terms:

For my personal workflow (others may differ), compiling to html is only done once at the end of a session, and the latency wouldn't matter if it could execute like a script. Weave.jl^[1] has a great feature called `include_weave` which has the features I like.

But take my feedback with a grain of salt. I generally just save things in folders and compile a pdf separately with many tables and figures.

[1] https://weavejl.mpastell.com/stable/usage/#include_weave

I wish there was a better way to run quarto as a script, as in, as fast as `source` in R and `include` in Julia. Current behavior

1. Has scoping rules that make it difficult to debug 2. Has low latency, making it frustrating for debugging.

Have you tried DataFramesMeta.jl? It has a tutorial for people familiar with tidyverse. See [here](https://juliadata.org/DataFramesMeta.jl/stable/dplyr/).

Maybe [this](https://m3g.github.io/JuliaNotes.jl/stable/workflow/)? The key is to work with modules, use Revise.jl, and Infiltrator.jl.

Jupyterlab is just an IDE. You can open up terminals, consoles (which are like notebooks, but are a console), and notebooks, on top of a text editor.

It's actually really nice. Works great over ondemand and is way more responsive than X-based apps.

Jupyterhub is a way to coordinate compute space / config across many users of notebooks or jupyterlab.

> The American problem is not one of a lack of houses. This is just a symptom.

We made it illegal to build housing, so we don't have enough housing. We could fix our housing crisis by re-legalizing housing _without_ fixing all of the (very real) problems you describe. You are simply engaging in whataboutism which will _not_ solve our housing crisis.

I think small businesses owners often treat their workers badly and are overall a reactionary force in American politics.

But sympathy makes a bit of sense when you think about it economically. The businesses that do make it in have more market share and are selected to be those that are politically connected. The regulations serve to keep other businesses out, and we should try to help people that aren't as connected start businesses as well.

Thanks for the detailed response.

Yeah allocation seems like the biggest hangup here. I would rather have a function stick to a "no allocating" contract and allow for some undefined behavior than have a function unexpectedly allocate to preserve safety.

What semantics are expected in other languages? This seems solidly in the realm of undefined behavior as far as I can tell.

I don't get these complaints about `sum!(a, a)`. Sure it's a bit of a footgun that you can overwrite the array you are working with. This doesn't rise to a "major problem" of composability.

The histogram errors seem annoying though. Hopefully they can get fixed.

In Julia you could have an `AbstractVector` type also be callable, or more likely a vector of callable objects (and the operation is performed row-wise).

I agree it's unlikely that a user will name their column `.data`. But it certainly saves developer effort from thinking about these issues.

The larger concern, really, is that Julia needs to know which things are columns and which things are variables in an expression at parse time in order to generate fast code for a DataFrame. It needs to do this without inspecting the data frame, since the data frame's contents aren't known at parse time.

One option would be to make all literals columns. But then you run into issues with things like `missing`, which would have to be escaped or not recognized as a column. Its hard to predict all the problems there, and any escaping rules would definitely have to be more complicated than R's. So we require `:` and take the easy way out, which has the added benefit for new users who might get confused about the variable-column distinction.

It would be interesting to profile the 2nd version though. Assuming the non-standard evaluation has performance benefits (which they do in DataFramesMeta.jl), are you eliminating those benefits when you use

    .data[[a_var]]

?

Let's say you have a data frame

    df = tibble(a = c(1, 2))

and you want to use a dplyr verb to modify it

    mutate(df, b = a + 1)

the `a` in the above expression refers to the column in `df`, but this means it's hard to reference a variable in the outer scope named `a`. Furthermore, if you have a string referring to the column name `"a"`, you can't simply write

    mutate(df, b = a_var + 1)

Contrast this with DataFramesMeta.jl, which is a dply-like library for Julia, written with macros.

    df = DataFrame(a = [1, 2])
    @transform df :b = :a .+ 1 

Because of the use of Symbols, there is no ambiguity about scopes. To work with a variable referring to column `a` you can write

    a_str = "a"
    @transform df :b = $a_str .+ 1

I won't pretend this isn't more complicated or harder to learn. Some of the complexity is due to Julia's high performance limiting non-standard evaluation in subtle ways. But a core strength of Julia's macros is that it's easy to inspect these expressions and understand exactly what's going on, with `@macroexpand` as shown in the blog post.

DataFramesMeta.jl repo: https://github.com/JuliaData/DataFramesMeta.jl

Yes it absolutely needs a new name!

DataFramesMeta.jl might be exactly what you are looking for then! The syntax is very close to dplyr, but has performance benefits thanks to Julia.

Here is a tutorial for those familiar with dplyr: https://juliadata.github.io/DataFramesMeta.jl/stable/dplyr/

> 2) I personally found that Julia community is slightly hostile to feedback and negativity. May be it is just me but it has way too much hype-driven-positivity that leads to delusion.

I want to push back on this a bit, which I acknowledge is very ironic. In the past few years people consistently post on Discourse asking for fundamental changes to the language to make it more resemble python, C++, or whatever their preferred language is.

People often say Go is great because there is "only one way of doing things", yet people are very resistant to being told "the way" to do something in Julia. This has happened enough that it's prompted a pinned PSA on discourse: https://discourse.julialang.org/t/psa-julia-is-not-at-that-s...

It gets tiring! And i'm not sure how the community should handle these requests, but I don't think it's fair to blame all of the negativity on the Julia community when these somewhat misinformed, or even bad-faith posts are so frequent.

Hello everyone. Hackernews has been discussing excel a lot the past few days. Fortunately, this conversation coincided with a release of a Julia package I've been working on, ClipData.

It's a very simple package that also displays some of the core strengths of Julia as a language: multiple dispatch and interactivity.

I can imagine someone slowly transitioning a complicated workbook to a script with this package.

Just that there seems to be an expectation of using Rstudio when using things in R that seem generic.

One pain point is that Rmarkdown uses a different pandoc installation when executed by Rstudio than from the terminal.

Yeah people in python are used to doing `python script.py` all the time, and that's not very convenient in julia.

sysimages are great, as is daemonmode. But really just do Revise at the REPL.

QtConsole is actively maintained. I don't use it but I do like it a lot.

Plenty of people use the REPL in terminal and sublime text or vim or whatever. I also dislike browser-based tooling and think Julia has done a good job avoiding Rstudio-style dependencies.

But if your point is the inability to do `julia script.jl` , yeah thats a pain point. Fortunately there has been some tooling to make running many jobs in a row easier: https://github.com/dmolina/DaemonMode.jl

> As far as I am aware there has not been any major push at the compiler level to speed up untyped code yet – although there should be plenty of room for improvements – which I suspect would benefit DataFrames greatly.

That's not quite correct. The major `source => fun => dest` API as part of DataFrames.jl was designed specifically to get around the non-typed container problem. And it definitely works. That's not the cause of slow performance.

I think the reason is that, as you mentioned, DataFrames has a big API and a lot of development effort is put towards finalizing the API in preparation for 1.0. After that there will be much more focus on performance.

In particular, some changes to optimize grouping may have recently been merged but didn't make it into the release by the time this test suite was run, as well as multi-threaded operations, which havent been finished yet, should speed things up a lot.

That said, this new Polars library looks seriously impressive. Congrats to the developer.

Is the Navy really a good example of command structures that promote effective UI?

According to Propublica, part of the reason for the Fitzgerald crash was bad interface design. https://features.propublica.org/navy-uss-mccain-crash/navy-i...

Julia!

> This would allow a `plot` function in Python, originally written for the float data type, to also work for float-like (quacks like a duck) data types, like `Decimal`.

This might come with sacrificed performance. If you want a different implementation for a different data type to get the best performance, writing one method makes that hard.

> The `Measurements` author very specifically (if I understand this right) implemented a `plot` functionality.

This is a good point. But in that example, note that `Measurements` also works with all of differential equations, and they didn't implement any differential equations specific code, unlike as you pointed out with `plot`. The fact that uncertainty propagates through a differential equation solver is pretty impressive, imo.

Notions of consent are different. You do what the doctor tells you, and its an enormous ethical violation to recommend drugs without revealing the way you personally gain from it, or to encourage people to recommend drugs that are not medically necessary.

With legalized drugs there isn't this trusted arbiter telling you what to do, so you can consent easier when you buy and take them.

I agree that it's fraught and there are tons of ethical parallels that need to be worked out but that's one reason.

You can't edit a function like that. If you add a new function definitoin for

    foo(x::Baz)

then you will never enter the method that has those if else statements. This is the benefit of multiple dispatch. You can't add more if else statements inside a method that is already written but you can add new methods.

This is a good feature! Sometimes, you want to use `if` `else` based on type information, say if your function is very large and the different behavior based off of type is only a small piece of the process you want to convey.

It's great to have the flexibility.

In Julia's case, there are maybe more "free hands" who can contribute to performance critical code. To contribute to `NumPy` you have to know how to program in C.