el_oni

Ive only used reduce at work half a dozen times and it does raise an eyebrow each time.

But for unioning a bunch of spark dataframes together i think

    df = reduce(DataFrame.union, list_of_dfs) 

is much nicer than

    df, *rest = list_of_dfs

    for other in rest:
        df = df.union(other)

People just get a bit funny, especially now you have to import it from functools

I wouldn't even say they make it fun, they make it "rewarding" they make it feel like you did something, but I feel worse after scrolling, like some vital essence has been drained from me.

I can't find the motivation to do anything at the moment. But if reddit or facebook get opened up i'll just scroll. It's almost like i've replace doing things with watching other people do things and that somehow makes me less likely to work on my hobbies because i'm not as good or far along.

AI has added to this, almost like, why bother bettering myself when I could probably shit out my idea in a handful of prompts? I need a dopamine fast or something. Might try staring at a wall

We started doing sourdough in lockdown for 1 reason. The shops nearby were out of yeast. kinda limits your options

Beautiful, I'm currently deep into getting our data into iceberg from firehose and I'm really curious what metadata is written, are bloomfilters being written for the columns i want? Has my compaction and sort jobs helped min-max statistics on those columns?

Will take a look when i get to my laptop!

They also point south. If you changed which end of the needle was painted red it would be just as useful for orienting to the south as existing compasses are for facing north

My partner and i used to harvest medlars from a community garden. We made medlar jelly from them when they had bletted. It kinda tasted like tea. Must be the tannins. We ended up making a sweet chilli sauce from it when we still hasnt eaten it when our chillis became ripe the following summer

There is a difference between dark roasts from a specialty coffee roaster and a dark roast from Starbucks.

The specialty dark roast will have notes of cookies, chocolate, nuts. Lots of brown roasty flavour.

Starbucks tastes bitter. With very little nuance. (Unless you cold brew it, then you can leave most of the bitter behind)

Rustler catches panics before they crash the VM and raises them on the elixir side as an exception. So your process might crash but the vm wont

It's sufficiently well known that as a British 30 something I understood what was being alleged just from the headline

Very cool, this should help me to be able to separate my different testing environments.

Thanks for sharing!

The boring company released a flamethrower[0] presumably to raise some capital.

[0]https://www.boringcompany.com/not-a-flamethrower

I've been using d2 recently [0] It's similar enough to mermaid but with the CLI you can output svg and png and have some decent looking diagrams.

[0] https://d2lang.com/

Yeah, this would at least cause me to email my boss and say "Can you just confirm, you want me to transfer $25 million to this account? I'll hold off until you give me confirmation in writing"

hell i do this if our tester hasn't managed to go over some aspect of our release. That way i get in writing from the product owner that he has OKd it, and if he sends me a teams message i ask him to email me confirmation.

I said python doesn't scale well and you say "it does if you use an escape hatch to a faster language"

Sure. Writing C++ that utilises your resources effectively then writing bindings so you can use that in python is great. But with elixir, if I've got 8 cores and 8 processes, those 8 processes run in parallel.

If I want raw cpu speed I can write something in rust, c, cpp or zig and then call it still using elixir semantics.

Not to mention that with Nx you can write elixir code that compiles to run on the GPU. Without writing any bindings.

I think we need to give it time. Python had a slow and steady growth from 1991 until today and it has eaten so much of the data analysis and ML world (backed by C++, C and more recently Rust).

But python doesn't vertically scale very well. A language like Elixir can grow to fit the size of the box it is on, making use of all the cores, and then without too much additional ceremony scale horizontally aswell with distributed elixir/erlang.

Elixir getting a good story around webdev (phoenix and liveview) and more recently the a good story around ML is going to increase it's adoption, but it's not going to happen overnight. But maybe tomorrows CTOs will take the leap.

You can read the transcript.

It's over training syndrome, caused by too much training without enough rest. The athletes that try to push through struggle to perform as well.

P.s. to make that function an actual generator you could use

yield from

Instead of

return

It depends what you are doing with the value.

If you are going to iterate through some of the resulting thing but not all of it then the generator means you aren't throwing away a bunch of the work that you've done.

It can also be more cache friendly. It doesn't need to allocate a whole new lists worth of memory.

One of the downsides of it being lazy is that if list x is mutated between when you create the generator and when you consume it then those changes are reflected in the generator.

I've done some micro benchmarks and it really depends on what you are doing. Profiling it with cProfile, Pyspy or using %timeit in an Ipython shell will tell you if it makes a difference.

What other books did you read that you would recommend?

Like instead of the "free software movement" the "software freedom movement"

they like it because

# title

this is the paragraph

is more natural to write than:

<html> <head>...<head/> <body> <h1> title </h1> <p>this is the paragraph</p> </body> </html>

when you're just jotting some thoughts down.

and data structures. Theres a book called Algorithms + Datastructures = programs which i've heard good things about.

I've also heard good things about 7 programming languages in 7 weeks, as well as crafting interpreters.

The latter 2 I intend to pick up this year

unless it paints a target on your back

I had a series of interviews at a big pharma company that was like this.

Multiple stages, conversations with team members,tech tests, psychology and aptitude tests. The only reason I turned the offer down is because they took several months to get me a contract and I didn't stop interviewing in that time

it depends what speed is most important to you.

When i was a scientist, speed was getting the code written during my break, and if it took all afternoon to run that's fine because i was in the lab anyway.

Even as i moved more into the software engineer direction, and started profiling code more, most of the bottlenecks come from things like "creating objects on every incovation rather than pooling them", "blocking IO", "using a bad algorithm" or "using the wrong datasctructure for the task". problems that exist in every language, though "bad algorithm" or "using the wrong datasctructure" might matter less in a faster language you're still leaving performance on the table.

> "Python is so slow that we have to write any important code in C. And this is somehow a good thing."

The good thing is that python has a very vibrant ecosystem filled with great libraries, so we don't have to write it in C, because somebody else has. We can just benefit from that when the situation calls for it

I think the thing with python is that it's always been "fast enough" and if not you can always reach out to natively implemented modules. On the flipside javascript was the main language embedded in web browsers.

There has been a lot of competition to make browsers fast. Nowadays there are 3 main JS engines, V8 backed by google, JavaScriptCore backed by apple, and spidermonkey backed by mozilla.

If python had been the language embedded into web browsers, then maybe we would see 3 competing python engines with crazy performance.

The alternative interpreters for python have always been a bit more niche than Cpython, but now that Guido works at microsoft there has been a bit more of a push to make it faster

There is a tsoding daily video with hot code reloading in C [0].

He dicusses techniques for this. one of which is a "migration" approach, similar to database migrations. Where, for example, you have a version number in your struct. and a function for migrating from an old version to a new version. I don't think he implements it in that video though.

[0] https://www.youtube.com/watch?v=Y57ruDOwH1g&t=15s

the only apps that i have with notifcations are, as you mentioned, instant messaging apps and apps that i'm using to build a habit (language learning, meditation that sort of thing). everything else is turned off

The creator, gingerbill, uses it to build real time flame and smoke rendering [0]. AFAIK they have vectors and matrices built into the stdlib as well as various graphics apis[1].

So I'd say from what I know the niche is "batteries included high performance code that can target the gpu"

[0]https://jangafx.com/software/embergen/

[1]https://odin-lang.org/news/major-graphics-apis/

They have really great pensions and mat/paternity leave and those sorts of benefits.

You get some lifers that work there but they also get a lot of contractors and people that leave after a few years for a private sector job with more pay but worse pension.

Source: work as a contractor in UK public sector