It's a lockdown. I could see myself spending the weekend doing this.
482 karma · joined December 25, 2016
What I see is a bunch of internet trolls getting together and trolling Wall Street.
Is that a bad thing? It depends. If you're just dicking around and you put in an amount you can afford to lose, it's all fun and games. But when you convince newbies to trading and investing to put loads of their hard-earned money, on leverage, to a meme stock, it's not so fun and games anymore, is it?
I'm all for teaching hedge funds a lesson by squeezing them, but it would've been nice if folks come out _mostly_ unharmed financially. Instead, you have lots of these r/WallStreetBets folks in the red on leveraged trades. Good luck making back that money as quickly as you lost it.
That does sound very different than Python. Admittedly, I haven't used Julia, because I never had to for professional purposes. For personal projects, I use PyTorch, because I find it easy to use, intuitive, and I have a lot of experience with it. Maybe one day I will be forced to use Julia and will appreciate what it brings :)
You can easily find out if you like it or not, you can use the C++ API for TF/PyTorch and you'll have a statically typed program running your model for you. Sure, it's not Haskell, but C++ is statically typed, and although it's not the most popular or loved language on HN, you can use it to get stuff done.
That's kind of my point. Most of the community has models running on GPU's and don't care too much about CPU-bound workloads, training or otherwise.
If you do care about that, you are in a relatively small niche of machine learning, statistically speaking. I am not denying it's existence, I'm just saying that your stack will have to be different if you want to extract the maximum level of performance out of your CPU.
> IIRC, a maintainer of a popular project/framework that I contribute saw improvements upwards of 2x when using C++ over python.
That's not surprising at all. Like I mentioned in the parent, if you profiled your code and you found that Python function calls are the main bottleneck, and you believe it to be worth investing time in getting rid of them, you can use the C++ API's of Caffe/TF/PyTorch/Whatever.
I personally don't work in simulations so I haven't ran into your problem. In the deep learning world, the CPU is unusable for any task (training, inference, evaluation, etc.), so I've never been concerned with things like function call overhead.
I don't understand why it's not possible. You're asking if it's possible in the language. I don't see anything stopping you from, as you say, writing your own framework, or simply doing it in TF. Perhaps my ignorance of Julia is showing :)
[0]: https://www.tensorflow.org/guide/create_op
[1]: https://pytorch.org/tutorials/advanced/cpp_extension.html
What problems does it solve for you? When building ML applications I mean. Because that's what S4TF was supposed to be.
ML researchers produce models that are used by ML applications for a particular purpose. The code to train the model _could_ be statically typed, sure, but I really don't see what the improvement would be. It would be more verbose, less readable, have significantly more noise. Just try reading some TensorFlow C++ code equivalents of their Python API.
From the "WhySwiftForTensorFlow.md" document (https://github.com/tensorflow/swift/blob/main/docs/WhySwiftF...):
.. static types: * can catch bugs at compile time instead of runtime. People get very frustrated when a silly type error brings down a long training run hours into it. * directly improve tooling experience like code completion/intellisense, jump to definition, etc. * make it easy to know what operations are Tensor operations.
> can catch bugs at compile time instead of runtime.
OK, but what bugs? For most use cases of TensorFlow that I've seen, if there is a type error somewhere, you will catch it long before it hits production, and if you only catch it 100hr into training, your training code is either non-deterministic or there's some other issue that is not type related. Unless we're talking about dependent types a-la Idris, I don't see how Swift's type system can help catch these kinds of bugs.
> directly improve tooling experience like code completion/intellisense
Sure, but type-hints provide the same improved experience with language servers like PyLance [0]. Not a reason to jump to a new language.
> make it easy to know what operations are Tensor operations.
Easy. Anything that starts with "tf." is a tensor operation.
Yes, I'm being a bit trite, but I was never convinced by the premise of S4TF, because I've never heard an ML researcher or engineer say "static typing will fix my problems".
[0]: https://devblogs.microsoft.com/python/announcing-pylance-fas...
But for applications that require a GPU (i.e, most ML applications) cutting over to Swift from Python will likely win you nothing in performance, and wouldn't be worth it at all.
I've been using Python for ML for the last 3 years and I've never felt this way. It might be that I'm not all about the hip new languages, but I don't really see the benefit of making Python more ML/Haskell-ish.
The ML use case for Python is roughly as follows: you design, train, and evaluate models. Then, if something is decent enough to use in production, you switch over your application to load and use that instead. I don't really see where Haskell or any language from the ML family can improve in that process.
Sure, the code that you used to implement your model may improve slightly, but I don't see that code improving significantly. The fruit of your labor is usually a protobuf file (encoding the TensorFlow graph) or whatever your framework uses to encode the model you built. The code actually surrounding it is very minimal for most use cases.
> In Julia, things are really small and composable. For example, you have a probabilistic programming library like Turing and a differentiable programming one like Flux, and it's trivial to implement some Bayesian neural networks.
There's nothing stopping you from composing things in Python. But it's simply not a goal of 99% of ML libraries to be composable with other libraries. You're probably never gonna run TensorFlow and PyTorch in the same application (trust me, I've tried, it was a nightmare) and I don't see why you would compose a TensorFlow model with a PyTorch model without incurring tons of overhead in your application around gluing these two things.
And that something better is certainly not Julia and it's definitely not Swift.
> Yes, I know the heavy lifting is done by C/C++/Rust/Cuda/Blas/Numba and so on, but, when you run simulations for millions of steps, you end up with billions of python function calls.
For ML purposes, if your model isn't running on the GPU, it doesn't matter if you're using Swift, Rust, or whatever, your stuff is gonna be slow. Like it or not, Python is one of the best glue languages out there, and the reason why libraries like TensorFlow and Torch are not used in their native languages (C++) [0] is because they're significantly simpler to use in Python and the performance overhead is usually one function call (e.g run_inference(...)) and not billions.
If you find yourself writing a simulation, and you need to optimize away billions of function calls, you can use the C++ API provided by TensorFlow.
Since ML has been a large rage for the last few years, and TensorFlow is one of the most popular frameworks that ML engineers use (although I use and prefer PyTorch) it seemed like the "perfect storm" for Swift to break out of the iOS mold.
But it was clear to me (and many of my peers) that while this was an interesting project, the need for it is really low, and that nobody is going to throw away years of experience with libraries like TensorFlow and PyTorch (via Python) in the trash just so we can use Swift, which for me isn't really that crazy of a language as all the hype makes it out to be, and I'm sure the same is true for many ML devs and researchers alike.
If you do the legwork, you can get mypy[0] type-checking done on your codebase in a way that is similar to how TypeScript does type-checking. There are stub files that are provided by project maintainers that are then used by mypy to infer things like e.g the return type of a function.
Type hints are also inline in the code, and not technically comments. They can be retrieved from class attributes using facilities from the standard library [1] and can facilitate other tooling that is specific for your project or that are more general.
> And even then, if you're so thorough with your typing, why not just use a proper statically typed language?
That would remove a lot of the benefit of choosing Python. Python is dynamically typed. Type hints make it possible to do type checking statically, but with a lot of extra leg work (as I described above). Making Python itself statically typed is not something that would interest 99% of the Python community IMO.
[1] https://docs.python.org/3/library/typing.html#typing.get_typ...
One thing that always makes me laugh though is the terminology in math. Things like "hypercube" just get a chuckle out of me.
I didn't know what the sensitivity conjecture is but your explanation using coloring a graph is very straightforward.
Last time I wrote any non-trivial Haskell was in 2014, so a long time ago, but I found that my biggest problem with it at the time was the really huge variety of Haskell in the wild. If you're doing simple stuff, you probably stick to the prelude and you'll be happy. But if you're doing anything that's a bit complex, you'll end up seeing hundreds of mini-dialects of Haskell in the wild, so much so that I found it really difficult as a newcomer to understand code on the net. In many cases it's almost like a different language completely, what with the user-defined infix functions, tons of currying everywhere, laziness, and the like, made it very difficult to follow code paths.
However, saying "just get GitLab and deploy it to your own server" glosses over the huge time sink it is, especially for small companies that are already short-staffed, to maintain something like that. I sure as heck do not want to be responsible for keeping my GitLab server up.
As you say, if you're writing an issue, put it in an editor issues.md file or something. If you're working on code, even better, just commit locally.
Classic example of buying a good using BTC: say it costs 100 satoshis. By the time I click and purchase, and by the time the BTC arrives at the seller's address, the value may have changed for the better (BTC price goes down - hence I paid "less") or for the worse (BTC price goes up - hence I paid "more"). For the most extreme example, imagine how people that bought a pizza with Bitcoin when it was worth 5$ feel right now. A 15$ pizza in 2009 or whatever is worth ~150K USD now.
We liked it because the chat history you see is essentially a deploy history, no need to login into some other website to check some obscure logs page to see who did what. We did end up having to debug the service that processed the chat messages maybe once, but never ran into an issue when we had to deploy a hotfix.
But I do agree with you. Cheaper laptops means a worse experience with almost everything.
Shame that EA acquired them.