Requests-Scala: A Scala Port of the Popular Python Requests HTTP Client | Hacker News Reader

Requests-Scala: A Scala Port of the Popular Python Requests HTTP Client

github.com

52 pointslihaoyi31 comments

Looks well designed, and Li Haoyi generally writes great software. Still, it seems the main use case for this lib is “simple, synchronous API where you don’t care about concurrency.” Personally, I wouldn’t use Scala at all for these problems. I use Scala a tonne in high load services, that make 100s or even 1000s of requests per second - for these, Play WS with it's non-blocking IO works great. For projects where small, simple, and synchronous makes sense, I personally reach for Python, not Scala.

I still think this is a nice library to have, but I don’t see it becoming big. It’s serving a use case that I think most people aren’t trying to satisfy when writing Scala.

In my experience, for most jvm apps non blocking io is a premature optimization that makes little performance difference. I once optimized an app, for instance, by switching to non blocking io, and was only able to go from 1000 rps percore to 1200. I got a better performance improvement by switching out the service's json library for a faster one, and that took far less effort. For the most part, I see non blocking io as a fad. It can make a noticeable difference in Performance, but in practice most apps are so poorly optimized that there are easier, lower+hanging fruit.

Fair enough, threads are indeed cheaper than many realize, even JVM threads, which are basically OS threads. But projects where I use Scala do tend to include a lot of service calls that are worth making concurrent (for example, hitting 3 services at once for a single request, or looking up 20 entities at once from a service that doesn’t have a bulk endpoint). And if you’re going to wrap this library with future calls, properly configure your threadpool, etc., you may as well have just used the very mature Play WS, and also get better efficiency. The APIs are pretty similar aside from Play WS returning Futures.

This would have been very helpful to me about two to three months ago. I was doing an interview assessment during that period and I was required to do it in Scala, although I had no experience in it. My experience with HTTP requests comes mostly from Ruby, Python, and JavaScript, and when I was looking for analogous solutions in Scala I found the ecosystem lacking. It took me way too long to do something I knew how to do already in multiple languages.

I know people sometimes like to say the JavaScript ecosystem is a mess, but it being popular and having multiple ways to do things was something I learned to appreciate during that assessment.

Reqeusts is a library which offers a drastically simpler API than the built in solution, and as a result the API has been ported to multiple languages. Are there other examples of libraries offering simpler APIs which got heavily cloned?

I only know Superagent for Node.js, which is really simple, but I don't know if it's simpler than Requests.

Off topic: Is Scala still being adopted by new teams? I did couple of Scala dev jobs and later the market kind of died down, want to know if others are witnessing the same.

Good question, since Google’s adoption of Kotlin was seen as the death knell for Scala.

Seen by people who understand nothing about either language then? The vast majority of Scala developers have nothing to do with frameworks that are "adopting" Kotlin. What Google does with Android is completely irrelevant to us. And I'm not sure Google uses Kotlin at all internally.

Kotlin is pretty much irrelevant outside Android, and even there only because Google most likely will never bother to move beyond Java 8 and Kotlin is the Android's Swift, regarding language replacements.

I wonder if Dart will supersede Kotlin now that Flutter is getting attention on mobile.

Eh... Kotlin sucks as a language but lets you access high quality Java libraries.

Flutter is getting some attention, however no one outside the respective teams, knows actually where Google is taking Flutter and Fuchsia.

And lets not forget Android also means Auto, Wear, Things and ChromeOS integration.

Dart is a successor to Javascript, not Kotlin.

Dart is being positioned as a cross compiling, Android and iOS (but not web) development language with the Flutter platform. It very much seems like a replacement for Kotlin's usage on Android, but only if you are also switching to this UI framework.

https://medium.com/dartlang/announcing-dart-2-80ba01f43b6

It has a very long road to travel regarding OS APIs and the Android team evaded the question at Google IO about what was their opinion regarding Flutter.

Additionally, according to the Chrome team, also at IO, the way to do iOS and Android development is via PWAs, not Flutter.

This is just Google playing with their teams.

These are two completely different things.

The use of Kotlin on the server side is probably zero.

The use of Scala for actual production Android code is close to zero.

We're using Kotlin in our backend and pretty happy about it. It has support for co-routines and great syntax, it's lightweight compared to Scala and IntelliJ plays great with Kotlin (no surprise as both of them are from JetBrains).

I won't even mention the learning curve, we started learning Kotlin using IntelliJ's support for converting Java files to Kotlin initially and then it took only one week to learn about the details thanks to its documentation.

Scala is useful in big data and ML, apache spark is written in it. It works very well for data transformations

Even if you omit all the activity related to Spark and ETL, I see nothing but growth in libraries, frameworks, job openings, meetups and conferences.

I personally love the language, its rare to find a language that supports both object oriented and functional programming styles, while at the same time being able to access all the libraries available for java. Its definitely easy to shoot yourself in the foot given the complexity of the language, but if youre a senior engineer and dont need to worry about poor coding practices on your team, the functional abstractions are very powerful.

It is however the push recently in Spark seems to be for better/faster Python support more than anything else. Probably because Databricks makes it's money from it's hosted notebooks environment which caters more towards non-engineers (who in turn would be more put off by having to learn Scala I'm guessing).

It’s really easy to find the answer. Search Scala here: https://stackoverflow.com/jobs

One of my friends was relocated recently with a very good offer and the only programming language he knows is Delphi. So I’m pretty sure programming languages never “die down”.

Yes. Scala market is smaller than some other programming languages but it is still growing.

For programmers, the important change in the market isn’t programming language adoption. It is expectation to work with multiple programming languages. Majority of good opportunities in the market are going to ask for programmers that can work with two or three programming languages.

Scala's developer mindshare is going down according to John De Goes [0]

> "Scala is receding on numerous observable measures"

[0] https://www.reddit.com/r/scala/comments/8xreuv/keynote_the_l...

This guy has an agenda. I would check his numbers yourself.

The math doesn’t check out, rankings contradict them (check TIOBE, or google indices, or even StackOverflow rankings). It’s sad to see this kind of conclusions in HN because they are not not only incorrect but dishonest.

Nice, would have been exactly what I wanted last year. Ended up discovering sttp which is also really nice.

btw: since you were one of the early adopters of Scala.js, will/can this be ported to Scala.js

This has a blocking API which is a no go in single threaded Javascript, so my guess is no

Super easy to wrap into futures or IO to make it asynchronous. It is scala afterall, not python

I'm not sure how that would work in Scala.js without redesigning the library to have an async API, at which point it's not really the same library anyway. The author said that sync API was a major feature of this lib.

Does it leak memory due to improper CRL handling like the python version?

Reply on news.ycombinator.com