Convert curl commands to code in several languages | Hacker News Reader

Convert curl commands to code in several languages

curlconverter.com

262 pointsdjha-skin86 comments

Original author here. Many smart people have contributed code over the years, but one warrants special mention.

About a year ago, verhovsky showed up out of nowhere. He rewrote the core of the application and increased the professionalism across the board. (dedicated domain, github page hosting, UI refresh, privacy improvements, and much more)

The tree-sitter PR is a monster achievement: https://github.com/curlconverter/curlconverter/pull/278

Search for parseAnsiCString in there. I don't think that had ever been implemented in JavaScript before.

For you, verhovsky, 10x engineer might be an understatement. Thank you!

how about C ... after all curl is just a frontend to libcurl?

Check out the `--libcurl` argument to curl.

https://everything.curl.dev/libcurl/libcurl

On one hand: "Nice! This is a neat functionality and I learned something today."

On the other hand: "Oh how far have we strayed off the Unix way..."

Seems to be mostly doing text-replacement and not actually understand what it's handling and what it can't actually do: https://github.com/icing/blog/blob/main/curl_on_a_weekend.md (not my article, saw the link on twitter today)

E.g. it turns curl ftp://host.com into response = requests.get('http://ftp://host.com')

Anybody else having reached the point of language indifference/boredom?

Everything I everywhere now see is just the same: Gluecode, library calls, unnecessary verbosity, Blackbox inside Blackbox etc.

And this project slaps it again into my face, with every example it’s literally always the same - only that language uses a colon, the other a semicolon, the other catches exceptions with ? etc. but in a way it’s all just the same.

How to overcome this?

I find, only pointfree notation has some beauty to it that makes reading code sweet again.

This project is a curiosity. It may help someone trying to build a quick one-off script (though, then I have the question of why curl wasn't good enough). In any real-world use-case, you can't just plop in some isolated code to make a HTTP request. I haven't looked yet, but I assume this website generates code that blocks the current thread or it spins up its own threadpool; neither of these things are acceptable for any production application.

In my experience, the main concepts behind application architecture remains the same regardless of what language you're using. Concepts like asynchronous execution and how it affects composability and resource management transcend syntax.

When you master the hammer you can use it to build stuff, it’s not a competition to know every toy language in existence.

> It’s not a competition to know every toy language in existence.

I would argue against this and claim that it appears to be a competition since “X written in {Rust,Go,SexyLang,…}” became the prevalent bullshit bingo.

Sure, however if you play a different game you can get good wage on $UNCOOL_LANG (is that Perl now or what?)

Do you want to win bingo (chance) or get steady-pay?

Prevalent is just what you're paying attention too - there are 359 other degrees to enjoy.

Juggling chainsaws is more impressive than hammers. But it’s all just juggling

The question you have to ask yourself is: Are you juggling for the sake of juggling or to make a good show for the crowd?

If your goal is the outcome, maybe don’t go into juggling bags of poo even though it’s more challenging than chainsaws because they keep changing shape.

I think people want the ability to modify software they use (that's the whole point of open source). It's easier to modify software in a language you know than one you don't. That's why these things exist.

Going back to the hammer example, imagine that you've spent your whole life using screws. You are probably going to buy furniture assembled with screws instead of with nails, because you know how easy it is to twist the screw out and put your modification in instead. A clear path exists in your mind, whereas with nail-based furniture, who knows?

Why is language indifference bad? You're correct, most language differences really come down to one language using a colon, the other a semicolon, etc.

Find a problem you feel passionate about solving and then use your skills to go solve it for mankind. You've learned how to be a mason, now go build something beautiful.

> How to overcome this?

You start with branching out to actually different programming languages. If you already know PHP, don't go and learn Java or C#, you basically already know those from PHP, you're only missing knowledge of the public API (what methods should be called when, naming conventions) and so on.

Instead, seek languages that when you see them the first time you say "wow, I don't understand anything" and sit down and try to learn one of those. For me, Clojure was always one of those languages that kindled the "what the hell is that" feeling in me, but after reading a lot of praise for the language, I started learning it anyways.

Now I work on Clojure(Script) code full-time, and programming is finally a fun thing to do again.

I won't try to learn Common Lisp or anything like that next, as I know that if I spent the time, it'd be easy, it's just Clojure but with different functions to call, different conventions, but mostly the same. So the next language I'm targeting is Rust. A C-like language, but new concepts around the borrow-checker and lifetimes, something I've been exposed to before, but not this literally.

>I find, only pointfree notation has some beauty to it that makes reading code sweet again.

I haven't written FORTH code in ages, but I still enjoy reading well written FORTH code for its pure literary pleasure!

https://github.com/MitchBradley/openfirmware/blob/master/for...

I've found programming language differences less and less interesting the longer I do this. They all do the same thing. Everything's pointers and stacks, usually not that far beneath the surface. Some are more or less ergonomic, but nothing (that sees any widespread use, anyway) is really very different. Just another set of quirks and (probably bad) tools to learn. Library availability & quality matters more to me than what the language looks like.

Instead of exploring different languages learn about different fields and domains. The language may be basically the same (outside of some more esoteric ones), but Data Science is a very different experience from Web Dev which is different from Embedded Systems. Get out of your comfort zone a little.

I prescribe Prolog, Lisp, Forth, x86 assembly, and Smalltalk. Each one is remarkably different and not just a coat of paint on top of C (which is 80% of popular languages). You'll have so much fun.

What about APL, or J? Would these be something different enough.

Work on solving problems using declarative languages and functional languages.

So, either Prolog, or Lisp.

Structure and Interpretation of Computer Programs is a good jumping off point for Lisp while also providing fundamental insights.

Prolog.. honestly, just google "basic" array problems and try and do them in prolog. It takes a mental shift, which is hard at first, and you feel silly not being able to count to 10. But learning how to think in this paradigm is useful - e.g. SQL is declarative.

Maybe try programming for Uxn.

https://100r.co/site/uxn.html

chop wood, carry water... or maybe the Turing version: read a symbol, move the head

Seek excitement not from the language, but from what you can build with the language

With the feature of browsers to copy requests as Curl and now this, it's funny to see curl commands be(com)ing some kind of lingua franca for HTTP request representation!

Now you can play a request and boom, you have your code to make this request in your favorite programming language.

It's interesting and I like it. It does seem to be rather opinionated without saying so. The PHP example requires an http library I hadn't heard of (rmccue/requests). Copy/paste of that code doesn't work unless you know how to find and install that library. The related tool they reference, however, does work using the PHP curl library.

I presume they were trying to keep the code to 4 lines or so. The curl library in PHP is rather verbose and requires 5 to 8 lines (depending on if you try to catch the error).

https://www.php.net/manual/en/book.curl.php

At the time we were using that particular library heavily at WizeHire. I had built the tool for personal convenience. rmccue/requests was likely one of the first "generators".

I expect that we'd accept a PR for PHP curl library implementation. Having a non-third-party option would be valuable.

I am on my phone so I can't contribute a PR right now, but here are tons of PHP examples: https://ntfy.sh/docs/publish/

Also check the other pages on the docs.

Here's one example:

   file_get_contents('https://ntfy.sh/mytopic', false, stream_context_create([
    'http' => [
        'method' => 'POST', // PUT also works
        'header' => 'Content-Type: text/plain',
        'content' => 'Backup successful '
    ]
   ]));

Someone contributed a PR that replaced the PHP output with libcurl

I wrote http-translator which is one of the linked related projects. It has a smaller feature set, but is just a few hundred lines and can be easily extended to support new frontends (input formats other than curl) and backends (i.e., output languages).

https://ryan.govost.es/http-translator/ | https://github.com/rgov/http-translator

Mine was a project for learning modern-ish JavaScript, and I found that packages for parsing command line arguments in JS were generally poor. I wrote my own shell lexer that is simpler than Eric S. Raymond's in the Python standard library while passing the test suite, and since Burp Suite generates curl commands that use Bash ANSI C strings, I support those too. https://www.npmjs.com/package/shlex

If I were to do further development, I would ditch my curl command line parsing in favor of building curl with Emscripten and having it generate the full HTTP request, then leverage the existing request parser.

curl already supports a —-libcurl flag that generates equivalent C code, and it might be easy to extract the curl_easy_setopt() calls from it.

Both our projects should probably support the built-in urllib.request for Python; Requests isn’t always necessary anymore.

Few other similar open source projects:

1. https://github.com/mholt/curl-to-go

2. https://github.com/incarnate/curl-to-php

Curl-to-Go also understands JSON payloads and will even generate the struct definition for you to fill in, so you aren't working with opaque JSON strings.

And if you look to their page you will find an even more complete list:

    curl-to-Go
    curl-to-PHP
    HAR-to-curl
    http-translator (Python, JS and JSON)
    uncurl (to Python)
    hrbrmstr/curlconverter (to R)
    curl-to-postman
    Insomnia
    Paw-cURLImporter

I use Paw[1] for this, which is obviously a lot more powerful by allowing me to easily toggle and/or modify query params, headers, edit body as structured JSON / XML / application/x-www-form-urlencoded / multipart/form-data, etc. Probably doable in other API clients like Insomnia[2], too.

[1] https://paw.cloud/

[2] https://insomnia.rest/

Postman enables you to convert your HTTP requests to several languages as well.

As a Java hobbyist, I'm dying seeing the difference between the Java boilerplate and the Python two-liner

This is not representattive of modern Java, which has much better APIs. Since Java 11:

  public HttpResponse<String> fetch(String url) {
    HttpClient client = HttpClient.newHttpClient();
    HttpRequest request = HttpRequest.newBuilder(URI.create(url)).build();
    return client.send(request, BodyHandlers.ofString());
  }

But of course, nobody uses the standard library since it came late to the game, and instead there are at least half a dozen popular HTTP libs commonly used (Apache, Jetty, Spring, OkHttp, Google, etc.). It's not too different from the logging clusterf*ck in Java, where there are now 9 ways to do it, and none of them are compatible with each other.

A few years back we used slf4j for compatibility between Java logging libraries. Have they managed to break it? Or are you after compatibility at a lower level?

Slf4j is definitely the way to go, but it is still VERY hard for typical enterprise devs to get it working correctly, which means that both your own code and then all the dependent libraries are all on the same page w/ regard to what level and format the logs are printed, and to where they're going.

The problem is that even if your own code uses SLF4j correctly as a logging API, and then wires up the correct logging implementation (e.g. Logback, Log4j2, etc), and then finally include the necessary SLF4j "bridge" libraries, you still need to ensure that every 3rd party transitive dependency explicitly excludes Commons Logging, Log4j, etc libraries.

It's a mess, and almost none of projects that I've worked on over the years got it 100% right the first time.

That being said, I see the exact same problem in a lot of the modern Python and Node code that's out there. It just becomes too complicated because there are too many logging options, and when a modern project - in any language - now relies on dozens of 3rd party libraries, then it's likely you're ending up with 5 different incompatible logging implementations in your final runtime, all doing their own thing.

It is a perfect example of the famous XKCD "multiple competing standards."

* https://xkcd.com/927/

I feel like more granular options would be useful here like "Python with beautifulsoup" or "PHP with Guzzle".

Why use Guzzle at all when you can use the PHP libcurl bindings directly? Guzzle is translating to libcurl either way...

You can say that about any abstraction layer. Why use php when we can just write everything in c?

Here's the one I often use for Ruby. https://jhawthorn.github.io/curl-to-ruby/

Quite useful. Thanks!

I expected C with libcurl on the list, but in a sense that might be the toughest to implement.

Maybe it’s hard to implement, but it’s implementation is a standard part of the curl tool:

$ curl —-libcurl curl.c ...

Seems like it should be pretty straightforward. curl_easy_init(), ..._setopt(), ..._perform(), ..._cleanup() covers the use cases I think.

libcurl only gets weird when you're threading multiple fetches or integrating with a GUI toolkit or something. Even then it's not hard at all to integrate with GTK unless you need absolute peak performance or are handling tens of thousands of simultaneous connections.

Re-using curl's C code and compiling to WASM is the dream.

libcurl does stuff like open files for reading, so you would have to at least modify the code to just pass the file name, you also have to modify the struct.

libcurl parses an array of strings, but curlconverter gets a Bash AST (or a list of strings if you use it from the command line). When there's a bash variable in the command, we want to generate code that gets that environment variable at runtime, e.g. `os.environ['MY_VARIABLE']` in Python so the struct needs to store pointers to AST nodes instead of strings/booleans/ints. Though to be fair curlconverter doesn't work this way either (yet), we convert AST nodes to strings/booleans/ints after command parsing and store that, then when we're generating the code if we see a $ in a string we assume that was a variable in the input AST.

Sorry, it's too late to delete my comment but I realize now that you were asking about generating C code, not which language the project should be implemented in. "Curl already has argument parsing code, why can't we just call it?" is something I've thought a lot so I assumed someone would also suggest it here.

I haven't touched it in years, but I made a utility to do the opposite in Go. Start with Go's request and transform it into a curl call. It was helpful for testing at the time. Looking at it, wow, that was 7 years ago!

https://github.com/sethgrid/gencurl

Something a bit different but I'm a huge fan of NSwag recently.

https://github.com/RicoSuter/NSwag

We use it on a project at work in C# and I've also been using it personally on pet projects with TypeScript.

Allows you to automatically generate REST API clients based on Swagger/OpenAPI specs and is very customizable.

Really nice if building out microservices and you're wanting to communicate between each other or building web apps and want to communicate with APIs. Saves a lot of time writing code to make requests and handle different status codes etc. Automatically generates all the classes/interfaces with sane naming schemes etc.

Even has a GUI application to generate the configs also if you wish.

I agree with the comments saying it should say "Language + Client library" instead of just language, as there are many ways of doing requests with different libraries.

Also, funny enough, the first command I tried didn't work as I expected, but thinking about it, it also seems hard to implement correctly. The cURL command I tried was "curl -v https://google.com/" but none of the implementations include verbose logging.

Weirdly, JSON is listed there, and it is absolutely not clear how JSON can be considered a language that can make HTTP requests. I wonder what client library that is?

The JSON output just dumps curlconverter's internal representation of a parsed curl command. You could use it if you want to use curlconverter to parse curl commands and then do stuff with that parsing yourself in a language other than JavaScript. You would install the command line tool then use your language's command-running library. For example, from Python:

    import subprocess
    import json

    result = subprocess.run(['curlconverter', '--language', 'json', 'example.com'], capture_output=True, text=True)
    command = json.loads(result.stdout)
    print(command)

I always thought this would be a cool feature to have in a http library. The library knows the best way to convert a curl command to its own internal representation. Something like requests.from_curl()

The go version seems to be missing timeouts. There is none by default and the --connect-timeout flag is not setting one either.

Rightly so. Unless you specify `--max-time` in your cURL invocation, all the examples should be missing timeouts, as cURL by default doesn't have any timeouts, only if you specify `--max-time N`.

This looks great, was working myself on same. Good that i saw the link without wasting anymore time.

Had completed UI part as of now https://news.ycombinator.com/item?id=30642254

I work on this tool (but I'm not the original author), feel free to ask me questions.

I have a similar tool for Go:

https://github.com/89z/format/tree/master/cmd/net

I hope there is a better way to stringify / parse these headers and stuff; curl options seems be the best way there is right now. Unfortunately Insomnia etc. does not parse these.

I've gotten a lot out of Chrome debugger's "copy request as cURL" (among others) functionality. I gotta like seeing more of this kind of thing in the wild.

If you have postman you can import curl commands and export them with a lot more options.

That being said turn off all tracking in postman. I got a creepy sales email once asking about my usage and they had a lot of details that made me feel creepy.

I just noticed yesterday that Postman has functionality like this, with over 20 options for the language or language+framework used.

Awesome work! I'm amazed at how many low hanging easy but really really useful projects are left to be done! It seems infinite

The amount of boilerplate Go code for a simple GET is astonishing. I love Go, but sometimes I miss the succinctness of Python.

While Go is definitely more verbose, Python benefits from decades of development of common libraries (requests, attrs, click, SQLAlchemy, Django ORM/REST framework, etc) that have become the de facto standard - there is a LOT of code hidden behind those simple interfaces and the hyper-dynamic nature of "clever" Python libraries make them mind-crushingly hard to grok.

I've developed in Python for >6 years, with tons of deep exploration of the common libraries, and been working in Go for like 3 months - in terms of my ability to understand them, I'll take most Go libraries over Python any day.

Also note that the Python example naturally has no exception handling, while Go "has to" handle the errors (enforced by convention and linters, you could always just throw them away)

also it's only one more line, but they didn't even bother to print the response

once you look at the fact that Go has to have `func main()` declared, the response printing, and the error handling, Go only has a couple more lines than Python

I'm struggling to get VBA to accept a .pem client cert as well as a username and password. Wish this has VBA.

I'm a bit dissapointed it doesn't convert into raw curl module function calls in PHP.

I came here to say this. It assumes that you have rmccue/requests installed, which is an odd choice since that library has 3% of the downloads that Guzzle has. Not to mention the native cURL module that I also expected to be used

Yeah, this is a really odd choice. I find https://incarnate.github.io/curl-to-php/ (found in the footer) way more interesting since it uses PHP's cURL lib

It does now, thanks to @iamdual https://github.com/curlconverter/curlconverter/pull/332

I want to see C++ which I would guess stars with importing asio.

Why should you do this in Java:

httpConn.getResponseCode() / 100 == 2

instead of:

httpConn.getResponseCode() == 200

204 satisfies the first check but not the second.

If you intend to do anything with the response you'd probably handle 204 differently otherwise you'll get the world famous null pointer.

I guess so it accepts all 2xx codes and not just 200?

Oh now I understand: its to match all 200 HTTP codes like 201, 202, 203 etc.

Because all 2xx responses are success; for simple (not Range or other special request scenarios) GET/HEAD it probably matters less in practice than for PUT/POST/DELETE, but 203 and 205 (and 204 for an existing but empty resource) are still possibilities.

I like `curlify` to convert from `python requests` to `curl`

Where is haskell :) ?

I'm surprised it doesn't support Javascript

But it does?

JSON?

Love curl converter!

They even have some resources to let you turn into python, go, and a few other languages.

That is very cool!

Reply on news.ycombinator.com