Convert Curl Syntax to Python Requests | Hacker News Reader

Convert Curl Syntax to Python Requests

curl.trillworks.com

166 pointskapkapkap45 comments

Author, here. curlconverter gets about 17k unique users per month. The code generated by this tool is probably in production all over the world.

But the codebase needs maintenance! I hired a junior developer (cf512) to do about 40 hours of dev work last month, but that contract has ended and I'd appreciate more volunteers. All the code is open source.

I see a few bug reports down thread. Please open tickets.

A big thanks to the top contributors, csells and jgroom33. Many others have pitched in over the last five years. Also, big thanks to Daniel Stenberg for writing curl in the first place.

Thank you for making this. Wouldn't hacktoberfest tagging help some visibility for contribution?

This tool is amazing. I use it daily. Please don't let it disappear.

I'm not the author of this tool, but I submitted it to HN because I just stumbled upon it after unsuccessfully trying to transpose a complicated cURL from Chrome devtools into python requests for a good 20 min. This worked instantly. Props to the author - https://twitter.com/nickc_dev

I've been looking for something like this for a while but I never wanted to spend the time actually searching when I could just spend the time converting it manually. Thanks for submitting this.

There's a bit of a hidden feature in Postman that can do this and way more, hit the code button in the right corner and you can convert your web requests to something like a dozen languages: https://i.imgur.com/0qUV8b9.png

Insomnia REST Client converts cURL commands into request sessions too.

Paw, too

I've also written a tool that converts curl commands to Python Requests or JavaScript XMLHttpRequest code. It is tested with curl commands generated by Safari, Burp Suite, and Charles Proxy. It can also translate raw HTTP requests, so it can be used when you have a packet capture.

The design is modular and separates the frontend (curl) from the backend (Python), so more input and output formats can be added. It tries to be smart about generating "clean" code, so it will, for example, remove the Content-Length header when it can be recomputed from the request content.

https://ryan.govost.es/http-translator/

It is also open source. Feedback or pull requests are welcome. https://github.com/rgov/http-translator

Is it all client side?

I wonder how many cookies/secret headers will be pasted by unaware and unknowing users.

This is neat, but also seems like a code smell. Do people really need to convert between HTTP request formats/languages much? When I see many independent implementations of an N-to-M mapping, it looks like the perfect use case for a standardized interface, like a DSL.

In fact, can't HAR do this? I'd love to be able to just pass a HAR string to any HTTP client library, and have it execute that. Or call "dump_har()" (or "--dump-har") on any client, and have it spit out HAR that I can take it to any other client.

When reverse engineering or automating an API my workflow will often look like this:

1. Open browser inspector, watch API requests for one I'm interested in

2. Right click, "Copy as cURL"

3. Paste into a text editor, remove some unnecessary headers, re-running the curl command to verify that the headers I removed aren't important

4. Convert into python requests so that I can play with the parameters

100% this. I do a lot of small automations against various API's (or sometimes against systems that don't API's at all) and have used this same workflow many times.

Also handy because even well documented API's don't have examples in every language, but most do have examples in curl. So you can easily take those example curl's and dump them out to something programatic in your language.

Right, and Firefox and Chrome both have "Copy as HAR". That should save a step, right?

Plus, you probably have a text editor that knows about JSON, so it's easy to (say) remove keys from a JSON file, or even just keep the keys on separate lines and add/remove entire lines. I haven't seen a text editor (even Emacs!) that was as good at editing shell command parameters. It's a lot easier to quote special characters for JSON, too, compared to quoting special characters for my shell.

HTTP is the standardized interface. If you wrapped it in a prettier DSL, people would still have to take raw requests and figure out how to make them in the DSL.

Postman also has the ability to import a curl command (from chrome dev tool) and then export to many programming languages.

I made a similar library! https://github.com/northisup/curlit (sorry for the poor documentation!)

OK, I'm not going to lie. I didn't even know you could copy requests from the network tab into curl (or fetch/ powershell/ etc.) like that. Embarrassing! :) Seems pretty useful!

One of my favorite parts is it grabs the cookie as well.

This is especially handy on Windows, where Chrome failed to even produce proper cURL commands in CMD format [1].

[1]: https://bugs.chromium.org/p/chromium/issues/detail?id=658956

[2]: https://bugs.chromium.org/p/chromium/issues/detail?id=798498

Here is a similar site that translates curl to Go:

https://mholt.github.io/curl-to-go/

Paw, which many people love, has a neat and similar feature: you can get extensions that convert the request you build in the app to any format: curl, python, etc.

It's very handy.

https://paw.cloud/docs/getting-started/code-generator

The "uncurl" library[1] does something similar and can be used in the terminal. I've found it extremely useful.

[1]: https://github.com/spulec/uncurl

I wrote a subroutine macos python to do the opposite.

On macos, requests is not included (although it can be installed with a little effort), but curl was there.

Note that you can easily automate interacting with a web page using the developer menus in safari and firefox.

monitor the network requests, then for the request you're interested in, use "copy as cURL'. You can copy/paste that to invoke curl in the same way the web page used it.

(sometime I have to use --cookie-jar /tmp/cook --cookie /tmp/cook)

Oh wow, not just Python - quire a wide array of languages are supported. Very nice, this will be handy for prototyping things.

reminds me of a little clojure macro i wrote to demonstrate "power" of macros in lisp: https://gist.github.com/keymone/d7725767dc0425a7bcf9, obviously not feature-complete but sufficed for the demonstration

For those interested in integrating such a feature on their website.

An helper for building API request, for example, there is this great open source project: https://github.com/Kong/apiembed/

It supports a dozen of language.

The capability to convert a network request from one format to another is the main reason I use Postman, which has a great implementation of this and a ton of tools and languages it can do it in. This looks great as well

This tool is incredibly useful, I use it at least once a week. Its incredibly useful to explore a site with network in devtools then be able to quickly turn it into a python prototype! Amazing work :)

Great tool, would be nice to also have a chrome extension for the same that augments dev-tools to reduce one extra step of copy/paste.

Failed entirely to work on my first attempt with:

    curl -XHEAD https://google.com

It is just flat out wrong on:

    curl -X HEAD https://google.com

In that it does a GET request:

    import requests
    
    response = requests.get('https://google.com/')

Clever idea, but my first totally valid example failed (it does OPTIONS, GET, POST, PUT, PATCH successfully)

In case the creator of this tool doesn't read this thread, you can report the issues on GitHub: https://github.com/NickCarneiro/curlconverter/issues

Even if the creator of a tool does read the thread, discussions like these are not the place to report bugs.

`curl --head` (short form: `-I`) is converted correctly though, and `curl -X HEAD` (or `-XHEAD`) does generate warnings telling you to use `--head`/`-I`.

https://github.com/NickCarneiro/curlconverter/issues/new

I independently used this tool a couple days ago. Helped save me a lot of debug time!

oh hey, I wrote the `Go` support for this tool.

How could I forget, PR #100.

Fyi, Go received about 1200 conversions last month. Thanks!

hey... that's super handy. thx!

this is one of the most useful little tools on good's green earth. I used to work for a scraping company and I used it all day every day - load a page with inspector open, find your request, right click copy as curl, paste into trillworks and boom you have the request you want to automate (modulo fiddling with cookies). A+ bang/line of code

Failed for this one

    Curl -vLkXPOST https://www.google.com

Because that's not a "curl" command. It's a "Curl" command.

Also it works if you split out the flags.

I feel like going the other way would be more useful.

I thought the same thing at first, but the site actually gives an indication of why this would be useful. The developer tools in at least some browsers can export a request as a curl command. So I can thus easily get the code to reproduce any request I see in my browser. That would have been useful a few times in the past, and more useful than the inverse.

Reply on news.ycombinator.com