Using Rust for 'Scripting'
chriskrycho.com
chriskrycho.com
Not that this is a bad thing, but it feels like a bit of a far cry from my hacked together workflow of throw code in iPython and write out when I’m happy. I don’t see anything here that makes Rust any more attractive than whatever your multiplatform language of choice is for this use case scenario.
That being said, the code is concise and readable, and the article was a short, pleasant read.
I actually do think that Rust's type system helps a bit, because it forces me to consider weird portability issues (such as the way Windows permits slightly invalid Unicode, which Rust represents as OSString instead of String).
And of course, cross-compilation with MinGW is a real help. Sadly, OpenSSL still poses a fair number of portability issues, since it's notoriously difficult to link statically.
Windows allows invalid UTF-16 while Unixes allow invalid UTF-8.
Not having to deal with differences between MSVC, LLVM, GCC and other obscure toolchains is a blessing.
The differences between MSVC, LLVM, GCC are nothing to worry about, compared with the early days of C89 or C++ARM in the process of being standardized, coupled with different levels of POSIX support.
I took the fact that it was so simple as an excuse to do the more complicated things alongside a very simple piece of code. That way, it's easy to see what the code does and then think about the implications of being able to run something like it (including something more complicated) cross-platform.
I'm wondering what part of the twelve step compilation process was found to be easier than simply installing Python on the target machine?
Does the friend have any ability to audit the compiled binary before he runs it? Can he make a small change when he wants to re-use part of the solution?
Instead the "friend" is taught the bad behavior of running untrusted (and I didn't see any tests, so untested and undocumented too) code in binary format on their machine.
The glob-rename is exactly the sort of problem a script is supposed to solve easily. I feel the author justified the use of Rust and cross-compilation simply to avoid usage of a Windows OS (or VM) for such a trivial task.
If the goal was to avoid installing additional run time/deps on the user's machine it seems that (#1) using powershell would have been the best option followed by (#2) a classic .cmd shell file as second best solution.
Compiling is easy. Configuring path variables and installing interpreters are arcane to non-developers.
Being paranoid in the software you run on your machine is a double edge sword. How many people vet their software before using it, and how much vetting do you really need to do?
It's not untrusted code, it's code that a friend wrote to help you out. That's not at all the same as installing random untrusted code off the internet. I write code to help out my friends with things they want to do - are you saying they shouldn't accept it from me just because they can't code themselves? I can give them the source code, but what good is that to them when they can't program and don't want to?
That part is something that you can control.
Giving instructions for someone non-technical to install Python and run a script with it, and then helping them figure out how to interpret those instructions, is a heck of a lot more complicated.
And yeah, you can diddle around with cx_freeze and then having to spend some time with InnoSetup or WiX to build an installer, and then do the same on OS X but putting everything into an app bundle inside of a disk image instead, and finally on Linux trying to figure out how to bundle something up to play nicely with all of the major distros and packaging systems, and get something to work, though it doesn't support cross compilation so you now need to set up and maintain three different build machines, which is definitely no simpler than the solution described.
All of these solutions have their ups and downs. But being able to relatively easily (modulo installing a linker and a few libraries) cross compile from Rust to a static executable for any of the major desktop platforms is pretty nice, after you've dealt with enough of these funky build and packaging systems or trying to walk people through figuring out how to add Python to their path on Windows.
Training someone to accept binary file and run it without question is just dangerous.
So you just tell you friend "Only run a binary when I physically come over to your house with a USB drive" or "Only when I call you on the phone"
Accepting random binaries from the net and running them is the main software distribution model.
for /r startdir %%i in (*.cha) do ECHO ren "%%i" "%%~ni.txt"Compiling Python to exe from Mac to run on Windows is actually a lot of work, though it's doable (I've done it more than once in the past).
ls -Recurse -Include *.cha | % {mv $_.FullName $_. FullName.Replace(".cha", ".txt")}
Also, worth note that these "twelve steps" are:
- inclusive of literally everything including installing Rust and adding dependencies; this is like including "install Python" and "pip install" for a Python tutorial. The point was to be approachable for anyone. I think it succeeded.
- inclusive of the "copy the executable to hand to a friend" step in both cases, which is intentionally over the top in the true listicle style
- inclusive of a bunch of steps you only ever have to do once to get full MSVC-compatible cross compilation happening for Windows.
It's almost like I was intentionally using this tiny project (which isn't especially interesting in its own right; as many have noted it's a one-liner batch file) to provide a one-stop shop for the basics of getting a cross-compiled setup with Rust. ;)
That was my initial thought reading this. Even the most devout Rust enthusiast wouldn't go to this much trouble to avoid executing 'find' one would hope (yes, find does everything you need here for those wondering)
If one wants to be closer to the metal, Go's cross compilation works out of the box, without all the LLVM/MSVC work that Rust requires (for now, that is). As long as you use the standard library (and packages dependent on the standard library), it will work right away. I've cross compiled Go from my Mac for Windows users when I couldn't get my Python setup to do so, worked perfectly.
(Sure, you can script easy things as shown in other comments, this is a bit more general.)
Re: Go, I've had the opposite experience. So long as you only use pure-Go libraries, cross-compiling works. If you use any C libraries (which last time I checked, include some of the standard library), you run into issues. For me, it was OpenGL bindings that made cross-compiling impossible.
That's not "closer to the metal"; it's reinventing parts of the native toolchain.
But that's only part of the story. Along with typing, scripting languages also had the major advantage of being fast to bootstrap and run (`ruby script.rb`) and were fully "batteries included" with good standard libraries.
These assumptions are certainly true, but they're much _more_ true when you're comparing them against "old world" languages like C, C++, or even Java. All of these require significant project scaffolding just to get running, and have varying levels of useful built-in utilities (C has almost nothing, C++ has the STL and maybe Boost if you're generous, and Java tried to have a useful core, but missed the mark in a number of places like HTTP). In the case of C and C++, complex build logic may also be required in the form of autoconf/automake/make.
Newer compiled languages like Rust just don't have these issues anymore (I'd also include Go, Crystal, and some others in the list). Project scaffolding is built right into the language (Cargo) and the standard library is about as complete as you can get (and probably better than the classical scripting languages — all of Ruby/Python/Perl missed a good API for some things like HTTP, which led to the rise of separate packages like Faraday/Requests/etc). Anything that's not in the standard library is easily retrievable through great package managers that are bundled in with the core language.
After you're over the initial learning curve of the language and standard library, it's possible to be nearly as productive in something like Rust as you can be in Python. As a bonus, you can also get reasonable reassurance that your program is correct before you run it — and even without writing an exhaustive test suite that exercises every line of code.
Certainly Java is kind of a hybrid worst if both worlds kind of thing, but C/C++? For simple scripting problems you don't need complex build logic. You can usually get by with "cc foo.c -o foo", which you can embed in the source file. Sure, if you have library dependencies it can be a bit more involved, but most modern IDE's (even less than modern) will manage that for you. It's particularly simple if you do static linking (which often makes sense with small scripts anyway). These days there are so many header only libraries that can remove even that boy if pain. On the deploy side, it's simple and likely faster to run than Rust.
Now, many C/C++ projects start with auto tools/cmake or similar heavy lifting, and long build times. But there's selection bias there: those projects aren't simple scripting jobs, but actually intend to exploit close system integration and support building with a broad variety of compilers and linkers. They do it because they want people to use the code and for a larger project, it isn't that much extra overhead.
Nah, the main pain with C++ in particular is the lack of a standard, simple library for network services like HTTP & databases. That problem seems to be finally getting resolved, but historically I've addressed by just standardizing on one of curl, cpp-http, etc. Once you do that with a modern C++17 compiler, and maybe their in Folly as well to make it truly easy, it starts to become a remarkably viable scripting language.
You will type a bit more, but the compiler will also catch a lot more stuff for you automatically, reducing the cognitive load. There are better alternatives, but if you are in a shop with C++ already, the advantages of just using the same tool as everywhere else are more than enough to justify it.
This means they know C++ best, as this is usually the language you use in those competitions. It's interesting to see the approach to scripting of these guys. While I tend to resort to shell pipelines and maybe throw in a short Lua program in-between, they write C++ code for everything, escaping to the shell for some stuff while using ordinary text files instead of piping, e.g. system("curl somesite >input.txt") and then fopen("input.txt", "r"). They write the glue between these calls, call the C compiler and run the binary.
They know their C++ and know how to write it fast; as a result, they can get a valid script running about as fast as I can do the same using a shell pipeline.
When I see this, I'm reminded how awesomely tools can be bent to do something they're not primarily designed for, when in the hands of someone skilled who knows them inside-out.
What? Rust basically has no standard library. At least Python doesn't require pulling in third-party libraries to do globbing, create a zip/tar, create a tempfile or send/serve http. Hell, it doesn't even do command-line arguments outside of manually parsing and iterating over the args vector.
>>> import httplib, json
>>> conn = httplib.HTTPConnection("httpbin.org")
>>> conn.request("GET", "/get")
>>> res = conn.getresponse()
>>> print res.status, res.reason
200 OK
>>> data = json.load(res)
>>> print data
{u'origin': '0.0.0.0', ...}
>>> conn.close()
zip = "0.2"
tempfile = "2.1.4"
getopts = "0.2"
hyper = "0.9.12"Second, if you're going to over engineering just look for someone who has already done that for you: http://www.den4b.com/products/renamer
It's no use to engineer something that has been solved (unless you're learning) if there are already better solutions. Even if it is an example a better one should be used.
The moral of the story should he use the best tool for the job, don't shoe horn a bad language for a into a solution, and also Google is usually the best tool for every job.
Overcomplicating is more like it, for engineering implies a formal specification document which includes requirements, as well as documentation to go along with the product, not to mention complete integration with the target operating system substrate.
This quickly hacked-together Rust program is light years away from that.
2. It wouldn't be a bash script; it would have to be batch or PowerShell, neither of which I know well, and neither of which I like.
3. It was fun.
For anyone else curious about Rust, my own learning spree was triggered after reading the "A very brief intro to Rust" Slide Deck [1] by Ashley Williams [2].
[1] https://ashleygwilliams.github.io/a-very-brief-intro-to-rust...
turtle is a reimplementation of the Unix command line
environment in Haskell so that you can use Haskell as both a
shell and a scripting language.
[1] http://www.haskellforall.com/2015/01/use-haskell-for-shell-s...
Also, I'm very excited to hear that LLD is apparently usable for linking on Windows! It seems like we've been waiting for LLD since time immemorial. :)
There are probably faster ways, but none that I could think of a the time.
For future reference, `nmap -p22 --open 192.168.1.*` does everything your rust script currently does.
Nmap is THE tool for the job of network scans. IMO, it should be in the toolbelt of every techie that deals with any sort of computer network.
Other quick ways besides the mentioned nmap: ping the broadcast address (if ICMP is allowed) or install avahi-daemon and you can just refer to "your-machines-hostname.local"
gci c:\somedir -r -filter "*.ext1" | %{Move-Item $_.FullName -Dest ($_.DirectoryName + "\" + $_.BaseName + ".ext2")}
Not true for js on Windows, though. Windows runs JScript natively (same language as JavaScript, only a different name).
It's especially handy for a one-file solution.
First, there are a handful of existing tools, as other have pointed out (I would personally go for AntRenamer).
Then, there are a few scripting languages already installed (admittedly maybe not the best or most well-known). A quick Google search would probably have given a result right away, since this is a pretty common scenario.
Lastly, I am a bit curious, why go with msvc and not mingw/clang ?
For MSVC over MinGW/Clang: part of my point here was to first learn and then document for the community how you do cross-compilation for MSVC. There are lots of projects out there where being able to link against MSVC is important, and it's also harder than linking against MinGW (which would have just been `rustup target add x86_64-pc-windows-gnu` and `cargo build --release --target=x86_64-pc-windows-gnu`, I believe; no extra linker stuff needed), so there's more value in documenting it and putting it out there for MSVC.
Since this is actually harder to do (from your own post), I worry a bit that a beginner might try to do it right away, to the risk of failing at it.
edit: not sure why I cannot write the jolly character here
It's so subtle that even the author might have missed it.
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned
To install a cross-compiler on mac:
$ fink install fpc-cross-i386-win32Single file executable. Job done.
Of course, if you want to write something in Rust, it's not much use.
Nope. The beauty of a decent systems scripting language is that it includes everything you need (and ideally is already installed everywhere).
Get-ChildItem . -Recurse -Include *.cha | Rename-Item -NewName { $_.name -Replace '\.cha','.txt' }http://flukus.github.io/2015/03/13/2015_03_13_Powershell-is-...
I'm working on a similar one using make instead, which uses shell commands and it's a lot better than both.
When I see things like these, it honestly makes me hate computers and IT.
Again: the point was simply to use an interesting "hook" and then segue into providing resources on cross-compiling with a super-simple example.
First of all, why doesn't this have a UI? It would be much easier for the friend to select the folder with the .cha files and they could see a nice progress bar while they are renamed.
However, if there are few files the renaming process would be too fast and the progress bar would flicker. An easy solution is to add sleep calls in the worker thread to ensure that the progress is visible.
Of course the user could have lots of .cha files, and the friend might get bored. A popular pattern is to display an overview of the program features in picture + text format while a long procedure is running. There could be one about .cha file renaming for instance.
It might make sense to also show ads, depending on the monetization strategy. Maybe the friend wants to buy something and they don't realise it yet.
Finally, when the whole thing is done, the friend should be offered the option to post it on twitter and/or facebook. Something along the lines "I have just renamed 100 .cha files using rename-it. Try it yourself at <address>". The address needs to be different from the blog, probably an SPA, but I'm not an expert at that. And one more tip: don't ask for the facebook login before the renaming, this makes users suspicious. It's much better to ask before posting and explain why you need it. This builds trust.
I said that Rust is not the right tool. I would maybe try Electron, it's also cross-platform, but the code can be reused on the back-end. For instance one can upload a zip of all their .cha files and they get a zip of txt files back.
Ultimately desktop apps such as these are a thing of the past. But I guess it's a nice exercise.