What programming languages are used late at night?
stackoverflow.blog
stackoverflow.blog
C programmers start the day a bit later, keep using the language in the evening, and stay up the longest. This suggests C may be particularly popular among hobbyist programmers who code during their free time (or perhaps among summer school students doing homework).
EDIT: this apparently only works on x86-64 Linux. If you are targeting something else, it might be slightly harder.
What? I do the same thing on both FreeBSD and macOS.
Valgrind is a serious savior for me sometimes, it's able to catch leaks, access violations, use of uninitialized memory, use-after-frees (and it can tell you where it was free'd), data races, etc. Microsoft's heap debugging stuff really doesn't measure up unfortunately, on Windows I'm frequently left grinding my face into the ground when I hit such issues. Fortunately they're fairly rare as I generally stick to C++ with more modern techniques which avoid most of the pitfalls - but when they do happen it can get nasty without a tool like Valgrind.
Helgrind is quite good for threading issues too.
If you're interested in C++ tooling, maybe watch through The Care and Feeding of C++'s Dragons[1] from GoingNative 2013. They start to talk about AddressSanitizer at 55m40s, but the entire thing is really good. The tools they talk about have just gotten even better and more widely available since then.
[1]: https://channel9.msdn.com/Events/GoingNative/2013/The-Care-a...
Address sanitizer has caught something on every codebase I've enabled it on. YMMV, but I encourage you to at least try it out and draw your own conclusions.
You forgot "bypass or instrument our custom allocators", "determine repro steps in the first place", "filter out false positives", "bypass or instrument your refcounting - because failing to pair a free() with your malloc() isn't the problem, nor even failing to pair decRef()s with addRef()s - breaking up cycles is", etc.
Sometimes the answer is "force your scripting language to garbage collect again".
Sometimes the answer is "track down an incorrect cast in code that ultimately causes a single byte corruption in your refcount".
Sometimes the answer is far more horrifying.
But I do wonder whether some languages have debugging problems that require "keeping more state" to solve them than others, such that the developers doing debugging in those languages would rather forge ahead and finish, than drop it and pick it back up the next day. That'd be an interesting thing to measure.
It'd also be interesting if some languages attract certain developers, because the attention-span and "mental stack size" the language designers assumed for the language, feel better suited to them. Maybe there are languages better for people who can juggle many or few facts at once; languages better for people who can hold three screens of text in their head, or who prefer to only have to think about one line at a time; etc?
I find (in any work, not just the rare C work) that when I get into a juicy problem I just don't want to stop. It's nothing to do with the task switching / reload cost, it's that I'm interested in what I'm doing.
So I think you're right that it's about what kind of developers a language attracts, but not because it effects the language designers. I think it has something to do with the languages used by people along the "programming is a job" - "code is life" spectrum. Look at Haskell's bump. AFAIK, Haskell is only used by people for whom code is a way of life.
(This is not an endorsement or indictment of either end of that spectrum. Some people who paint are workers, others are craftsmen, others are artists. All use paint.)
That sounds so profound. Did you just make that up, or is it a old paint industry koan?
I used to stay late at work sometimes for this reason - not more than an hour or so but I found that even when I stayed late I often didn't manage to solve the issue that day - instead I came in the next morning fresh and was able to solve it better after having slept on it.
Now, that's true of debugging in my experience, but if you're knocking something out quick and don't want to have to get back into that groove the next day, it's a different story.
for my sideprojects i use mostly C because it is easy and straightforward - you just write what you need without satisfying all the other requirements. At work it is Java, C++, Python.
With the tools, finding and fixing that sort of issue isn't generally very difficult.
Memory unsafety is terrifying, but not much of a time-sink in ordinary development. Only once in my career have I found a memory bug that was difficult to track down and fix (it was a stack overflow in a C++ program on an embedded OS).
The real problem with memory unsafety is that I nave no idea how many memory bugs I didn't find.
Valgrind a lot both with finding and diagnosing memory bugs. But it cannot solve everything.
I've used VBA for stupid things like Excel based apps (not stupid at the time, but in hindsight there was a better way) and stupid things like HMI software with VBA hooks (GE Fanuc Cimplicity, in case you were wondering).
We recently moved our main development to github so the repo seems a bit empty, we're certainly happy to get more contributors if you are interested, as the community is fairly small and specialized. Everyone is welcome.
Source: I'm one of the current main developers of the project.
(.) Note: I actually have no idea if it's the 'biggest'.
Is Ganeti related to Ganpati? Sorry, it's been a bit since I left Google. Memory is fading.
There are only so many words. And many of them similar.
A large part of our jobs is figuring things out.
People looking for so-called production-grade tech are usually looking for ready-for-managers tech in my experience.
However, new versions of Haskell (the language) are created by temporary committees (which are not composed by Microsoft).
GHC (the main implementation) is an open source project with developers from different organizations. According to this page (https://ghc.haskell.org/trac/ghc/wiki/TeamGHC), SPJ is the only one employed by Microsoft. He is one of the core contributors and he owns a large part of the code base, but not to such an extent that GHC can be called a Microsoft project even if Microsoft would want that. Haskell is not a MS product, it has been under development since well before SPJ was employed by MS.
Finally, I'm not aware of MS funding the project beyond paying SPJ's wage.
Exercises are great and all, but once I try to go past exercises, I start getting lost in layers of monad transformers, and I struggle to figure out the "right" way to do anything. It just feels like there are all these layers of abstraction and complexity that are required for even simple applications, and I can't seem to reach the point where it all clicks for me and becomes manageable.
Reading the TOC and chapter notes for that book, I'm not convinced it'll help me. Even the "final project" at the end of the book is described as "a little weird, but small". I don't need more weird, small examples, I need a full-fledged application! I've tried picking out random Haskell OSS apps and studying them, but without newbie-friendly documentation to walk me through the code, I find it very hard to follow what's happening....
You will immediately spot some places where a monad will help you. Where that happens, use the monad, elsewhere, keep it simple. With time you will regret some of that simple code, when that happens, change it (Haskell's best use case is on refactoring). With even more time you'll get a taste for where it's good to go complex from the start.
I agree with you but the book has chapters where you write a URL shortener and the final project is an implementation of the `finger` Unix command. For me, the book was a stepping stone towards materials like Write You a Scheme (Scheme interpreter in Haskell).
I will admit though, I read Haskell Book for the ideas and learning typed functional programming. I wasn't planning on using Haskell everyday for projects.
As an example, the first "real" project I wrote in Haskell was one or two weeks into learning it: a web testing framework/DSL (a la Capybara[0]) that spoke a (custom) protocol over socket to a headless webkit instance[1].
Looking back, I abused stuff like type-classes and such, but then again, it was all a learning experience. I should probably be a little embarrassed about how I left the project, but in the interest of providing some inspiration, here it is in all its gory -- I mean, glory: https://github.com/cstrahan/happybara
[0]: http://teamcapybara.github.io/capybara/
[1]: https://github.com/thoughtbot/capybara-webkit/tree/master/sr...
I tried that. When I got stuck (fairly early on), I ended up switching my project from Haskell to Clojure :/
[0]: https://en.wikibooks.org/wiki/Write_Yourself_a_Scheme_in_48_...
http://haskell-servant.readthedocs.io/en/stable/index.html
Servant allows you to write a type that describes a server/API, and automatically derives a specification for it. You can then write handlers for the different API endpoints, and it will only compile if every endpoint is correctly handled. That includes content types, status codes, and so on, everything is verified, and things are often derived for you. All (de)serialization to/from JSON is automatic (although you can write your own instances if you want your JSON to look a certain way).
For example, from the Servant docs:
type API = "position" :> Capture "x" Int :> Capture "y" Int :> Get '[JSON] Position
:<|> "hello" :> QueryParam "name" String :> Get '[JSON] HelloMessage
:<|> "marketing" :> ReqBody '[JSON] ClientInfo :> Post '[JSON] Email
GET /position/:x/:y/ returns a Position object as JSON
GET /hello?name=whatever returns a HelloMessage object as JSON
POST /marketing given a JSON request that represents a ClientInfo object,
returns an Email object as JSON
You can then write functions of the following types
handlePosition :: Int -> Int -> Handler Position
handleHello :: Maybe String -> Handler HelloMessage
handleMarketing :: ClientInfo -> Handler Email
If you add a new API endpoint, and forget to write a Handler for it, you'll get to know. If you change the Position type to work with x, y, and z coordinates but forget to update your handlers, you'll get to know. If you'd like to also allow clients to request HTML instead for some endpoint, just change the '[JSON] to '[JSON,HTML]. The Haskell typesystem will make sure that someone requesting a text/html Content-Type doesn't get hit with a 500 or something.
Note the Maybe String there: if you wrote a handler for the /hello endpoint that had the type String -> Handler HelloMessage, Servant would complain: you're expecting a query parameter, which the client doesn't have to provide. This is in stark contrast to, say, the "NoneType has no attribute whatever" problems that one risks having to face with Django: if it compiles, it meets the spec.
Of course, static typing won't prevent you from responding to /hello with Lovecraft quotes.
--
> I start getting lost in layers of monad transformers
Handler is actually a monad transformer, and it's a great first one: it's essentially[0]
type Handler a = ExceptT ServantErr (ReaderT Config IO) a
* throw a ServantErr
* read values from a Config object
* perform IO actions (read files, make other network requests, log things, "fire missiles")
After a couple of months, you realise it's not too wrong to say you understand how to use them, and you slowly begin to be able to rely on the type system for support. Libraries like Servant are a great example of how this can really, really help. I stopped to think last week how similar it is to pair-programming with a really intelligent but sometimes obtuse friend who likes pointing out how "this doesn't follow from your assumptions", all the way from
> "Wait, but you can't add two books together."
to
> "What if someone PUTs to /login?"
which is what we had above, to
> "And what happens if I try to withdraw all my money exactly when the payment I've made to you is getting executed?"
(in this case, you discover the wonders of software transactional memory[1]). A better typesystem allows you to let the compiler handle more of the busywork that you'd originally have written tests or comments for.
It is sometimes confusing starting out (when one discovers that 3 isn't an Int, but a "Num a => a", for instance), but it gets better. Once you get past the Project Euler/"look ma, infinite lists!" stuff, "real" Haskell does have a tendency to make people underestimate how much they really know, but as I've discovered, taking the plunge reveals that one has progressed much farther than one thinks.
Also, #haskell on Freenode is one of the friendliest places I've encountered on the internet[0].
[0]: http://haskell-servant.readthedocs.io/en/stable/tutorial/Ser...
[1]: https://www.schoolofhaskell.com/school/advanced-haskell/beau...
[2]: Here's a hilarious example (NSFW language warning): https://gist.github.com/quchen/5280339
--
Other stuff:
Corrode is a C-to-Rust converter written in literate Haskell (as in, it's like a blog post that you can compile and run, to give a slightly strained analogy).
https://github.com/jameysharp/corrode
webshit weekly actually "quoted" a previous HN comment of mine about it, in which I was falsely slandered: I'm not a Rust evangelist!
I wrote a small "script" to make tmux status bars a while back that was a good example of how fun "scripting"-style thing can be in Haskell.
https://github.com/mrkgnao/remux
And there's always XMonad!
> This is equivalent to an API that defines these three endpoints:
It would be cool to use the text that follows this as the spec; i.e. write a converter from this description to Haskell.
> GET /position/:x/:y/ returns a Position object as JSON
Or, you know, a quasiquoter (which is sort of like a Haskell macro thing, although Template Haskell is closer to that).
Depends on how complex you want to do things. You can just use yesod [0] to create a web app and compare it to an existing one you'd written.
An interesting use of Haskell is with concurrency [1], so anything along those lines should give you something pretty awesome.
[1]: http://chimera.labs.oreilly.com/books/1230000000929/index.ht...
About what extensions people consider kosher, Stephen Diehl's "What I Wish I Knew When Learning Haskell" has a great list that sorts them into "common", "uhh", and "you obviously know what you're doing".
That's incredibly easy, of your list of "problems to attack and programs to write some time soon" pick any one and get busy. It'll feel real-world very soon and there's no better way to grasp "real-world Haskell". The first iteration will be ugly, but on the plus side you'll notice that yourself and that's quite some progress already. Keep at it a bit more and perceived "decent post-newb proficiency" is within reach. Well-rounded mature libraries for "real-world periphery" (DBs, json/yaml/html/xml/all that jazz) abound, too.
What I find much more annoying is the lack of freelance projects. It's literally the language of academics, hobbyists, weekenders, late-nighters, githubbers. (Certainly also suitable for bootstrappers imho in a "Viaweb-Lisp-advantage" kinda way) If you want to avoid full-time employ, you're gonna have to keep doing .net/java/golang/node and such projects. Haskell code-bases I feel are private well-guarded silos within academia and a number of corps, no wider "projects ecosystem" to speak of.
http://www.thaiopensource.com/relaxng/derivative.html
This document describes an algorithm for validating an XML document against a RELAX NG schema. This algorithm is based on the idea of what's called a derivative (sometimes called a residual). It is not the only possible algorithm for RELAX NG validation. This document does not describe any algorithms for transforming a RELAX NG schema into simplified form, nor for determining whether a RELAX NG schema is correct.
We use Haskell to describe the algorithm. Do not worry if you don't know Haskell; we use only a tiny subset which should be easily understandable.
You don't have to use the most esoteric constructs the language offers, or get into the weeds of how lenses work (just a random example of a seemingly 'esoteric' library) or language extensions to write code that does something useful.
That said, a project centred Haskell book would be nice to have.
I remember Lisp having a fixed place on that list (pun intended?) for a number of years, copious amount of caffeine, weed, and patience eventually allowed me to grok it. Haskell has been more stubborn in that regard. I get the functional part just fine, but the part where you get side effects and I/O and such confuses me to no end. Maybe I should just give up on Haskell finally and try OCaml, which seems to offer many of the same benefits.
Nowadays Haskell is my #1 programming language for hobby projects—at least when GC is acceptable. And often when I’m trying to design some code in another language, I’ll prototype it in Haskell first to get more clarity of thought and faster iteration.
A contributing factor might be that a lot of French companies include meal vouchers in their benefits. Generally employees are given the choice to deduce some amount from their payroll and get nearly double that amount in meal vouchers, because the vouchers are government-supported including a tax concession. You can use these at most restaurants in Paris.
You can, of course, use the vouchers to grab a sandwich anywhere and eat it in front of your code, but overall it encourages actual restaurant breaks, since you basically have more money specifically to spend on your lunch.
[1] http://www.edenred.co.in/Our-Offerings/Work-life-Benefits/Ti...
To me, who rarely pulls up SO, correlating SO questions and page views with usage is a bit biased.
There's a lot of SO content regarding working around JS/DOM incompatibilities between browsers. Without those incompatibilities through the years, SO might (incorrectly) think that JS usage is lower than it is.
You might not be an average SO user?
I use it constantly. Can you say more about:
1. What your programming set up is.
2. What you do when there is an issue?
I didn't know there was a reasonable alternative. I do the Google -> SO a lot. The only time Google brings up a different page is if the question is REALLY basic like how to use a function.
Edit: added this postscript:
I have found a lot less need to look up things like "are there optional parameters in .len()?" because the IDE helps with that.
I need to look up strange errors that I don't understand. (Like when you install a new library and your code breaks). Those are almost never in the official documentation because they didn't plan for it.
1. What is your development environment?
2. What you do when there is an issue?
Read the manual: https://www.gnu.org/software/bash/manual/bash.html
Just sit down one weekend, and read it from beginning to end. The information contained therein is just about exhaustive.
Assuming you touch bash scripts daily (as I do), there's really little excuse for not mastering the language. Referring to SO every time you stumble into an issue is inefficient compared to the one time cost of actually developing working knowledge of the subject at hand, and it also limits you to the canned solutions that a bunch of random people on the internet have proposed. I'd bet my money on you being more capable than a random stranger -- assuming you've read up first.
When I was at an earlier stage in my career, I'd hit up SO every time I ran into an issue. And then I had the realization that I could just read up on:
* The spec for your language(s)
* The source code for the compiler and/or VMs that power your language(s)
* The manual for the software you use
* The RFCs that apply to you
* Both the public API and the internal source code of the libraries you use
* Etc
I'm not going to claim to be a great bash script writer, but I'm better than 80% of what I see out there. Throwing together random fragments is going to cause nearly as many problems as it's meant to solve.
I'm not setting up a "you hate the shell" statement here - hashing out gripes and dislikes can sometimes be a great way to jump straight into the middle of an iterative positive feedback loop of learning.
(Or at least I think so, anyway. I think that if I did that myself it would identify the (sometimes not intuitive) blocks and glitches I have with things more quickly than I figure them out in the end.)
Oh, also - note that there is a lot I outright hate about the shell myself, but which is due to its, shall way say, overly organic development over the last ~20 years.
I think I'm basically stuck on the worst part of learning curve. Btw I use console for pretty much everything from git shell to vim. But when I have to run-in-parallel/fork two processes, pipe X from some to third one that has to repeatedly do curls to some service and check http response, there are 10 things I need to google, starting from: -how to fork process, how to pipe process, how to IF in bash, how do LOOP in bash, etc. Worst thing is, I've googled this 10 times before, but never remember the syntax etc.
The fact that you're already in the console puts you in a pretty good position: now you just need to stay there whenever you perform complex tasks. I have very few bash scripts myself (the little code I actually have saved is scattered everywhere; most of my scripts are on some disks that are offline at the moment). Rather, I primarily use the shell as an interactive way to let me hop to wherever it is I need to go as a sequence of steps.
Ultimately using Haskell (or Scala) will also definitely get the job done, but with that you have to create a file, hammer the code out (and write whatever boilerplate for forking and execing and piping), then save it and run it. With bash you just type something and hit enter. See what happens, hit the up arrow, edit, repeat. So it's a standard REPL, but this REPL can launch and pipe and fork in much less characters than other languages can.
I'm very curious about the use-case you loosely described. There are a few ways I could interpret what you said; depending on which you mean, what you describe might genuinely not be easy in bash.
If you need to fork two processes and merge their output, that is definitely not easy to do in any shell, AFAIK - you can do (proc1 & proc2) and hope for the best, but output might get mis-buffered and partial lines from one process might get merged with partial lines from other (eg, read(4096) from one might get "hello world" and read(4096) from the other might get "...blah\nblah\nbl" and then you could end up with "blah\nblah\nblhello world" - I've seen this happen once when two processes fight for priority on a terminal). AFAIK this is "not supposed to" occur :) but doing your own line buffering (ie, making a program, like you're already doing) would be critical if you needed this kind of merging.
If you're just doing simple things like running a program that outputs a list of URLs, doing some simple testing on that list, and then doing something else based on the results, that's quite easy.
Here's a highly-contrived script that
- gets a series of URLs
- directly pings all HTTPS URLs via curl and collects the output in output1.txt
- pings all HTTP URLs against https://example.com and collects the output in output2.txt
- parses the contents of output1.txt as a series of comma-delimited lines, then looks for lines that contain "URL" after the 2rd comma, and looks for a URL on the next immediate line after the 1st comma
- only fetches URLs matching the above specification that are HTTPS, chops up the URL from https://example.com/path1/path2.ext to https://example.com/path2.ext, and runs wget on the result
- runs 50 copies of wget in parallel (xargs will start new wgets as old ones finish) on the contents of output2.txt (warning: this will result in a thoroughly messed up terminal if you ever do this, but it works great!)
program_that_outputs_urls | sed -n ',^https://,{p;b1}w/dev/stderr;:1' \
2>(while IFS=$'\n' read x; do curl -s "https://example.com/?url=$x" \
|| echo "$x fail"; done > output2.txt
) | while read x; do curl -s "$x" || echo "$x fail 1"
done > output1.txt
u=0
while IFS=$',' read -a x < output1.txt; do
[[ "${x[0]}" =~ fail\ 1$ ]] && continue
((u==1)) && [[ "${x[1]}" =~ ^(https:\/\/[^\/]+\/)[^\/]+\/([^$]+) ]] && {
wget "${BASH_REMATCH[1]}${BASH_REMATCH[2]}"
u=0
}
[[ "${x[2]}" =~ ^URL ]] && u=1
done
xargs -P50 -n1 -I{} wget "{}" < <(grep -v fail$ output2.txt)
- the sed command first grabs HTTPS URLs (,^https://,{...}), prints them directly (p) then jumps to :1 at the end (b1); if the {} block doesn't fire that means the ,https://, match didn't work, so that means the (w/dev/stderr) runs, which does what you think (GNU sed accelerates writes to /dev/stderr; on other UNIXes that path - or /proc/self/fd/2 - needs to exist for this to work)
- 2>( .......... ) is a bash subshell thingy that spits all of stderr (fd 2) into a subshell (this is also great for "command | ... do; ... x=1; done" situations - you'll lose your "x=1" association from the other side of the pipe. Doing "while ... do; x=1; done <(command)" keeps your "x=1" accessible!
- < <(...) is a way to redirect output from a subshell into the input of another process, similar to the reverse pipe explained above.
- The "2>(...) | while ... done" structure is simply the way I've expressed pushing stderr into the subshell, while pushing stdout into a pipe.
- "while IFS=$',' read" is the way you get `read` to use a different internal field separator. As an aside, I often use "IFS=$'\n' x=($(ls -1))" as a way to fix the "everything in the array is one of the words in the line" problem, as well as similar "unfixable" issues when bash is parsing input with tabs and spaces in it, or when you're using "read -a" to split things into arrays.
- The "if (u == 1) ... done; if (this_is_a_url) u=1" is a standard structure you've probably used elsewhere that implements the "look for URLs on the next line" behavior.
- Nobody's checked the git access logs, but I'm personally pretty sure bash's regex functionality was checked in from a cave full of dragons :) - besides the \/\"\/ picket-fence issues it often has its own ideas about matches. You cannot single- or double-quote strings; your quote will be considered part of the regex (:D :D). On bad days just pipe everything through "awk '/^https:\/\//{ print $1 $2 $3 ...; }'" or similar (yes that command needs editing but you get the idea).
If you want to explain the use-case you described in more detail, I could probably take a crack at figuring it out. I kind of went off on a very random tangent here, but it was fun.
(Bash won't solve every problem (particularly tooling and things that might evolve over time; see also GNU autoconf :D), but it is awesome when it can be used to speed interactive tasks up.)
I've had Haskell on my todo list for a little while actually.
EDIT: I tried to focus on readability here. Things could be quite a bit more concise but that requires firmer grasp of interactions between >>=, <$>, and point-free functions.
Here's what I have so far:
{-# LANGUAGE OverloadedStrings #-}
module Main where
import Prelude hiding (appendFile)
import Data.Monoid ((<>))
import Network.Wreq
import Control.Lens
import Control.Concurrent.Async
import Data.ByteString.Lazy (ByteString, appendFile)
main = do
urls <- promptUserURLs
responses <- mapConcurrently get urls
let responseBodys = fmap (view responseBody) responses
responseInfos = fmap (\(url,body) -> (take 5 url, body)) (zip urls responseBodys)
httpUrls = filter (\(url,_) -> url == "http:") responseInfos
httpsUrls = filter (\(url,_) -> url == "https") responseInfos
mapM_ (appendFile "output1.txt") httpUrls
exampleSiteResponses <- mapConcurrently get httpUrls
let exampleSiteResponses' = fmap (view responseBody) exampleSiteResponses
mapM_ (appendFile "output2.txt") exampleSiteResponses
promptUserURLs :: IO [String]
promptUserURLs = go []
where go :: [String] -> IO [String]
go urls = do
line <- getLine
if line == ("q" :: String)
then return urls
else go (urls <> [line]) -- lol, this is a very bad way to snocThis is very interesting. Does Haskell not have reasonably easy-to-use EOF indication?
I'm reasonably impressed this is as short as it is; being able to shove everything into maps (and get concurrency for free) is nice.
I can understand why Haskell is less widely used though; this is... OH, "take 5" is substring, that's why you did "http:" and "https"!
...I was going to say "hard". :P
Thanks for taking the time to write this!!
{-# LANGUAGE OverloadedStrings #-}
module Main where
import Prelude hiding (appendFile)
import Data.Monoid ((<>))
import Network.Wreq
import Control.Lens
import Control.Concurrent.Async
import Data.ByteString.Lazy (ByteString, appendFile)
main = do
urls <- promptUserURLs
responses <- mapConcurrently get urls
let responseBodys = fmap (view responseBody) responses
responseInfos = fmap (\(url,body) -> (take 5 url, body)) (zip urls responseBodys)
httpUrls = fmap fst (filter (\(url,_) -> url == "http:") responseInfos)
httpsResponses = fmap snd (filter (\(url,_) -> url == "https") responseInfos)
mapM_ (appendFile "output1.txt") httpsResponses
exampleSiteResponses <- mapConcurrently get httpUrls
let exampleSiteResponses' = fmap (view responseBody) (exampleSiteResponses)
mapM_ (appendFile "output2.txt") exampleSiteResponses'
promptUserURLs :: IO [String]
promptUserURLs = go []
where go :: [String] -> IO [String]
go urls = do
line <- getLine
if line == ("q" :: String)
then return urls
else go (urls <> [line]) -- lol, this is a very bad way to snocI'm curious about "(zip urls response" though. Does that extend to the "responseInfos)" on the next line, or am I reading it wrong?
(I noticed the "fmap fst" and "fmap snd".)
Wonders where to start to figure out how this works
Perhaps zip can be more simply explained by a simple example:
out: zip ["a","b","c"] [0,1,2]
out: [("a",0),("b",1),("c",2)]
out: fmap fst [(0,1),(2,3),(4,5)]
out: [0,2,4]
out: fmap snd [(0,1),(2,3),(4,5)]
out: [1,3,5]Also - is [] a list (or mixed type) and () an array (or same-item type)?
If you're doing a lot of successive transformations of data of the same type, putting it in State will prevent you from messing up the ordering (or needing to come up with fresh names).
{-# LANGUAGE OverloadedStrings #-}
module Main where
import Prelude hiding (appendFile)
import Data.Monoid ((<>))
import Network.Wreq
import Control.Lens
import Control.Concurrent.Async
import Data.ByteString.Lazy (ByteString, appendFile)
import Data.List
main = do
urls <- promptUserURLs
responses <- mapConcurrently get urls
let (httpInfo, httpsInfo) = partition ((== "http:") . fst) (zip urls responses)
mapM_ (appendFile "output1.txt" . view responseBody . snd) httpsInfo
mapConcurrently (get . fst) httpInfo >>= mapM_ (appendFile "output2.txt" . view responseBody)
promptUserURLs :: IO [String]
promptUserURLs = go []
where go :: [String] -> IO [String]
go urls = do
line <- getLine
if line == ("q" :: String)
then return urls
else go (urls <> [line]) -- lol, this is a very bad way to snoc
{-# LANGUAGE OverloadedStrings #-}
module Main where
import QuickScriptLibrary
main = do
urls <- promptUserURLs
responses <- mapConcurrently get urls
let (httpInfo, httpsInfo) = partition ((== "http:") . fst) (zip urls responses)
mapM_ (appendFile "output1.txt" . view responseBody . snd) httpsInfo
mapConcurrently (get . fst) httpInfo >>= mapM_ (appendFile "output2.txt" . view responseBody)I must admit I have so many questions, like what the dot does, what fst and snd and >>= and OverloadedStrings and zip do... :)
Oh, and what "snoc" means.
>>= is monadic bind... I won't try to explain it here. It's somewhat similiar to flatmap in Scala which has some easy explanations.
zip, I just remembered I explained already.
I definitely have no excuse, I need to go find a decent Haskell reference-tutorial-manual. Know of any good ones for people with no former experience with functional programming (and ADHD :P)? I'm aware of Learn You a Haskell, just wondered what your experiences were.
1. Read the entire UNIX manual (wasn't that big at the time) 2. Read Rochkind "Advanced UNIX Programming" 3. Read K&R "The C Programming Language" 4. Read K&P "The UNIX Programming Environment"
sometime later I also read (parts of) Stevens UNIX Networking.
I'd say these made a very strong base from which to learning everything else.
If I'm really stuck, I opened 3d party github, and scan issues, to see if there's something similiar. Lots of times there is issue or something, but I'll admit, lots of time there isn't. This is when I start googling, which sometimes take me to Stackoverflow. If not, I use gitter to ask authors. But this happen very rarely. Mostly all I need to do is dictated by library types I use. Occasionally, with shitty libraries that bring external configuration I have to check their docs on github.
But I think it may be language related. Or experience perhaps. I had to do a bit of javascript with node.js (2 weeks +-), and I visited SO very very very often.
Normally whenever I'm learning new language or framework, navigating to sources is my fasttrack to both language and framework/lib. There isn't a better way to learn what's right way to do stuff than to see authors of battle tested libraries solving difficult problems, and learning from their style. I can recommend that :)
I'll admit your mileage may vary, and this may be specific to my job, but I'm pretty happy with this workflow, and I don't have anything against people using SO or something.
Edit: I had forgotten important part - occasionally I spin REPL and try what library really does there. But that's also one of last resorts, when I'm stuck :) In Scala we have http://www.lihaoyi.com/Ammonite/ which makes it super easy to fetch and use external dependencies etc... so I guess tooling is also important part.
I took the former approach, for five or six years now (probably longer), I've been programming with lua. At some point I hit a stage where I don't have to google errors, since they're either logic errors or syntactic errors. Syntactic errors are easy to track down, since either errors are spat out, or the behavior changes in a predictable way. Hitting this stage in lua has helped me to hit it easily in other languages now as well. In my work I use Erlang, and the only thing that consistently stumps me is the architecture of the things that I deal with (Erlang happens to have fantastic warnings though, and everything else can be dealt with via tests)
But those would not effect the results. Presumably a good C# programmer uses less documentation just as much as a good Haskell programmer.
True, but most people I've bumped into who write C# do it professionally, and have built up a body of knowledge. By contrast, most people I know (including me) who have messed around with Haskell, have...messed around with Haskell.
I could be way off here, but I'd guess that there is a smaller percentage of mature Haskell programmers within the Haskell community than mature C# programmers in the C# community.
But that doesn't mean C# users would look things up on SO more from 9-5 than they would after 5.
We are talking about the same people.
(The study, I assume, shows that people use C# for work and Haskell for a hobby.)
> I end in SO sometimes when looking for some discussion on general concepts i.e. advantages of X over Y, but even then it's not always very helpful.
SO can be useful when what I'm curious about is more theoretical and/or subjective in nature, and thus can't be readily attained from regular reference material.
Here's an example where someone (quite helpfully) points out the connection between monoids and monads: http://stackoverflow.com/a/3870310
I knew the definition of monoid and monad, but what I lacked was enough intuition to (readily) see the connection on my own. It's much more convenient to borrow someone else's insight, rather than sit indefinitely meditating on the nature of a given problem. In those cases a resource like SO (or any other forum, for that matter) can be quite useful -- even if you've read up as much as you can.
Now that I think about it why isn't there canonical questions/answers from most voted SO questions? Sure top ranked questions and answers usually are very well explained, but I think there could be more done in the way of removing friction by rewriting them so they adhere to some sort of convention/standard and then maybe turn them into structured types like jigsaw puzzle pieces and you'd only need those to program anything.
I would prefer a generic Developer/Sys Admin wiki. I'd host one myself, but I don't think I can afford that kind of bandwidth/storage at the moment.
For why?
SO tends to be problem solving around programming patterns or unclear errors.
In the languages I have the most experience in, I don't need either. I can usually just reason out a solution to any problem that crops up. I know how to debug, and reduce problems "fluently" in those languages.
Most problems get solved in the time it takes to make a cup of coffee.
Secondly, if you're looking at patterns, like "do x in y", then for new patterns, RosettaCode tends to give you all you need, but if it's a somewhat familiar pattern, it shouldn't take you more than 10 minutes with no guides.
But, truth in it all, SO just isn't that useful for the problems I come up against.
Recent example: Is a Associated List or UNORDERED_map more memory effecient under GCC?
After enough pain, I've decided that stability is probably the most important feature of a tool. Clojure, ClojureScript, MySQL, and Mongo fit that bill perfectly for me. When looking at libraries and APIs, I try to get a gut-sense of how stable they are, and I never upgrade if my only reason for upgrading is there's a new version number out there. Also, I make a lot of sub-projects, so that any breaking-change upgrades I need to make for legitimate business reasons are isolated to one .jar.
I'm also an Ubuntu -> Mac convert for the same reason. My time is just too valuable these days to lose money on tooling, and that keeps me off SO. :-)
Please don't take this as "SO is bad" -- I think it's really great, and have answered several questions there, voted in elections, etc.. But these days, experience and stability in tooling have the side-effect of not much SO time.
I don't use SO much, either. My professional work (presently) consists of two things:
1. Writing web services (can't go into much more detail)
2. Developing (what will soon be released as) an open source framework for service orchestration (also can't go into much more detail)
This is all written in Go.
Technologies used: MongoDB, Kafka, gRPC, REST/HTTP
I'm familiar enough with the semantics/specification of the above tech, so most of the unknowns boil down to (occasional) lack of familiarity with the libs used. In that case, I'll occasionally read the documentation; more often I'll read the generated godocs; and most often I'll just grep/peruse through the code for a couple minutes, and then I'm set for the next couple weeks.
Granted, I have used SO more in the past. In those cases, it was because of stuff like futzing around with Grunt/Gulp in a JS project, or dealing with APIs trying to be too cute via metaprogramming (Ruby, I'm looking at you), etc. It calls to mind a quote[1] that resonates quite well with me:
"One reason people's skills plateau is because they try to master simplifying extrinsic complexity, and not intrinsic complexity."
2) Look at the docs (of which I always keep a local copy), and the source code (of which I try and keep a local copy).
I want to understand not only how to solve the problem at hand, but what caused the problem. Not understanding the cause of the problem will lead to a solution which will only sometimes work, because it misses some other leak in the abstraction.
I believe that a big reason why I work this way is that I started programming when the internet was still in its infancy, and the official documentation and my coworkers was pretty much all I had to work from.
Now then, this isn't always possible, but when I can't figure it out (and my own coworkers can't figure it out - they're my second resource), I find that the SO answers are some combination of wrong, under-documented, out of date, or just have questions with no answers.
As much as I prefer to continue with 2, I do appreciate Google and its ilk when I come upon some really sticky problems - but it only really occurs a few times a quarter.
EDIT:
> are there optional parameters in .len()?
Official docs will answer this very quickly, no IDE required. Of course, having an IDE can be quite useful.
For Python - Standard Library - Built In - length()
For Javascript - MDN -> Standard Objects - Array - Array.length
And so forth.
I've had similar experiences with SO being out of date or kind of poor - when I run across those I have been trying to add a newer answer (I do a lot of Spring so I run across this a lot).
Day job is C#, and C++ mostly, with a good bit of Java, bash, perl tossed in. Personally, I find MSDN sufficient for most of my questions re: C#. For the rest, mostly blog posts, TBH.
Side projects I use Python. Again blog posts and docs are all I need.
I wouldn't say I never use SO, but I can't remember the last time I posted a question and I rarely use it as a resource for looking up information.
Mostly I do Windows/MFC/C++ or embedded stuff these days, for which MSDN is sometimes more useful than SO (although unacceptably slow page load times!). Occasionally there is something with a search resistant name or where the usage examples don't help for which I need to go to SO.
For Windows CE, there is almost no user community. If the docs don't help and the (extremely fragmented) source doesn't help there's no option other than to give up or break out the debugger and reverse-engineer things. I found a bug in Microsoft's "structured exception handling" this way.
It's a lot more useful for C#, which I'm less experienced with.
Professionally, I work at Google and internally we have an extensive corpus of knowledge for various problems and programming patterns that really are tied to in-house technologies that don't really exist outside of the company. That's mostly the #1 reason why I almost never stumble upon SO.
> 1. What your programming set up is.
I code in Python, Bash, Haskell and sometimes Go. Plus a couple of internal languages that aren't really interesting (mostly config languages). I write code with vim and manage my project with tmux sessions.
> 2. What you do when there is an issue?
I look at the logs, look at the code, I look internally for any resources we might have, I spend some time trying to figure out what is going on and usually that takes care of 90% of the problems. If that is not the case, then depending on the type of problem (is it an internal library? is it something externally available?) I either ask on the internal IRC for specific teams in charge of that platform, or I go to the external internet in search for help. Most of the time, if that is the case, the situation is so specific that I can't really find an relevant answer on SO so I more often than not just end up in a #freenode channel for that library/language and directly ask questions to the interested people.
This bears repeating.
READ \clap\ THE \clap\ FUCK- \clap\ -ING \clap\ LOGS.
READ \clap\ THE \clap\ FUCK- \clap\ -ING \clap\ CODE.
I'm amazed at the devs who just scroll up in the logs to the first error they see, then copy-paste that into google or SO.
2. I program in Perl primarily, and have been for over 15 years. There's not a lot language wise that I need to look up. When I forget the specifics of a particular built-in, I can easily look it up in perldoc. If it's a problem with a module, I can use perldoc on the system or use metacpan.org for a better representation with all the bells and whistles. CPAN modules generally have excellent documentation. If I run into a problem, I google it, and sometimes that leads to SO, sometimes not. Sometimes I google knowing that I'll find the same SO entry I have every year for the last few years for that snippet I can't ever seem to remember.
I think the sweet spot for SO helping you is often when you are learning a new language. As you become more proficient, the majority of the questions will likely be things you could answer, not ask.
When I was picking up another language a few weeks ago, however, I constantly ended up on SO with little weird questions. It feels like one of the big hurdles to get over with a new language and tool set is to learn to ask the right questions the right way such that you can actually find the answers in the reference efficiently. SO works as a nice bridge because threads are started (and searchable) by the questions of real newcomers like oneself as opposed to bring written by the experienced who optimize for strict(er) correctness and succinctness.
This is fascinating to me. What kind of issues are you finding solutions to on SO? The only times I ever find myself using the programming SO are if I'm learning a new language or looking up algorithm ideas for a problem.
If there's a linking error or something, the solution (if it exists) is usually on a mailing list, not SO. This may come down to differences in tool selection.
Re. question 1 I mostly use Haskell, Python, and C(++).
Interesting choice of question. When I do have tooling problems, I find that SO is often the first Google search result and I do use it in those cases. However, I don't find that tooling make up a frequent set of problems that need to be dealt with. Thus, while I can't say I've never used SO, it is a rare stop.
More frequently, the problems I run into, and am more curious to see how others have approached them, tend to be tooling agnostic. If you're using C++, Haskell, or Javascript, vi, Notepad, or Visual Studio, the problem remains the same. Computational problems, not tooling problems, if you will. I find (which may just be my failure to find them) in these cases, SO more often than not has nothing of relevance.
> 2. What you do when there is an issue?
Still Google, but the results are more often personal websites and academic papers.
For stuff like "are there optional parameters in .len()?", I always keep a tab or two on both the documentation of the language and the library I'm using at the moment. I really like how some languages come with specific tools for documentation (ri in Ruby, the `doc` function in Clojure) and I try to use them as much as I can when available.
For #2, this is the most common google->SO for me, but typically this happens only when something very unexpected happens, which isn't that common. Typically, my problems are around what library/functions I need and that is usually solved through online reference material.
In this case, the point the article is trying to make is actually something different: "What languages do people use for fun", and I'm having a really hard time coming up with any plausible alternative explanations for their data.
For example, I am attempting to use a namespaced library installed with Pip in my Lambda function. It doesn't work - can't import the library. Google gives a bunch of noise. SO gives a bunch of noise.
The solution? Asked a coworker, got an answer back immediately. Import the 'site' package, and set up the current working directory as a site directory. This makes the library .pth files work properly.
As I said in a cousin comment, it also helps your office/team discover what knowledge is lacking. I used git there, but linux (unix in general) is another example in my office. So many of my colleagues have only done Windows-based applications, or embedded (running a one-off RTOS or an off-the-shelf RTOS). We've been moving workload to embedded linux (for reasons), along with moving some development work to linux (because of tooling available there). If everyone googled it, they may get their answers. But no one would realize that it was a systemic issue because there's no communication.
Coworkers and friends are a resource to be cherished, not abused.
More generally, I'd hate to work in an office where I have to be wary of interrupting the people. That sort of environment is toxic to working in a team. I encourage developers who want to work in a quiet place without being bothered to go somewhere outside of the main office.
C++ devs tend to look at http://en.cppreference.com (using an English locale in this example).
To me, it tends to bias the sample towards the subset of developer population using SO, which would not be a representative sample size for a true population sampling in a statistical problem - it would be oversampled in one group to the exclusion of all other groups.
I have found a lot less need to look up things like "are there optional parameters in .len()?" because the IDE helps with that.
I need to look up strange errors that I don't understand. (Like when you install a new library and your code breaks). Those are almost never in the official documentation because they didn't plan for it.
Besides, with more stable languages (C++ has a 3 yr standardization cadence and new features are not covered well on SO) you can find in-depth answers on SO. Otherwise, many mailing lists (yes, they still exist) are a better source of more current and more detailed treatment of the topic than, "hey, how do I do X so that I can move on to my next thing at work."
Don't get me wrong, I love SO, but I feel it's more about, "how do I solve this problem now?" vs "I want to learn more about this topic." It's real value is in the comments that others contribute "around" the accepted answers.
The C and C++ sections of StackOverflow are good for finding language lawyers who can give you an exact answer, with citations, to "is this snippet well-defined?" type questions (the TL;DR is usually "if you have to ask, then it isn't"). But they'll scold you for tagging those sorts of questions with both C and C++, even though it's perfectly reasonable to do.•
—
• If you're asking about C, it's helpful to know whether your program will still have the same behavior when compiled as C++; if you're asking about C++ but your snippet doesn't have anything C++-specific, it's helpful to future viewers if the answers for both languages are in the same place, since more often than not they're the same (which also means it's helpful to make it clear when they aren't).
:-)
(Not trying to knock SO here, this is a very cool analysis, but I do feel that some of these problems are relevant.)
Haskell naturally attracts a more skilled developer who is more likely to dig into documentation/blog posts rather than running for help on SO.
I used SO all the time when I was a newbie. But I only use it now for obscure issues, usually as a last resort.
Still, most of this particular blog post is about relatives between hours of the day per language, rather than comparisons between languages, so it won't be too affected.
Personal example: I started to learn Elixir, but I don't know where are these communities. So I rely on SO or other open sources/articles on the web.
I'd say based on this graph it, and knowing Haskell myself, that choosing Haskell is still a viable way to attract talent. (that Haskell-spike in the animated chart half through the article is... opportunity I guess)
Github would be the better indicator. Especially if Github could include anonymous stats for private repos too. That would give a good indication about the commercial viability of the various languages.
I'm one of those people who uses Haskell a lot at night time and I rarely use SO for it. I often find answers in documentation, blog posts, IRC, and books. Whereas during my day job I tend to run into a lot of obscure framework errors when I work with Ruby on Rails or the various weird quirks in Javascript where SO is indispensable. I have different kinds of queries with Haskell rather than copy/pasting error msgs, likewise when I use Erlang/Elixir.
Another example is Reddit:
/r/python 122.1k users
/r/javascript 82.4k users
/r/java/ 51.4k users
/r/php 40.8k users
/r/cpp 35.7k users
/r/ruby 31.1k users
/r/csharp 28.2k users
/r/haskell 22.8k users
/r/golang 20.7k users
/r/c_programming 19.6k users
/r/swift 17.1k users
/r/rust 16.1k users
/r/sql 13.6k users
/r/scala 10.4k users
/r/lisp 10.3k users
/r/clojure 10.0k users
But no doubt it's not as popular professionally as other languages, sadly, but it's not quite as fringe as this particular data source indicates.
;-)
I rarely use stackoverflow (like once a few months), how do we factor in the data which we don't have in it? What's the accuracy of the current data? And how can we foem conclusion?
The post should say, "What tags are used to ask questions on stackoverflow at night?" Because, as powerful as they are, SO can't quantitatively decide which languages are used at what time, they don't have the data!! I learned an entire language without ever looking at a SO qn, it was Go language.
Some people I guess are just so bright that they don't need it. That is not most people.
Someone contributing to an official Stack Overflow blog seems like a likely candidate!
Sure, the conclusions are the result of a fishing expedition into the usage data for something to comment on, rather than something scientific that began with a question or tests a hypothesis.
However, while the total number of questions isn't so meaningful, that is the denominator used to normalize the data across languages. Think of it as `n`, where the confidence interval for Java is probably smaller than the one for Sharepoint.
I think it is best to think of this as exploratory analysis rather than a study exploring a specific question. It would be interesting to see further analysis into what puts Paris and London on one end and Moscow and Tokyo on another.
Let's say that Stack Overflow makes programmers more productive, but some cultures might have a tendency to think that it is "cheating" to use the site while at work. Addressing that would be an easy way to increase market penetration. Maybe they should get some submarine articles published in Japanese and Filipino trade publications.
They used IP geolocation to determine the users local time.
It's funny that the runner up is on the opposite side of the abstraction spectrum: assembly. Students, longbeard contractors, and hax0rs must be the nocturnal contingent there.
Meanwhile, COBOL programmers are stuck in bank basements with no internet access.
Maybe because they're often in finance?
Why wasn't Java compared?