1,244 karma · joined January 12, 2012
No.
Mature software tends to accumulate requirements ("1) must work and 2) must be fast and/or small"), that are different than the original requirements ("1) must work and 2) must be easy to modify because I'm still figuring it out").
When those requirements eventually become more important than the original ones, the way you code it changes to match. It's a natural progression.
Top-notch programmers find ways to keep all 3 requirements, to the extent permitted by the problem space.
So now we have the situation where an unelected, private body (the American Psychiatric Association) wields immense power over people's daily lives.
This is the problem the author is alluding to in this passage:
> On some level we’re to imagine that the American Psychiatric Association is a body with real powers, that the “Diagnostic and Statistical Manual” is something that might actually be used, and that its caricature of our inner lives could have serious consequences. Sections like those on the personality disorders offer a terrifying glimpse of a futuristic system of repression, one in which deviance isn’t furiously stamped out like it is in Orwell’s unsubtle Oceania, but pathologized instead.
"Are you now, or have you ever been, a member of the Lavabit Party?" [1]
"Your Honor, respected members of the jury: In 2013 Mr. Karana donated funds to an organization known to be in collusion with terrorists, as designated by the State Department and the Department of Homeland Security. He is by no means an 'innocent man' as he claims in this trial." [2]
[1] http://en.wikipedia.org/wiki/Mccarthyism
[2] Assume a trial in 2025 completely unrelated to this topic, with the Terrorist Sympathizer designation coming in 2023 after the leadership of the fundraising organization was taken over by people you've never heard of.
(edited for formatting and grammar)
Security nerd mode activated; solutions like this make me a little twitchy, even when I have to employ them myself.
At the risk of stating something you already know, for the sake of pedantry the security implications of this fix are (at least) as follows:
- If you're checking the signatures of the packages you're downloading, this is probably OK, since even if an attacker spoofed your DNS to route to her own package archive, she would still have to compromise the package signing key to run her code on your system. On top of that, if you're using a hosting/PAAS provider, she'd have to compromise their DNS infrastructure first as well. - If you're not checking package signatures, then hopefully your system doesn't have any "interesting" information (including username/password combinations that might be useful on your or other sites). The hosting/PAAS provider DNS system is still a barrier, but now you're down _two_ of the protections on the chain of code executing in your name.
As always, there are multiple-order-of-magnitude differences in the amount of effort any given element of security is worth; the above fix might be just fine for 99% of applications, while for the remaining 1% some extra thought would be worthwhile. TBH I have no idea how common such "code hijacking" attacks are in practice -- if any "real" security professionals have that info, I'd be curious to hear your thoughts.
Offered in the spirit of helping folks with managers asking "why can't we just turn off SSL?"
On the other hand, if he honestly felt that Snowden did the right thing but is saying the opposite in order to keep his job intact, you could say there's a different kind of "irresponsible" going on.
Hypothetical: Should prospective S/TS clearance recipients be asked if they would reveal classified information if they thought that NOT revealing it would undermine the Constitution? It would be similar to asking a prospective soldier whether he or she would obey an order from the Commander in Chief that clearly violated the Constitution (e.g. "assassinate the Speaker of the House").
Such questions just aren't asked; but that's the question Snowden asked and resoundingly answered.
Static typing gives you useful things (with a trade off), not having to write tests isn't one of them.
I disagree; static typing is, effectively, having an automatic set of tests automatically run for (by the compiler) you that you don't have to run/create/maintain yourself. As someone who's moved from statically-typed languages to Python (2.x), I can't count how many times I've made errors that "should/could have been caught for me by the compiler", if there were such a thing in Python. It slows me down in nontrivial ways. It would be good if we could use 3.x-series annotations to achieve much of the same thing, but the world hasn't gone 3.x yet.Your point about the value of unit testing in general is well-taken, however.
Analysts were warned to follow existing “minimization rules,”
which prohibit the N.S.A. from sharing with other agencies names
and other details of Americans whose communications are collected,
unless they are necessary to understand foreign intelligence
reports _or there is evidence of a crime_.
The NSA got a green light to dragnet Americans' communication on behalf of the FBI. Simply stunning.It looks like the "pure markdown" part is built into reveal.js now: https://github.com/hakimel/reveal.js#external-markdown
Might this also address your requirement to read new slideshows (via browser refresh)?
https://maps.google.com/maps?q=cap+chat+quebec&hl=en&ie=UTF8...
Yep, it's the 4th most powerful known meteor impact: http://en.wikipedia.org/wiki/Ren%C3%A9-Levasseur_Island
I love Wikipedia.
You're right that cost/benefit analysis is hard for transformative things like this, but the citizens throwing their hands up and saying "I don't care WHAT it costs" is like Pavlov's bell to politicians.
To clarify my points a bit, the optimizations I alluded to (in "highly optimized internal codepath") were meant to include things like using a generator, i.e. at no point is there an actual array of input random numbers. The fact that in numpy the 300-element "array" and the 3,000,000-element "array" had identical timings suggests exactly that; I disagree that it's an issue of internal representation, unless the concept of a numpy array subsumes the concept of a generator, in which case I think we're all saying the same thing.
That kind of optimization is only possible in this case because by the definition of randomness nobody could know what the values were until they were enumerated, so it's 100% transparent to use a generator. That's not how real-world data works, hence my forced-native-array measurement and pudquick's reply.
Test 1: Boring, small array of integers
In [28]: arr = range(0, 300)
In [29]: %timeit [(arr[3*x], arr[3*x+1], arr[3*x+2]) for x in range(len(arr)/3)]
10000 loops, best of 3: 27.2 us per loop
In [30]: %timeit numpy.reshape(arr, (-1, 3))
10000 loops, best of 3: 45.2 us per loop
In [31]: %timeit zip(*([iter(arr)]*3))
100000 loops, best of 3: 6.25 us per loop
This roughly matches the article's timing ratios, so far so good.Test 2: Use numpy's random number generation to get a small array of floats
In [32]: arr = numpy.random.ranf(300)
In [33]: %timeit [(arr[3*x], arr[3*x+1], arr[3*x+2]) for x in range(len(arr)/3)]
10000 loops, best of 3: 54 us per loop
In [34]: %timeit numpy.reshape(arr, (-1, 3))
1000000 loops, best of 3: 1.06 us per loop
In [35]: %timeit zip(*([iter(arr)]*3))
10000 loops, best of 3: 39.7 us per loop
numpy is two orders of magnitude faster here; it's evidently using a highly optimized internal codepath for random sequence generation, which I'd guess is a common thing to do in numeric analysis. I assume it's using a generator, so there's no actual array being created, blowing up the CPU cache lines etc.Test 3: Verify that analysis by interfering with numpy
In [36]: arr = [x for x in numpy.random.ranf(300)]
In [37]: %timeit [(arr[3*x], arr[3*x+1], arr[3*x+2]) for x in range(len(arr)/3)]
10000 loops, best of 3: 26.2 us per loop
In [38]: %timeit numpy.reshape(arr, (-1, 3))
10000 loops, best of 3: 48.5 us per loop
In [39]: %timeit zip(*([iter(arr)]*3))
100000 loops, best of 3: 6.55 us per loop
Yep.Test 4: Larger data set, no interference
In [40]: arr = numpy.random.ranf(3000000)
In [41]: %timeit [(arr[3*x], arr[3*x+1], arr[3*x+2]) for x in range(len(arr)/3)]
1 loops, best of 3: 624 ms per loop
In [42]: %timeit numpy.reshape(arr, (-1, 3))
1000000 loops, best of 3: 1.06 us per loop
In [43]: %timeit zip(*([iter(arr)]*3))
1 loops, best of 3: 335 ms per loop
The numpy time doesn't change at all from test 2 despite the larger size, but the others suffer. Again, I suspect numpy is being intelligent here; my guess is that it doesn't actually apply the function and generate the real output, it just wraps the random generator in another one.Test 5: Larger data set, interfering with numpy
In [44]: arr = [x for x in numpy.random.ranf(3000000)]
In [45]: %timeit [(arr[3*x], arr[3*x+1], arr[3*x+2]) for x in range(len(arr)/3)]
1 loops, best of 3: 321 ms per loop
In [46]: %timeit numpy.reshape(arr, (-1, 3))
1 loops, best of 3: 354 ms per loop
In [47]: %timeit zip(*([iter(arr)]*3))
10 loops, best of 3: 83.6 ms per loop
There we go; we're back to roughly the original timing ratios.So, surprise! You always have to measure. Measure, measure measure. My bias is to write code first for legibility and modifiability, and then optimize hot spots if needed (and add comments, please, when you do so).
Without doing deeper analysis I'd say one moral of the Python story is, this shows the potential power of generators. But in real-world data sets this isn't always ideal -- is it faster to load up the whole data set in memory and blast through it, or load it from disk on demand with a generator? In really high performance scenarios, is it faster to preprocess the data to fit into the CPU's cache lines? You can't tell without measuring, and you have to measure in the environment you're deploying to, since the answer may be different on a machine with 1GB RAM vs. one with 128GB RAM, or 32KB L1 cache vs. 8KB.
Potentially even worse, since it's so insidious, is that the fear of getting put on "the list" can have a chilling effect on the perfectly legitimate activities of loyal, law-abiding citizens.
Suppose you're a person of color or muslim, who happens to be interested in learning about the enemy (e.g. Al Qaeda). One obvious way would be to look at their forums online. But wouldn't you be at least a little bit afraid that someone's watching for people browsing to those sites, discovering your ethnicity and background, and slapping you onto the list?
From the other side, why wouldn't the watcher put you on the list, just in case? There are no negative consequences for the individual doing so (because no oversight/warrant procedure), and what if he/she "missed the next terrorist" (i.e. maybe you, as far as they know)? It's a no-brainer and could even be rationalized by otherwise well-meaning people as being patriotic. Imagine the poster on their cubicle wall of the burning WTC towers, with "NEVER AGAIN" underneath.
This warrantless, unsupervised paroxysm of fear has got to stop. We are Americans. We do not live in fear. We do not sacrifice our freedoms for security. We are willing to accept that some of us may have to die because we exercise those freedoms, and we know that the answer to "What if there's another 9/11?" is "Then we will survive and triumph through that, too."
Precisely, and I agree with your disagreement there. They tout "zero deaths from terrorism" as though that were the primary objective. It isn't. I'd rather die in a terrorist attack while living free than live in fear of the state.
On the other hand, it's impossible to have a society anything like ours if you build around zero-trust systems. If you say that fundamentally nobody in power can be trusted, for any length of time and under any system of checks and balances, then you may as well advocate true anarchy or survival-of-the-fittest.
Without an external mechanism -- as external as the courts are to the Justice Department -- we're putting our trust in men and not in laws. This nation's foundation is the rule of law, not the rule of men. "Trust us" only works when there's another trusted adversary overseeing the trusted, who are themselves subject to scrutiny.
Were I to use Haskell in a real-world project with teammates, I'd have to keep an iron grip on the complexity of the code the team writes. Maintainability (mainly readability/comprehensibility as far as I'm concerned) is far more important for the kind of work I do than theoretical purity and hypothetical applicability to classes of problems the team isn't facing.
I could easily keep things on the rails on my own, but I suppose it's simply unsettling thinking about using Haskell for something sizable; probably due to my own lack of experience with it. I'd love to hear about sizable real-world Haskell projects, since I think FP is the future of software engineering.
I think you're being awfully generous to HN...
But seriously, the key takeaway from articles like this one is that marijuana acceptance is finally becoming mainstream; five or ten years ago this kind of article would have been considered quite controversial for a major news organization like CNN, and may have been quashed at the editorial level.
Thankfully, it's only a matter of time before it's legalized and we can empty our jails of people who should never have been there in the first place for possession. The sooner the better.
But I suspect that the main reason why it's hard for most people to talk about his work is that they lack the historical context to understand what he's talking about. For someone who has never studied or used FORTRAN, ALGOL 60, or any of the macro languages he alludes to, his criticisms seem very abstract. For someone who has seen those languages, they're concrete and visceral.
Looking back twenty years ago, I now understand that one of the highest-value courses in my CS degree was the "Survey of Programming Languages" class. We spent two to three weeks studying and working with each of LISP, FORTRAN, ALGOL-60, Smalltalk, and a couple others I don't remember. I enjoyed the class but at the time I wasn't developed enough to think more than "man, people sure have come up with some strange ways of doing things; ok, back to C, I love my filesystems class."
Now I recognize how precious that exposure was. It's kind of like traveling; exposure to different cultures teaches you as much about your own culture as about theirs. You don't always realize what assumptions you're making until you see other people making different assumptions.
This is the value of formal CS education, i.e. Dijkstra's life work. Some people ask whether a CS degree is worthwhile when the Internet makes it so easy to learn how to program. It's the difference between university and vocational training; if you only want fix cars, you just need some skills classes and time spent apprenticing in a mechanic's shop. If you want to be an automotive engineer, you need an engineering education. If you just want a job, you don't need college; if you want to participate in the core of what our civilization has to offer, you need an education (self-study or formal, doesn't matter).
(Tip for the kids in school today: If you're at university and you find yourself frequently saying "Why do I have to learn this crap? I'll never use it!" then you might just be wasting your time and money. Do yourself a favor and drop out, unless someone else is paying for your play time. But if you learn for the sake of learning, if you recognize that learning how to learn, to prepare for a lifetime of learning, is the point of education, then stay in school, since you'll reap the rewards many times over.)
I'm still trying to figure out how Python projects scale to any appreciable size; I suspect maybe they don't. I've worked on several million-line codebases in statically-typed languages, are there any truly large Python projects?
Any positive or negative response on a request-by-request basis
would allow our adversaries [China] to accumulate information
and draw conclusions about NSA's technical capabilities, sources,
and methods. Our adversaries are likely to evaluate all public
responses related to these programs. Were we to provide positive
or negative responses to requests such as yours, our adversaries'
compilation of the information provided would reasonably be expected
to cause exceptionally grave damage to the national security.
This is completely accurate. If the NSA actually responded to these requests, it would be trivial for an adversary (e.g. China) to probe the extent and nature of the information the NSA collects, just by having their agents regularly file FOIA requests.My main problem with these programs is the lack of oversight and any kind of public controls to prevent abuse by bad actors. I don't mind when judges issue warrants to gather private information; that's oversight, with a level of public visibility eventually involved. Warrants are accounted for right there in the fourth amendment.
But these programs are known, thanks to Edward Snowden, to have shoddy controls over individual analyst access to the information. That alone reveals an institutional bias inside the NSA against respecting citizens' privacy; if they treated it as a really big deal then Snowden wouldn't have been able to gain access to individual information as he has claimed. Consequently I have little faith that justice would be served on analysts or departments who use the information for their own purposes, and that's the core problem for me.
We know the FBI treated Martin Luther King as a potential terrorist. If the NSA's information had been available to them, why wouldn't they have asked for it, formally or otherwise, if there were no negative consequences?
What does that mean for political activism given today's more tightly integrated Department of Homeland Security?
Someone who secretly has access to Warren Buffett's movements and communications could get a jump on the market, stealing a little bit from all the honest players.
If you're protesting the government's eminent domain seizure of your home, but you only have enough money to pay the lawyers for another couple of months, the government can just wait you out. They'll be able to figure it out just by looking at the outside of the bill collection notices sent to your home, and the fact that you sent email to several lawyers who are known to work pro bono.
The problem is that the mere existence of this kind of information will draw the corrupt among us like flies to a corpse. I'm not all that concerned about the personal motivations of the people who currently have access to that information, although power does corrupt. I'm more concerned about people who will start working for those agencies because they're intrigued by the possibilities.
If the information must be gathered, then it is essential that it be a really big deal for any analyst to get access to someone's communication metadata and movements. High visibility, court oversight, logging and review, the works.
Side note: I went to the Austin Restore the Fourth rally today; couple hundred people I'd guess, mostly middle-class-looking. Pleasant energy. It's hard to tell whether that's enough of an indicator to make politicians start wondering about their campaign pocketbooks; I'm not optimistic.