How not to check the validity of an email address
dellsystem.me
dellsystem.me
<?php
if (!admin())
echo "<script>window.location = '/signin';</script>";
?>
I've had the same kind of experience hiring a 'real' consulting firms. We were sent a PhD, who banged out a pretty standard looking interface for us that worked terribly. Ostensibly we could've sued, but that just takes more time, and doesn't really fix things.
Anyways, those are the things that a code review would catch.
Sooooo I declined but someone took the job ... for under 8$ an hour.
How much quality can you expect for that price???
Another part of the problem is, developers take that work. I probably would myself. I'd hate myself for it but if I have to take shit pay to make ends meet I'd rather do it coding than not.
An application I've just been "repairing" recently has a spot where it uses two separate queries to pull two full table sized lists of values, then manually joins them with a loop, and then manually re-orders the joined values into groups selectively ignoring some rows, and then embeds the the whole reordered list in a web page. The page takes around 20 seconds to load. Switching it to use a single properly formatted SQL reduced load times to under a second.
Another legacy app I'm employed to "repair" has one single 'template' for every page on the whole site. Its first ~500 lines conveniently consist of a giant and highly nested if/else clause to set the page variables and inline javascript.
Such things are the result of "IT experts," "Software Managers," and "Product Administrators" who've never done real software/web development in their lives hiring random "programmers" who have history or psychology degrees and think they can program because they made a form in PHP.
It only gets lovelier when eventually somebody realizes it's a huge security risk and hires an outside development firm to "secure" it. (Giant eye roll. If they couldn't vet a programmer, you can bet they're great at vetting security consultants and contract developer shops.) Did you know that randomly moving code into folders named "private" and "public" for a few thousand dollars can solve giant architectural and security issues like ridiculously easy XSS and SQL injection?
I don't know what the deal is, but a huge proportion of people writing code are plain incompetent.
At my last company we fired someone who created huge amounts of work for everyone (he thought he could secure page content and alert messages by using base 64 encoding as a stand in for hashing and encryption, for example) and a few months later he was hired as lead developer by a pretty reputable educational business.
... sigh
I'm not old enough to be responsible for stuff like that but I am incompetent enough.
Now that is a great conversation starter! I assume you think you know more than the senior/lead/architect on the team. You might, but have fun with that mentality. It's not sure to last. :)
oh, oh! I'm doing one of those. Only, it's a modern, MVC version, so there's actually a couple of dozen controllers with a single function each, and all actions snake through The Great Maze of Ifelsedom to set their rightful values, before traversing it once more on the page view file
Manager: "Uhm, I thought you said changing that label would be a 5 minute task?"
git commit -m "$(curl -s developerexcuses.com | sed -n '/<a.*>/{s/<[^>]*>//g;p}')" awk -F'>' -v RS='<' '/a href/{print$2}' lynx -dump -nolist developerexcuses.comWe need you to change it back because the system that scrapes that page is relying on the page load time.
That's pretty unfair to people coming from history or psychology who actually can write good code. Just because you don't have a degree in CS doesn't mean your code is shit. This is purely anecdotal, but my predecessor at my current job was a CS graduate and wrote code like in the OP.
One of the real issues has to do with the mentality of coding. There are people regardless of background who approach coding as a job, and those who approach it as a craft. You want the latter, not the former.
Here's my rule:
If you don't look back at code you wrote a decade ago with some degree of horror, you are either an extraordinarily good coder, or you aren't a good coder at all.
The question is:
Are you improving? Or are you beyond improvement?
But what this means is I rarely come in the next week and wonder what I was thinking (it does happen, but rarely). More often I look at things, over a few months figure out better solutions to coding problems and my style changes accordingly.
Then again I started learning ios programming then so I might have a bias/reason for it. >.<
The two most common issues are inability to create sanely defined and decomposed schemas and clearly written queries, and poor ability to structure code in an organized way.
That said, I have known a couple very good coders without any kind of formal education. They just seem to be outliers.
When dealing with formally trained developers I've certainly encountered the incompetent also, but I'd say at an easy majority of trained coders I've worked with are competent enough to not be hard to work with.
Mind I you I do not claim to be a "superstar developer" or anything. The state of the art tends to change just a bit faster than I can keep up with these days.
At one job, I judged coding ability by the number of times your SQL woke me up due to huge I/O usage[1]. Many a CS graduate cannot read a query plan.
1) if it looked like a reasonable query and it was just Sybase's crap optimizer, I gave them a pass. Although, after a while, you learned to force the indexes.
At the time, the majority of incoming CS freshman did not even harbor the most basic ideas or curiosity about how a computer functioned. They had all heard they could make mounds of money. When I took elective CS classes in college, I had graduate CS students attempting to copy off of me.
It's possible CS students and programs have changed since the 90s, but based upon the CS graduates I encounter, I expect they haven't improved that much.
Code which can be fixed easily isn't "legacy". Code where a replacement would need to be bug-for-bug compatible in order to avoid breaking things is "legacy".
You could be writing legacy code today.
But other things change. The library changed its interface. You coded your crypto lib when MAC-then-Ecrypt was all the rage. You aren't handling HTTP/1.0 headers. Windows 7 doesn't even have those 16-bit drivers any more.
I hate the "throw everything away and rewrite with the brand new frame work every two years" that some parts of the web development community seem to have (and then get angered when you ask about a version over six months old because it is what you installed everywhere). But sometimes software does rot if it's old and no one is trying to keep it up-to-date.
The great thing about this definition is that it doesn't matter how old the code is - if it can be changed without worry about introducing unwanted behaviour because of test coverage, then it's not legacy.
It also means that you can write new legacy code right now!
I once made the argument that a team within my organization was actively producing new legacy code. It wasn't my most diplomatic moment.
It's always the function that's been rewritten twenty times that gets labelled as legacy code.
Hooray for short term job security!
Or you could figure out just an ID (typically a student ID number, although more than a few were social security numbers, apparently), and use "1066" since they had a backdoor PIN in quite a few releases. Battle of Hastings, eh?
Want to know how users did web security instead of asking their admins for a proper .htaccess/server-level config setup? That's how.
Security? On start, the Java app downloads a connection string to the MySQL DB. Which contains the root login for both SSH and MySQL. Then it prompts for your password and queries the Users table to see if you're allowed. And it helpfully logs this info into user's temp directory. "Ah yes, this is a known limitation in our current design."
Edit: They repeatedly lied to customers about various security fixes (I didn't do full disclosure on the numerous issues I found). They didn't care as they were sold for a world of money, then the entire product line was nixed. Most customers don't seem to care (I've found similar stuff across the board); hackers are targeting lower hanging fruit for the time being.
Moodle was pretty ick to begin with, but you should've seen the state of one install I had to work on by the time I got to it.
I still have nightmares.
I feel you.
It worked, but god damn; I literally quit that job because of the stress of working with Moodle day in day out.
> Clearly you must be joking
No shit.
Worse yet, was an initial claim that it was more efficient to do it that way. That was followed up with a claim that doing it differently wasn't possible.
Needless to say, I stopped working with that team of "developers".
This is always the worst. I've had experiences like that on many an occasion, where the person is simply like "huh? what's wrong?"
You can't really fix that level of sheer incompetence, ignorance, and arrogance all wrapped into one.
Sure you can. You can tell them why it's wrong and point them at reading material to fix it. Fixes incompetence and ignorance, and if you're lucky, arrogance. All wrapped into one.
Everyone had to start somewhere.
This is a classic excuse... so what if it leaks e-mail addresses (who looks at source code anyway?), makes clients wait (at least server's CPU is intact), and is just plain illegible (there is always a debugger if you want to fix a bug).
The smartest people I know are acutely aware of how little they actually know compared to how much there is to know.
Here's my own personal story. The other day, I had a brain fart regarding my password for my online banking account. So, I got lazy and just clicked the "forgot password" link, answered the security questions, and within seconds, I got an email. It had my old password in it. Yes, my bank stores passwords in clear text. sigh.
BTW, I'm also looking for a new job right now, so if you're after someone with 2 brain cells to rub together who also happens to be a decent Python programmer, shoot me an email. (It's in my profile.)
A reversible hash isn't so much a hash as a bijective function. The security of all stored passwords then depends on the secrecy of the "hash" function.
I used that site to show my boss his plaintext password to explain why MD5 alone is barely more than security though obscurity when trying to convince him that we needed to salt them as well - he agreed with me on the spot.
So I had gotten tired of people forgetting their single-english-word passwords and making me overwrite their MD5 hashed password to a known-value that mapped to something like "password123" (yes, no salting for the hashes). So instead of manually resetting the password in the database all the time, I banged out a small web app that ran on my machine for printing every user and reversing all of their MD5 hash'd passwords. It didn't work for the ones who had chosen actual, random strings for their passwords, but that was maybe 1% of cases.
And then I shared the IP address to my machine as a link for every other engineer in the company (all 3 of them). One of the other engineers freaked out that I had "exposed" the passwords, but as far as I was concerned, the passwords were already exposed. He shut up when I pointed out that the work was done and that I had other things to do, things that were his responsibility but he couldn't do because he had a habit of taking on too much work.
From that point on, any time I had more than 2 repetitions to do something, I'd write the most basic of web app to do it, and I'd shove it onto that little server on my machine. The future repetitions would invariably come in and I'd save tons of time not doing it the manual way.
Seriously, this was easy stuff. Don't take this to mean I'm bragging about it. I'm mentioning it because it is so simple and so obvious of work to do in these cases, and it eventually got me fired. I made the mistake of trying to get credit for the work I did, for saving the company time, freeing myself up to do other work, and all the CEO could see was that I was no longer able to charge 3 hours to create new accounts in the system now that it took less than a minute through my admin app. "Wasn't bringing enough value to the company." One of my reports found a quarter million dollars in lost licensing revenue. Wasn't bringing enough value to the company.
So it's not just programmers who can be grossly incompetent. Oh, they certainly can be, I've had to clean up my fair share of systems. But I've found far more often that systems are bad because the programmer's manager was an asshole idiot who made unreasonable demands and forced the programmer to make compromises. Maybe that programmer wasn't the best programmer, but nobody can do as good of work as they are capable in that situation.
[1]: https://www.pcisecuritystandards.org/documents/pci_dss_v2.pd...
†Requirement 8 (part 4):
> Render all passwords unreadable during transmission and storage on all system components using strong cryptography.
‡Introduction and PCI Data Security Standard Overview:
> PCI DSS applies to all entities involved in payment card processing–including merchants, processors, acquirers, issuers, and service providers, as well as all other entities that store, process or transmit cardholder data.
(It appears this does not include cardholders.)
>(It appears this does not include cardholders.)
Ah, very informative. So, here is the problem: the fact that crypto is specified when it's the wrong tool for this particular job (that of storing my online banking password). I don't want my password encrypted. I wanted hashed and salted with something like bcrypt, and I don't want it sent in the clear over email.
There is a whole other layer which is very good at handling incorrect or undeliverable addresses.
The solution involved creating an entire cache/hash layer on the client using local storage.
At the end of the presentation I had to try to be as respectful as possible when I asked why basic browser caching and content expiration weren't enough. Apparently it wasn't even considered -_-
I suppose they know what they are talking about.
localStorage is synchronous, and so putting huge amounts of data in there can delay page load.
Admittedly, when I was working on it, we were trying to wrangle it into something it really wasn't meant to do, but that fact - that it wasn't a good fit for our application - wasn't at all obvious until I finally started encountering all the "gotchas", and reading more about them.
I could imagine doing something like this if you wanted to support say es6 modules, and so each file needed to be processed after it was loaded to work in the current browser. In that case, I can imagine wanting to cache a processed one on the client side, although if you control your whole stack, it'd be better to do that work on the server.
The article suggests performing "server-side membership testing, which is O(1)", but I think this is a bit too much — you can do even easier server-side validation without the list of all valid e-mail addresses, just the information that "@[anything but these two domains] is not an OK target".
Then the customer won't pay you because you ignored their requirements. They might even sue you because you breached contract.
If the customer asks you for a mound of poo you write them a mountain of poo; you try your damned hardest to make sure that the room smells like poo when they are using the app. If they go home and tell their wife and kids about the giant mound of steaming poo they have been using all day then you have succeeded because they will go back to the one developer who knows how to stick to their senseless requirements.
At the end of the day all the other developers are giving them apps built on principles that only make sense to developers - principles that only really make sense for millions of users and not the few thousand that they have. Principles don't put a house over your head and food on the table - money does.
It does if you want to give the user instant feedback when they type an address that is in a valid domain but does not actually exist, as Ntrails pointed out. If you think a requirement doesn't make sense, you're supposed to ask the client, not just arbitrarily ignore it.
http://thedailywtf.com/ is still going strong! Be there or be ... competent?
http://programmingpraxis.com/2009/08/13/the-daily-wtf-malici...
Actually, I'm not quite sure why or whether it's good to be reminded of this. Maybe it's better to try and forget it.
UPDATE: they //did// get sued, but won, and the above patent is invalid now. Way to go! http://en.wikipedia.org/wiki/Blackboard_Inc.#Legal_matters
if (!Boolean.FALSE.equals(aBoolean)) {
// ...
}
case x is
when true => y;
when others => z;
end case;
if (aBoolean == true) {
....
} int landing_flag;
...
if( landing_flag ) {
do_landing();
}
We changed every instance of if( flag ) to if( flag == specific_flag_value ) to ensure that particular bug didn't rear it's ugly head again. I keep doing that now.
Good luck to the poor chap who will have to figure out what happened when that bug hits.
Also, you introduced a new error condition: a corrupt packet that should set a value of 1, but arrives as a value of 2 will not initiate the landing routine.
The right thing to do, IMO, is to prevent corrupt data packets from doing such stuff. Checksum the packets or, better yet, checksum and encrypt them. That prevents the enemy from taking over your plane.
Finally, I do not see how 'no proper bool' is relevant here. If the packet contained a single bit indicating the value of the flag, it still could get corrupted.
Yes. You seem to imply that's a bad thing?
> Also, you introduced a new error condition: a corrupt packet that should set a value of 1, but arrives as a value of 2 will not initiate the landing routine.
A corrupt packet should not do anything, so that's good, not an error. We do not want the landing routine to be accidentally triggered in flight. Missing a valid packet is much better than triggering on an invalid packet. (It's a UDP protocol, so the entire system is designed to handle missed packets. Ground station re-sends commands until positive acknowledgement is received from the aircraft)
> The right thing to do, IMO, is to prevent corrupt data packets from doing such stuff. Checksum the packets or, better yet, checksum and encrypt them. That prevents the enemy from taking over your plane.
Exactly right. We were already using a checksum in the datalink, and the corrupted packet that caused the crash passed the checksum as valid! During the post analysis of the crash, I discovered that it was using an 8-bit XOR checksum implemented years earlier. 8-bit XOR is ok for detecting single bit errors, but is not good at detecting burst errors -- it does not detect ~12% of highly corrupted packets. I also updated the system to use a significantly more robust checksum after that incident.
I think I wouldn't add it, though. Time is better spent on tooling that checks the variable doesn't get an incorrect value.
In a language that does have a proper Boolean type I still think checking equality with literal true/false values is a bit silly.
If I were writing something, and I really wanted to make it clear that I was testing a boolean to be true, I might write that. Then I can be absolutely certain the person reading it in 5 years won't misread it.
Actually, the more I think about it, the more I'd be inclined to do that in the false case ( ie if(aBoolean == false)... rather than if(!aBoolean).. ) because I worry that it's too easy to skip over the '!' - and I personally read that as "not aBoolean" rather than "aBoolean is false"...
I'll bet it complies to the same thing anyway, so it's just about readability at this point.
if (((((a == true) == true) == true) == true) == true)
That way, you can be even more certain!
Should I?
If it's somehow clearer to say if (x == true) instead of if (x), why isn't it also clearer to say if ((x == true) == true) instead of merely if (x == true)? And so on?
There is a grey zone in the middle, move in there.
As so many posts on HN have said lately; writing code is easy, it's the reading that's difficult.
It's funny, you still seem to think there is a "right and wrong" here, and you can't see that coding style is just like writing a poem - each individual will do things a little differently.
Edit: your original post said "If I were writing something, and I really wanted to make it clear that I was testing a boolean to be true, I might write that." What does this even mean? Simply writing x isn't really really clear, so you write (x == true) to make it extra clear? if (x) is testing boolean to be true and it's crystal clear. What else can it mean?
The clearest way I can think of to write that, that's hopefully less prone to misinterpretation is if(x == false) rather than if(!x)
Like I said, these are just my personal opinion and an expression of how I code (and likely things that I find difficult or often misread when reading the code of others)
It's interesting you admit that people are doing it "the long way" because that's how it naturally flows out of their head.
Doesn't it make sense that it would naturally flow into their head the same way?
What's the goal here - write very tight, concise code that fits some arbitrary standard of "correct"?
Or to write code that flows out of and into people's heads easily?
boolean updateMsg(boolean pPassed) {
if (pPassed) {
incrMsgCounter();
logger.info("Message sent");
} else {
logger.info("Failed to send");
return false;
}
return true;
}
Other than its returning a boolean being pointless, I mean?
If not I may be compelled to create one.
And even the dots are a bit iffy.
I like it.
Link: http://www.whatwg.org/specs/web-apps/current-work/multipage/...
I check it with this RegEx to make sure it's in the right format of [string]@[string].[string] to make sure that user at least tries to enter an email address, but beyond that sending and email and getting a response is the only way.
Is not
> [string]@[string].[string]
It's mere ^.+@.+\..+ (in both POSIX extended and PCRE dialects) that properly represents the latter.
And even such regexp filters out many technically-valid but obscure cases of RFC-compliant email addresses (for example ai, io, kh and ws TLDs have MX records, and supposedly hostmaster@io should be a valid email address).
I'm also disappointed I lost a couple of minutes of my life reading about this stupidity as well... just because it got 233 points.
So, I'm looking at YOU 233 who upvoted this. WHY DID YOU DO IT?
Just click the downvote button on my comment. Don't bother to explain anything.
Oh, I know, you disapprove I called you smart. Sorry about that.
"Why don't I see down arrows?"
I vouch for the code, wrote it myself with some really good advisors around! ;-)
Get in touch with me personally, I'll give you a brief introduction!
return userNamesStr.indexOf(curForwardUserName) >= 0
(I'd love to not have to explain sarcasm, but people have an incredible difficult time understanding it here.)
There is certainly value to the post but, yes, the flaming gets old - very quickly.
These days I do my best to try to remember we all make mistakes, and that instead of flaming whoever is responsible for a mistake when it rears its ugly head, it's probably best to take it as an opportunity to discuss what I saw as shortcomings in the code with them and turn it into a learning experience if possible. Sometimes of course this isn't possible when you inherit code from devs who are long gone. Either way, I'm not sure any good is done throwing all that negative energy into the air.
1.) Prevent typos etc. Regex or Mailgun or Kicksend is enough. 2.) Prevent bounces, prevent wrong signups one needs to do double opt in.
2.) I urge developers to step back and think about the requirements they get. Concerning those I would have thought what validity in this situation means, e.g. light validation in JS, send validation / validation list on the server etc.
at my university we use a similar system called OWL, which replaced a system called Web-CT. Both are horrendously slow, with fantastically poorly thought out interfaces.
There is a ton of money to be made here. Low hanging fruit.
if input_email in valid_emails_set:
send_email(input_email, another_param, etc)
It's fairly trivial to define your own __contains__ in python.
Although for a set of 80k items and long-running processes (fcgi or wsgi) you could also load the whole thing in memory directly and not bother with a custom `__contains__`.
Meth. Two week binge. SilkRoad.
I can't imagine opiates did that.
And the only thing you'd do on coke is more coke in combination with hating yourself; not coding shit like this up.