128 karma · joined July 18, 2017
I was writing database systems professionally, back in the days before the RDBMS concept was even a thing. So here's my (enormously long and convoluted) 2 cents worth. Please be sure to pack a sandwich and get hydrated before you continue.
Say you were dealing with doctors and patients, and needed to store that information in a database. Back in the day, you'd typically use a so-called hierarchical database. To design one of those, you need to decide, what is the most common access method expected to be: getting the patients for a given doctor, or the doctors for a given patient? You'd design the schema accordingly. Then, the preferred access method was easy to code, and efficient to run. The "other" access method was still possible, but harder to code, and slower to run. The database schema depended on how you thought the users would access the data.
But that is absolutely what NOT what to do with an RDBMS. Certainly you look at the users' screens, reports, and so on - but that's just to determine what unique entities the system must handle - in this case, doctors and patients. Then you ignore how the users will access the data, and work out what are the inherent logical relationships between all the entities.
Your initial answer might be this. A doctor can have many patients, and a patient can have many doctors. As any competent relational designer will instantly know, this means you need a resolving table whose primary key is a composite key comprising the primary keys of the other two tables. So if Mary was a patient of Tom, you'd add Mary to the patients table (if not already there), Tom to the doctors table (ditto), then add a Mary/Tom record to the resolving table. By that means, a doctor could have any number of patients, a patient could have any number of doctors, and it's trivially easy to write simple, performant SQL to access that data however you want.
But then you'd have a ghastly thought: patients can also be doctors, and doctors can also be patients! Say Tom was also a patient of Mary! Now you need a Tom record in the patient's table, but that would inappropriately duplicate all his data from the doctors table! Something's clearly wrong. You'd soon see that from a data modelling viewpoint, you don't want doctors and patients as separate entities - you want a single entity Person, and a resolving table to relate arbitrary people in specified ways.
So what?!!
So this. IMHO, many developers using relational databases have absolutely no idea about any of that. They design hopelessly unnormalised schemas, which then need reams of ghastly SQL to get anything out. The query planner can barely parse all that crap, let along optimise it. The database has to stop every five minutes to wet its brow and take a drink.
So here's my advice to any inexperienced relational database designers who have actually managed to get this far!! If you can answer the following questions off the top of your head, you're probably on the right track. If you can't, you're lacking basic knowledge that you need in order to use an RDBMS properly:
- what is a primary key? - what is a foreign key? - what is an important difference between primary keys and foreign keys? - what is a composite key? When would you use one? - what are the three main relations? - what is normalisation? - what is denormalization? - what is a normal form? and so on.
Just my 2c! :-)
> I'm not OP, but I suspect they meant that Python and some libraries would have solved your problem in a few days of work.
I wrote software professionally for 30 years. I have a good understanding of the work required to implement a business requirement! I cant really see myself spending 10 years on a one-week task :-)
PS. I didn't downvote you.
About 20 years ago I inherited enough money to retire on for the rest of my life. But the only evidence of those assets was, a dot matrix printout that anyone could have done in five minutes. So over the next few weeks, I created accounts in the various share registries and online banking websites, and manually cross-checked the printed list against those accounts. All was good, but I wanted to do that on a regular basis, and it was clear that doing it manually, on a regular basis, was not gonna happen.
I am personally -- NSFW SPOILER ALERT!! -- a long time Windows/IE user, and I knew that IE has a comprehensive automation interface. In literally a few lines of VBScript (say), you can programatically start IE, navigate to URLs, parse the DOM, create new local documents, and so on. But although those interfaces are very simple, they're not robust against failures, and have various weird and wonderful corner cases that will sometimes trip you up.
So over the next 10 years or so, I created a robust, general-purpose IE automation wrapper, and wrote some applications on top thereof. The result is that at any time, I can click a few buttons on a handsome user interface, and enter a single master password, at which point my application will automatically log into each share registry and online banking website in turn; download all new and amended data therefrom; transform all data into common formats; store it in a local cache, so it doesn't matter if it later disappears from the website; then creates a single share holdings document, showing all share holdings from all share registries, and a single online banking document, showing all transactions from all accounts at all banks since the start of time. Both documents have dynamic, user-defined sorting and filtering. They also hilite significant changes; check all share holdings against banking credits to confirm that all expected dividends were received; and so on.
In other words, I've completely automated various important financial checks that I simply couldn't do by hand on a regular basis. The downside is, I'm tied to IE - and - several hundred thousand lines of VBScript! If anyone would like to re-write all that code for Chrome or whatever - for free - you're more than welcome to contact me!!!
Seeing what happend to Stephen Martin, I ask myself, could that happen to me (if I was diving there), and if so, how could I reduce my risk? The answers (for me) are Yes It Could, and Don't Dive There!
My view is, who's to say they won't do it again?
> Most people have no issues diving in Malta
Until they do! :-)
PS. Not trying to start an argument, just stating my view from this side of the pond :-)
This is probably a silly question, but: The article says he has to use performance.now. But isn't one of the mitigations for Spectre and/or Meltdown, to reduce the accurance and/or resolution of performance.now? If so, would that affect his code?
Not after this: https://www.theguardian.com/law/2016/jan/17/briton-fights-ex...
I won't summarize that article, since it's all quite clear as long as you read to the end. The upshot is, that anyone diving in Malta is subject to bizarre legal risk if something goes wrong. It's a bit like stopping to help at an accident, and then being charged with causing it!
The whole thing raised a massive stink amongst BSAC divers, since it seemed (at one point) that BSAC insurance wouldn't support him. I think he's still 6 figures out of pocket.
TL/DR: do not dive in Malta.
>> Perhaps the key to their success was typically programming work estimations ;)
No, you should always normalize programmer estimates by increasing both the unit and the measure by one. So "a few weeks", say 4 weeks, will actually take: 5 months! So he still took 12 times longer than the properly normalized estimate! :-)
My first experience in this group was watching them debug a problem. They had an old FORTRAN listing spread out over a table-sized (10Mb!) disk drive unit. The bug was in 5 lines of code, but it just didn't make any sense - it was as if one line was simply not being executed.
Eventually one of the guys went over to the command console and looked at the object code on disk. "Hey, that 'IF' statement isn't in the object code!". More head scratching ensued, until someone looked more carefully at the grubby thumbprint at the start of that line on the dirty old listing.
Under that grubby thumbprint was ...
[can anyone quess?]
It's amazing how heavy a burglar safe is - even a small one. I have one just big enough to store some documents, spare phone, keys, and some cash. But it took two guys and a trolley to get it up the stairs to my apartment. Even if a burglar could rip it out of the floor, and push it down the stairs, I doubt that they could lift it up into their car or whatever.
Fair comment. But I guessed that all he wanted was something that would stop a burglar from bashing it open in 5 minutes. It's probably fine for that purpose. But I might say something to him next time.
It depends on the type of safe. I'm not a locksmith, but I have researched this, and this is how I understand it.
There are "security" safes (aka "burglar" safes), and "fireproof" safes. A security safe should be bolted to the floor, to stop the bad guys ripping it out. It comes with pre-drilled holes for that purpose. However, a fireproof safe should not have any thru-wall holes, since a fireproof safe must protect against the deluge of water that will occur when the fire is put out. Thus, fireproof safes are often glued instead of bolted.
Some time ago, a relative showed me his new safe. It didn't have any pre-drilled holes, so he drilled some in, so he could bolt it to the floor. I chose not to tell him the following: (1) The fact that he could drill any holes in the safe, using his trusty Black & Decker drill, showed that it was a fireproof safe - not a security safe; and (2), drilling those holes destroyed its value as a fireproof safe!
In summary, you need to decide what's most important - theft protection, or fire protection. Those are different types of safes. The problem is they look identical to the untrained eye.
[edit: sp]
Australia has a system called "imputation credits" or "franking credits".
Say a company BigCo makes an operating profit of $100. They decide to pay that out as a divident to shareholder Mary. Australia's company tax rate is 30%, so BigCo sends $30 to the tax department, and the remaining $70 to Mary.
Say Mary's personal tax rate is 50%. On the face of it, Mary would pay 50% of the $70 (ie. $35) to the tax department, leaving $35 in her own pocket. However, this would mean that the original $100 profit had been taxed at 65% ($30 + $35)! This is seen to be inequitable, and a disincentive to owning shares.
So instead what happens is this, Mary gets a "franking credit" of $30 along with the $70 dividend. The franking credit is just a book-keeping entry to record the amount of tax that BigCo paid on its original $100 profit.
At tax time, Mary's accountant proceeds as follows. (1) Add the franking credid ($30) to the divident amount ($70), thus reverse-engineering BigCo's original profit ($100). (2) Apply Mary's personal tax rate (50%) to that whole profit, getting $50. (3) Ensure the tax department gets that $50, regardless of who and where it comes from!
In this case, BigCo has already paid $30 of the $50 total tax obligation on the original $100 profit, so Mary is only up for the remaining $20. By that means, BigCo's original profit of $100 is eventually taxed, in total, at Mary's personal tax rate of 50% - not at the double-taxed rate of 65%.
This might just seem like a whole lotta shakin' goin' on - but wait, there's more!
Say the dividend was paid to another company (SmallCo) instead of to Mary. SmallCo's tax rate is also 30%, so when you apply the franking credit, you'll find that SmallCo pays no tax on the $70 dividend.
Now say it was paid to a self-managed superannuation fund (SMSF) in so-called "transition to retirement" phase. These are only taxed at 17%. 17% of $100 is $17, so the total tax expected by the tax department, on the original $100 profit, has been overpaid by $13 ($30 - $17), and the tax department will refund that $13 to the SMSF! The SMSF gets the $70 dividend, and not only pays no tax on that, but also, gets a $13 refund cheque.
Then we get to the apogee of this process, the quintessence of financial wizardry, a situation so warming to the heart as to impose some risk of dangerous palpitations. Say the SMSF has gone into "pension mode". In that mode, it pays no tax. Thus, when the fund receives the $70 dividend, it also receives a full refund of the $30 tax originally paid by BigCo!
As I understand it, BigCo can actually choose to pay the full tax, part of the tax, or no tax, on its origial $100 profit. The dividend is correspondingly said to be "fully franked", "partly franked", or "unfranked". A dividend recipient ("X") might get a mixture of franked and unfranked dividends from various sources. "X" would add all franking credits to a notional "franking account", then use that account at tax time to offset their normal tax obligations.
[edit: typos]