batou

Certainly not an anedote. I was an Oracle and SQL Server DBA for a number of years amongst other hats on stupid big datasets and loads. Add to that 25 years' experience getting companies out of deep trouble that everyone else has given up on. I know my shit.

Orthogonality: I haven't introduced anything here. Very rarely does any conceptual model of reality fit into the relational model. It's more imperative than that. Everything is usually crudely shoehorned into it because it's a compromise that people are barely willing to make or because they don't understand how to model a system properly.

Duplication: there is duplication in there. The versioning is very inadequate and API stability is the key to success on this. Plus also, this is a minor part of the application to consider. It's no different to issuing SQL. The protocol is different, that is all.

Lock in: There is no ORM lock-in past the platform. If you isolate everything properly i.e. use command-query-separation then this is a non issue. It's trivial to replace the ORM. You can even do it piecemeal. We've done it. I yanked out bastardisd ADO and EF out and stuck Nhibernate in. If you couple all your logic into the database, no banana. That luxury goes out of the window.

They may be underused, but when your vendor pulls a 26% price hike on half a million quid's worth of kit, can you afford to bend over and take it?

It's a tradeoff, but not one I'm willing to make on medium to large scale systems where there is a capital risk.

Yes. It's probably why I play a synthesizer. It's like, excuse the analogy, a giant audio dildo.

Korg M3: https://www.youtube.com/watch?v=oeUN0W3WI0o (this is not me, I'm nowhere near this)

Edit: further thoughts. It's certain chord transitions, pads and vocals that do it.

I conceed. Some good points there.

Stateful:

If I have to load the stored procedure into the persistence engine then that step is required. This is no more stateful than queries in the application but it means that the relevant state in both the application and the database engine needs to be reloaded and constantly sychronised. Ergo, two times the work.

CHECK constraint violated is no good for humans. Prevention is better than cure here.

Why shouldn't I enforce unique constraints in the application?

1. Open a transaction

2. Get a user by name from the ORM.

3. Exists? Tell user that the username is already registered.

4. Doesn't exist? Save new User instance.

5. Commit transaction.

Steps 2 and 3 can be as arbitrarily complicated as you need them to be, are fully testable and cheap with anything that uses MVCC.

Yes that's true as well. JIRA does that in some configurations. Does my head in.

Definitely. Which is why we still use one. It's cheap, already there and does the job.

I think it's quite simple really. Sure it's a bit ugly but most APIs are behind the scenes. As long as the API itself is clean, that's not necessarily a problem.

In this case though, the problem with growing the permission set is you then need rules to define the sandbox "size" outside the process, then tools to administer it, then the policies to be deployed with the program.

If you're using tame which is restrictive, based on how they do privsep in OpenBSD, you're forking tasks off from a control process and then removing privileges. So the program defines the policy internally, very granularly and correctly.

How you'd do that with seccomp-bpf with forked child processes etc would be vastly more complex. Consider OpenSMTPd for example which consists (I can't remember the exact separations): master process, smtp listener, queue, writer. These are all forked from the master process and talk to each other with pipes. How would you mandate the call limitations efficiently if this was externalised? Also how would you assure that a misapplied external policy doesn't compromise your system.

We are exactly there. We're having to remove vast swathes of stored procedures and rewrite everything.

Despite many attempts by people to prove otherwise. Literally every "enterprise" product I've seen tried to do this and fucked it up royally.

Exactly. If you isolate this as an programmer's concern then you can skip the administrative effort and assume that the application configures itself properly rather than relies on someone to experiment with producing a sandbox for it to run in. The latter is error prone hence why things like SELinux have permissive and enforcing modes. SELinux turns your job into a reverse engineering one which isn't fun.

I'm slightly bitter that I've become a bit of an SELinux expert over the years. I shouldn't have to be one, hell the tech shouldn't even need to exist in 2015.

No that's about it. Granted they give you efficient paths to get it back again and store it on the right shelves but that's about it.

The key benefit is probably familiarity.

I'm not saying they're an invalid tool, merely just a single tool in a batbelt of a million solutions.

That's a poor argument because we're not discussing the arbitraty boundary of storing versus working with data as the lines between those are very blurred. This is even more the case when you use stored procedures which work with data close to the storage.

While we're on this subject, RDBMS are no better at storing data than any other technology out there[1]. In fact when you start thinking abstractly like this, other tech such as Riak makes sense for a lot of workloads.

The only real benefits of RDBMS' are fixed schema, fungibility of staff, the ability to issue completely random queries and get a result in a reasonable amount of time and the proliferation of ORMs.

[1] Caveated on insane design decisions like MyISAM storage engine and MongoDB as a whole.

Counter-counter-points:

1) Stored procedures aren't that much faster than issuing plain SQL over the connection. The main performance bottlenecks in a RDBMS are cache and IOPS. Regardless of where you execute those, they are all inside that black box after the query is parsed. You also get the added pain of cached query plans which fuck up performance when the query optimiser makes assumptions about table statistics and stores them with a stored procedure. (SQL Server and Oracle at least are guilty of this).

2) The only place I've had SQL injection attacks in the last few years is where people have used dynamic SQL inside stored procedures (sp_executesql) and not escaped them properly. Outside of that, both ORM and proper parameter binding make this entirely irrelevant.

It's completely clear cut IMHO.

Hmm no.

Avoid state at all costs. Stored procedures are stateful. Schema and migrations is pain enough already.

Write me a check constraint that validates an email address being put in a varchar column and reports back a sensible message which can be bound to an entry field with metadata about the error.

Write me a constraint and key arrangement which is unique across two and three columns in separate groups.

No. You're wrong.

That is quite frankly horse poop.

We use stored procedures. Not by choice; this is a legacy we're stuck with. It is nothing but an unadulterated disaster of a technology regardless of what you use it for. I'm talking 45,000 stored procedures here. 2000 tables. TiB of data. 50,000 requests/second across SOAP/web/desktop etc. It's hell.

Problems with stored procedures:

1) Performance. More code running in the hard to scale-out black box. You're just hanging yourself with hardware and/or license costs in the long run.

2) Maintenance. All database objects are stateful i.e. they have to be loaded to work. The sheer complexity of managing that on top of the table state results in massive cost. Add to that tooling, version control costs as well. Have you tried merging a stored procedure in a language which has no compiler and very loose verification?

3) Orthoganality. Nothing inside the relational model matches the logical concepts in your application. Think of transactions, domain modelling etc.

4) Duplication. You still have to write something in the application to talk to every single one of those stored procedures and map it to and from parameters and back to collections.

5) Transaction scoping. Do you know how expensive it is to introduce distributed transactions? Well it's a ton more cash when EVERYTHING is inside that black box.

6) Lock in. Your stored procedures aren't portable. Good luck trying to shift vendor in the future when you hit a brick wall.

Now I know it's popular to bash on Rails and I wouldn't use it personally but there are people using the same model on top of other platforms, like us.

Sorry but databases are just a hole to put your shit in when you want it out of memory. If you start investing too much in all of the specific functionality you're hanging yourself.

You're right but 99% of the problems I end up solving with SELinux are problems that fit tame perfectly. The rest of the problems are architectural i.e. the process doesn't have privilege separation mechanisms etc.

I really like this solution. Far better than pissing around with SELinux configuration and it's well contained.

Your app is compiled from MSIL to native on MS's servers if it's like Windows Phone. I think they can recompile everything if a dependency changes for example and push a new app version for you.

That's the ones! Never seen inside one before - thanks for posting!

Well they were pretty crap. There was a metalised bit of plastic for the card and the reader burned certain bits of the plastic to gate the light. If you bought a £5 card you could cut a hole in it, fill it up with epoxy and use a permanent marker to fool it into thinking you had a £20 card :)

Edit: my memory is faulty; they were units, not £. Been a long time :)

I've done this! You can see them with the naked eye quite easily. Just shine a light diagonally at the surface. I wouldn't bother reading them by hand though. You could take a photo of it with a digital camera then post-process it and stick it through mathematica or something.

However you might as well just use a card reader.

I used to hack stored value phone cards back in the very early 1990s. First they had an optical system which was easily circumventable with a torch a knife and some epoxy glue. Then there were the smart card ones which were easy to hack by scratching off the programming voltage pad on the card.

I got to the later mag strip cards like this and couldn't be bothered any more.

Either that or early warning?

It this on windows phone too. It's not the phone's fault.

I owned one for a bit (A500 and A1500). AS I was in the UK, this was wiped out by the Acorn Archimedes coming on the scene.

I can't say I ever "loved" it nor found it reliable. By 1998 it looked quaint compared to NT4 etc.

They'll still be reinventing the Amiga when I'm in my 80s I'm sure. I remember back in 1998 a friend had a bastardised A1200 in a PC case with bits of sticky tape holding it together. I was constantly lending him my soldering iron. 17 years later...

I have nothing against this for ref, just that it appears to be an eternally unsolved problem.

Interesting. Either way that sounds like a glorious way of using your time.

I pull 120Gb a month down. That's going to be a fucking big OTP and side channel.

That's fine until you have a 4 MLoC behemoth with 50 developers (I use that term loosely) that isn't separated into logical concerns properly like your average "enterprise product".

I've seen those things and what you end up with is a dev team in tears because some chump has fluffed the source tree, the integration build is still running so no one else knows and checks out the trunk followed by boom and 2 days of paralysis while everyone works all their changes back into it for a checkpoint build.

All mature products tend to evolve into this state unfortunately.

Ergo, that works on small things but doesn't scale. Minimally, using feature branches and integrating them in turn is the only situation that works on those sorts of products.

It got replaced. Plus I have three children already :)

Sounds just like the 2011 i7 MBP I had or "the cockburner" as it was officially named.

Might be the CPU throttling due to heat. There is no active cooling system on the new MacBook by the looks so I reckon it's thermally managing itself by doing less causing other problems.