Hacker News Reader

Comment by phs318u | Hacker News Reader

The reason that both Access and Excel use is so prevalent in corporate “shadow IT” land is because there are many parts of the business that have problems for which only a negative or marginal business-case can be made for IT to solve it (given the “get out of bed” costs of most IT departments). It’s a barrier-to-entry problem. Excel and Access are cheap enough and fly under the corporate IT radar (no involvement needed), that these marginal problems can be addressed in a self-serve manner.

I’ve worked in organisations that have tried to kill these tools off but unless you can lower the cost-to-play or offer a “better” self/serve alternative, you will fail.

View on HN

Access is the perfect tool for an intelligent, technically-minded person with limited programming experience to create an application to replace spreadsheets.

There are a lot of those kinds of people out there, and they're extremely useful in introducing minor optimizations that other people wouldn't be able to find. Access is for that guy who says "I know there's a better way to do this," but doesn't have access (no pun intended) to a team of programmers and a project manager.

I didn't major in programming in undergrad but I've taken classes here and there, so in my first job out of college I replaced a really awful system of spreadsheet-jockeying with an Access DB.

I considered other options, but that it's self contained and NOT a web application is a feature, not a bug. I couldn't get access to the corporate database, so I just ran the Access DB on a network drive. It's still probably there, ten years later, running happily on its own.

Honestly, it seems like the fix for Access... is a better version of Access. It fills a very useful niche, between spreadsheets and full-fledged applications designed by programmers. It's so much easier to make a quick app that works for your organization than get an external team of programmers involved, who will probably tell you "no" or remain unconvinced that you're worth helping.

With Access, you don't need political clout, you don't need years of experience, you don't need a title. Just build an Access application and get kudos from everyone in your team for making their lives easier.

These are excellent points, I think it’s underestimated how difficult it is to get approval and budget for a small project that doesn’t directly contribute to the bottom line or have huge savings. These small but impactful solutions using Access act as both a stepping stone to greater skill sets, a way to build POCs for non-tech or low-tech people, and to avoid months of politics in big organizations.

I’ve had projects that were red-taped before getting off the ground due to resources being placed elsewhere. I then take those project ideas, and instead build them in Access as a POC using shared drives, splitting the database, etc. By the time I have 100 users saying how useful the application is in a couple months, other bigger projects haven’t even finished a project plan. Meanwhile we’re then ready to scale the solution using an appropriate stack and basically can just say “replicate our POC features but improve performance, security, accessibility, etc”. So far that’s worked pretty well for me.

In summary: find people’s Excel files with a mess of VBA and formulas —> see if the use case should be expanded —> build POC in Access without permission/budget from a bunch of people —> see how it goes and then plan to scale with the evidence you’ve gathered from your POC.

I taught myself Access 97 when I was about 10 years old and helped a teacher set up a small database to register library books. I couldn't have done that with a real SQL database, but Access was somewhat discoverable. Never underestimate the empowering power of tools that can be learned without any external help! I hate Access, but I love it too.

I know you say that being something other than a web application is a feature, but we have Microsoft's replacement for access in Office 365 and it's PowerApps combined with Flow. I haven't been able to dive too deeply, and it definitely has a learning curve, but so did access. Further, data is actually backed up and permissions can be applied.

I also have fondness for Access, but I think that we have a chance for something like that again.

O365 still gives you desktop versions of the Office software. It does makes me a little worried, but I'm hoping Microsoft isn't suicidal and won't pull the plug on desktop software, leaving us only with the half-featured web versions.

Flow is interesting, but AFAIR it requires extra clicking in the administrative interface to get running, so it might not work for shadow IT as well as Access does.

I am hopeful that between MS's financial commitment to the suite, improved JS performance in the browsers, and a set of practices coming around, it will reach parity in the next few years. I run Linux and while I miss features, I can do my job in an O365 org rather than having to have a mac or run a windows client for office suite.

On the one hand, I'd like it too. I use Linux as a daily driver - but I recently replaced my sidearm machine, and I keep it on Windows, so that I have a computer on which I can run desktop Office.

On the other hand, I'm convinced that once Office on-line reaches feature parity with the desktop, they'll quickly drop files, and turn Office into just another SaaS that holds my data for ransom. I don't want that.

I couldn’t agree with you more. I’m currently involved in a small startup where loads of administrative, transactional data is contained and manipulated by hand in an ever-growing Excel spreadsheet.

I’m a Mac user but there’s plenty of Windows machines with Office lying around, and my first gut instinct was to start throwing something together and importing the Excel file to populate the DB, and construct ‘robust’(er) queries & scripts for interacting with and getting reports from the data.

Perfect? No way. But within a couple of days I was able to get something working more reliably and (albeit temporarily) “shored up” a major calamity waiting to happen. Am I proud of my work? Absolutely not. But it was quick, cheap, and cheerful. Eventually when urgency has lessened and there’s more budget (in terms of mind-space, time, money) we can start looking for developers that will take my kludge and make it into something more bullet-proof.

For now, the leap from Excel to Access has been a huge leap forward (the kind that the physically illiterate offhandedly describe as a “quantum loop”, not realising that quantum leaps, though discrete in nature, are tiny).

I think Developers also underestimate IT Setup Costs for a DB since its so easy to do it locally. Setting up a simple RDBMS like MySql or Postgres and skinning with with a basic, out of the box UI is an afternoon of work if you need to look up commands, or minutes if you have done it recently.

But setting up something for a Team immediately opens questions like Access Rights (Can we limit this just to the team, or even have profiles?), Security (Needs to plan nice on the VPN/intranet), Scalability (in theory easy, but provisioning/managing VMs is hard), and Recoverability (where are the backups stored and who makes them). Those are all hard problems from not just a technical perspective, but also a business one.

But that is just the technical hurdles, not the process ones. In a well (overly?) managed system it can take a handful of people multiple meetings and possibly hundred of man hours to add a new field to a form. Adding the field is trivial, but deciding all the business rules around it is hard. If you run your own system, you can ignore any established process.

So its two fold. Its not just IT start up costs, its also getting everyone to agree on changes to an existing system, versus running your own.

I would argue many of the costs associated with migrating from Access to a "real" DB, is not the actual migration script, but the reassessment of all the business decisions that went into the design of the Access schema. Usually a committee evaluates if they are valid, and what to do with existing data to make it conform to a new schema.

I've worked in multiple organizations that run small Access apps here and there and for us the main cost is just sitting down and doing it.

My last job was at an HR company and things were historically done through Access and spreadsheets, until an IT guy came along and started writing C# web apps, rose in prominence and now runs a small internal software team. I worked for that guy.

It's not terribly politically difficult to convert existing applications, but it's mind numbing work and it's bottom priority. There's not a huge need to change something that already works, and the only reason you do it is you know it can't stay like that forever.

Plus, you need a server (ie. obtain one, set it up, and maintain it). With Access you just host it on the team's shared drive.

In a large organization, getting a small server when you don't work in IT is not simple at all.

Add to that getting additional software installed.

In most large organisations non IT users have very limited permissions, even to their own machine.

Yeah exactly. Working in an IT Dept when someone said "ah they've built a spreadsheet to do it, but now they want us to add that functionality to the main DB" we'd all roll our eyes because it was another lashed-together thing that we would have to unpick and re-implement. But the fact is, if they'd come straight to us and asked us to do it instead of doing it in Excel first, they would have had nothing, for years and years.

“But the fact is, if they'd come straight to us and asked us to do it instead of doing it in Excel first, they would have had nothing, for years and years.”

Very true. As soon as us professionals take over the dev speed drops drastically and costs go up.

I killed a project a few years ago that had spent about $8M on some project to implement a pretty straightforward business process. No end was in sight. The catalyst was Access 2003 going end of life, and this little app doesn't work.

I don't know why one of a 3 BAs couldn't figure out WTF was happening and fixing it. We had an intern futz around with Access for a few weeks and it's been running fine to this day.

Also you can consider the spreadsheet versions as a nearly feature complete & tested specification.

My dream is to build a modern take of the fox/dbase family. I starting with the inner language

http://tablam.org

Access, as concept, is great. Is exactly what many need.

But it have a lot of small deficiencies that make it disliked by developers. I have used FoxPro, and it have almost all the right things instead (Fox is Access, but goooood!... except a bit less user friendly).

I wish I could dedicate to build this. Is weird to me why this kind of tool have so little support?

I really like the idea of developing a modern Access alternative, but why are you trying to create a new language? Aren't QBE and SQL sufficient?

Why not something that is like a cloud SaaS version of Access, but can be extended with some procedures and later used as a GraphQL backend like Hasura?

Salesforce is (among other things) basically a cloud SaaS version of Access. And people pay a lot of money for it. See my other comment https://news.ycombinator.com/item?id=21403393

True, when I think about what I've seen, what people are building with Access and what people are building with Salesforce, there's a lot of overlap.

SQL/GraphQL are query languages. Good for that, but not for fully develop an app.

FoxPro have his own language that allow mix SQL and imperative constructs:

https://en.wikipedia.org/wiki/Visual_FoxPro

This mean:

* You code login in foxpro

* And the stored procedures

* And the forms

* And the reports

* And the script glues

* ....

"Just" adding SQL/GraphQL to a RDBMS is absolutely not enough. Both are too limited.

For example:

- Can you build a btree with SQL? No

- Work with the terminal? No

- Make a visual grid? No

- etc

Only query and maybe crud.

There was a FoxPro inspired Python tool I remember from about 10-15yrs back: https://dabodev.com/

A bit more coding than Access, but it was DB independent on the backend.

Looks like nothing has happened on it for quite a while though.

If you come from the dBase / Clipper experience, then Harbour could be considered a more modern version => https://github.com/harbour/core

Everything needs to be self-service as much as possible.

Need a SQL Server / PostgreSQL DB for your project, got click some buttons on a page and get back a URI for your database.

Automate, automate, automate.

That goes against typical IT policy that if people don't use something they can't break it, so let's make sure they can't use shared IT resources.

I believe Bill Gates' assertion that "90% of software hasn't been written" is more true every day.

There are so many "workgroup" size (1-10 users) apps that simply don't get written with today's tool stack.

I don't disagree, but isn't it a sign of much deeper issues if this "Shadow IT" land exists at your company in the first place? Why is it that experienced programmers can't efficiently solve a problem that a business user can handle in Access?

Are they under-staffed? Neck-deep in spaghetti code? Lacking business knowledge and the opportunity to acquire it? Bottle-necked by a lack of business analysts or testers?

I hear about this too often and nobody seems to think it's a major problem. IT departments can have a queue that's 2 years deep and people don't even bat an eye, they just think "Oh, that's how IT works!" And that's not even taking into account all the requests people aren't submitting because they've given up hope of getting any dev time.

The problem is most of the people writing code in Access (or monstrous Excel macros or in apps like MatLab etc) are not trained programmers. I think the article sums it up well by using "power user" to describe them - technical people with no formal programming training. Think Engineers (the non software kind), Accountants, Plant Technicians these kinds of people.

In my org (I'm an Engineer at a manufacturing plant) a long time ago someone senior made the decision our company was not in business of hiring programmers so any coding is done by contractors.

So yes people have implemented things (Engineering models for example) in MatLab, or Access, or as an Excel spreadsheet etc as "prototypes" ostensibly intended to be rewritten by a real programmer. At some point these models invariably find their way into production environment and suddenly you have some kind of mission critical thing written in Access (or VB6 or Matlab or Excel).

Then IT has minor panic a contractor is given the job of trying to translate the "prototype" into "real" code and has a battle on their hands.

Often programming contractor doesn't have technical understanding of what model is doing, regardless of choice of language the code engineers write tends to look like Fortran (i.e it's essentially all math formulas), even comments probably don't help when they say stuff like "/* Ergun's formula to calc Delta P */" that doesn't mean a lot to someone without a chem eng background. I've seen contractors produce bugs by doing things like misreading a spec and adding a variable called Fe203 (i.e Fe "two hundred and three) instead of Fe2O3 (Fe "Two Oh three" - a chemical formula). You can't really blame contractor he doesn't know anything about chemistry - he was just hired to rewrite this monster in java based on a spec.

So I don't think its a simple as turning problem over to an experienced programmer and expecting success.

I have seen all of these symptoms at numerous large companies. The backlog is years deep and the costs associated with engaging the professional programmers could be staggering. The truth is, not all solutions need anything more robust than what Office can provide.

Agreed, I think it's because developers like to over engineer everything and the managers like to over engineer the processes.

So many of these access excel solutions should be a days worth of work, simple perl cgi scripts with a minimalist UI deployed by rsync. Instead we have to use our super "productive" modern frameworks, split everything into a thousand files (god forbid you embed an sql query in the only place it's called), add unit tests, etc. There's certainly times for the later approach, but most businesses need much more of the former.

So because developers don't have a reasonable platform to pinch off random little projects others step in.

"So many of these access excel solutions should be a days worth of work"

I don't know why you think a lot of VBA would be replaced by a little perl. Of course, my perspective has a lot to do with the fact that it was essentially impossible to get a new perl module installed where I used to work.

But developers often don't appreciate the importance of presentation (and other) details in reports for managers. Your reference to "a minimalist UI" is telling.

Even though Access and Excel can be buggy, unstable, and annoying, it doesn't make much sense to use anything else if you are automating a report that was previously assembled by hand in Excel, and needs to match precisely.

And often IT types like to exercise power by gatekeeping - if you aren't doing "real" programming, you don't need a Turing complete solution, so Office ends up being the only option. I've been told that if I can select a list of columns from a dataset, and some filters, by pointing and clicking, that's all I, or my managers, need for reports.

Honestly, I think a lot of people find fulfillment in their work through being the person who can say "no" to people, particularly managers that are theoretically higher ranking. And also by expressing themselves through creative decisions when others fail to specify details. I think that using Office/Access/VBA may be correlated to rejecting the value system of most developers, rather than a technical judgment.

> But developers often don't appreciate the importance of presentation (and other) details in reports for managers. Your reference to "a minimalist UI" is telling.

We're talking access here, at best it's ugly and at worst you've got a gaudy background image(1) and a color scheme that would give the disability compliance officer a stroke. Throwing in bootstrap or a more minimal css framework is a huge step up in terms of presentation.

> Even though Access and Excel can be buggy, unstable, and annoying, it doesn't make much sense to use anything else if you are automating a report that was previously assembled by hand in Excel, and needs to match precisely.

I'm thinking of scenarios a bit more complex than that. Access apps generally have a few data input screens, multiple users, etc. Not complicated but not as simple as reports.

I'll admit that I do run away from anything to do with reports, but usually that's because they've installed some "easy to use, no developers required" reporting system that the non-developers can't use and makes life 10 times harder for the developers. If I can just write sql to shove data into an html table or excel template (where we can have the best of both worlds) I'm more than happy too.

1. I actually think of some of these old access programs when I look at windows new built in mail app, who the hell adds a background image?

"We're talking access here, at best it's ugly and at worst you've got a gaudy background image(1) and a color scheme that would give the disability compliance officer a stroke. Throwing in bootstrap or a more minimal css framework is a huge step up in terms of presentation."

I'm talking about creating Excel reports based on pulling stuff from (possibly a random assortment of) databases. Where you can use any feature of Excel. Not that the Access application is distributed to people who care what it looks like.

It seems like you can't even imagine a complex report that isn't an interactive application. So I think we're just talking different languages.

Honestly, I was just talking to someone in the organization I work in with the same lack of understanding. He was like "you have a point and click interface that lets you choose some columns from a table and some filters using simple boolean criteria, what else could you (or your manager) want?"

I want the ability to define all the business rules to produce the formatting and munge the data, I guess. And to structure the code in such a way that it's flexible enough to handle major changes. I need regular expressions. I need to run a diff algorithm on text. I need to use XML and REST to talk to SharePoint. I need to scrape information from a system that I only have access to through a web browser.

Basically, I'm using Access/Excel to do what I used to use Qlikview for, or just plain Perl, and it seems to be less of an "impedance mismatch" as people like to say. Also it doesn't cost as much as a car as Qlik licenses did.

god forbid you embed an sql query in the only place it's called

-gasps-

Because developing software is at least an order of magnitude easier and faster when the developer is the user?

This is so true.

Re: Why is it that experienced programmers can't efficiently solve a problem that a business user can handle in Access?

Because existing web UI standards suck rotting ass to hell and back. The standard was not meant for CRUD and still isn't even if you add gajillion layers of (buggy) JavaScript to emulate a real GUI.

We! Need! A! Real! GUI! Standard!

Just master rails -g and you can be done in an afternoon.

I agree that if you tune it for shop conventions and know its warts well, one can indeed be productive in it. But getting to that point is not necessarily short or easy.

I think that is a very fair and accurate assessment.

Sometimes it is budget or just needing someone to drive it through a heavy process.

It is also not always worth it if you cannot show a real business case. The thing might help you and it might be worth it if you hack something together yourself but if IT is suppose to build something that is available, has support, has disaster recovery, is patched etc. it might no longer be easy to find the business case for it.

Exactly, to get professional IT resources you need to demonstrate a business case, go through many decision boards and budget allocations, defend it for months, and be ready to be pointed out if it doesn't reach its promises.

So in the end people hack it in Excel or Access. No need to ask permission, no budget issues, no blame if it's not great.

Also: little or no waste happened if it doesn't pan out. Your Access prototype doesn't seem to make you and your team work faster? Copy out the little data you've put in there to Excel, SHIFT+DEL the Access file, and you're done. You've wasted few hours. That beats wasting several man-months of IT time only to not use the result in the end.

Access is agile in the truest form. It lets you prototype and refine in an extremely tight feedback loop, until what you have is a working solution for your specific problems.

Reply on news.ycombinator.com