Software horror show: SAP Concur
blog.plover.com
blog.plover.com
We had this issue with a certain report that was outputting weird values. Company had to pay the partner for support. I’ve skimmed over the code and it was processing hardcoded index positions. I repeat the person who did that had no concept of a loop. Since the query was returning more than 250 results the remaining rows were being ignored.
The solution? They increase it to 500. Certified. SAP. Partner.
Other issues like having to create folders on the shared NFS volume because developer couldn’t arse to create parent folder if it doesn’t exist. New supplier? Need to mkdir otherwise those long running reports would be inconsistent. Ah… let’s not forget CHMOD 777 EVERYWHERE when handling file systems.
Don’t even let me start on the “move to standard” utopia and the horrors that come with it.
I didn’t dare to touch that thing. First it was the company could lose support if the supplied code was modified and second if they knew I could fix stuff they would treat me as some cheap labor alternative.
IME (not sap but adjacent) partners cert and “levels” only has to do with how much you earn for the company.
Competence only comes into play if you’re so egregiously incompetent you start sinking projects and costing product rep’, and neither is really a concern for SAP.
They are an extremely formulaic organisation. Progression paths are clearly defined, and practically everything is run by a management system. This sounds good, in principal, and does result in a general baseline of reliability.
The flip side of this is that everyone is promoted to the limit of their competence, product development and implementation are rapidly spinning revolving doors, from top to bottom, and roles, not people, own products and decisions.
The result is a sort of methodical malcoordination, and seemingly insane decisions being taken at every juncture - unless you view them all through the lens of an emergent process driven by a simple set of rules and constraints: cover your ass, don’t do it if it isn’t explicitly ordered, always follow the management system.
It’s terrible. It’s just that it’s very reliably terrible, and that has its own appeal to other reliably terrible corporations. It’s all minmax, least worst, etc. - quality does not matter in the corporate world - consistency does.
Unrelated, but this reminds me of a time when I had to write some code in some proprietery programming language in some industrial automation project to program a PLC. The language was similar to Visual Basic but lacked support for basic things such as for loops. I spent some time creating a "meta-language" that would compile code down to this language with added support for loops and some other stuff. Then I would paste the compiled code in the proprietary software each time I had to change anything.
Good help the company if someone every gets access to the SAP servers.
This is not a SAP-specific problem. I've seen highly paid programmers with veteran-like CVs deliver absolute crap in the PHP world. If there's a shortage of good programmers, you'll get this kind of thing. Moreso if it's in a un-sexy field of work like SAP.
Furthermore, amongst the already-low-priority UX issues, I can imagine UX of the expenses application being rock bottom, because which business ever called up their ERP vendor and said “we need to make it easier for our staff to claim expenses”?
Businesses that were fed up with fielding questions from employees about how to claim expenses?
In my experience it's reasonably common for companies to switch expense claim systems because their existing one was rubbish. I haven't worked for any SAP-using megacorps though tbf.
But their sales team probably plays less golf, let's put it this way
I have a background in finance, and I think everyone is missing the problem that this is trying to solve.
People are careless when they claim expenses. They claim for things they lost the receipt for and then they try and pressure some accounts clerk into forcing it through. They make careless mistakes like multiplying their milage by the wrong rate, because they don't see why they should have to look up the right one. Then their line manager doesn't get around to signing it off, so employee chases accounts team, and accounts team have to chase approvers. Senior people are usually worst.
The accounts team then hand to justify why they processed a claim despite x,y and z...to external auditors.
Setting an expectation that if you don't get it right up front, the computer won't let you through is a very powerful selling point to a CFO who has to control this rubbish
In the US, maximum allowable per diem rates vary by location and are set by the Internal Revenue Service.
Unless your per diem is completely over the top, you'll need to expense flights, hotels, team dinners, client dinners, etc.
A few years ago Google, for example, cracked down and started requiring company cards for expensive purchases like flights.
I even remember a colleague when I worked in California boast to me about his $10000 claim recipe and the kickback.
Since it was a 5 man startup at that point I don't think it was worth the credit risk though.
Ye well ... dunno who to blame then in the CC mess.
This only adds up to a substantial benefit if you’re doing major international travel (carefully) or handling big-ticket hardware purchases on your card. Meals and travel is almost certainly a net loss.
This is a significant underestimate. Consider that the Citi Double Cash card gives unlimited 2% cash back on everything, and that a lot of other cards offer even higher rates (I've seen 5%) for individual categories.
They do care about making sure they reach the volume commitment on the bulk airfare and hotel rooms they have prepaid for in order to receive a large discount.
Prepayment that only works for the airlines and hotels if they’re not simultaneously discounting this large corporate customer’s rates AND paying fringe benefits to the end user.
And since they can’t tell you to not travel, this is generally why large company CFOs have policies in place that mandate these categories of spend are done via their travel agency, which was dominated by Concur prior to the SAP acquisition.
I haven't experienced that, though I presume you have.
First full time job I had a biz amex. Last full time job was a biz visa. Both with my name on card and neither were mine personally. I didn't pay the cc bill.
I had to do expense reports and categorize each item.
(edit: terrible stt)
To my understanding, from conversations with people on the finance side of things, the point of this game is that:
1. If you get everyone to do their own expensing; and
2. If you set up a system that's robust against mistakes made by people with no understanding of corporate finance, which is almost everyone in a given company (that's the "if you don't get it right up front, the computer won't let you through" principle you mention); and
3. If you tell everyone to use it and that it's their responsibility to be good stewards of corporate money they have the privilege of using, etc.; then...
4. ...you get to hire less people on the finance side of things.
I think it's another case of large organizations chasing legibility[0], to the detriment of everyone within. It's similar to what a lot of other office/business software does: it lets the company get rid of a class of specialist, whose salaries are clearly visible on the company books, by distributing their work more-less evenly across everyone else in the company. This, obviously, causes everyone to lose productivity, as they're being distracted with work that's outside their specialization/expertise, but that only adds up to an "unexplained" overall small productivity drop of the organization (and the frustration of most employees) and is not clearly visible in the company financials.
--
[0] - To borrow the term from Seeing Like a State.
If you lose a receipt, or aren't given one at all, for a reimbursable expense, should your employer just tell you that you're out the money for it?
And employees are definitely going to spend the time necessary to claim expenses. They may skimp on other tasks, but expenses are money, so they are definitely not going to miss that one.
Big companies love metrics and spying on you, and the time spend on corporate apps is an easy thing to measure. So if you see that your employees spend countless hours on ERP, it should rise tons of red flags.
I suppose the thinking is that by having each employee of a big organization do a small amount of work, they can avoid hiring a central person who just accepts receipts in envelopes and deals with them.
Or maybe that by putting up roadblocks like Concur, some employee expenses will not need to be reimbursed.
The problem is that this requires educating every employee on how to use these systems. That's a waste of time and a drag on productivity. Better to hire that central person who will learn all the quirks of whatever system is used and let the other 99% of the organization do their jobs.
When companies need photos of their employees they find a photographer who provides that service. They don't ask each employee to learn about lighting technique and do the post processing according to a written set of standards in hopes the photos will look consistent. But with Concur that's what happens.
I don't think the savings would offset the system cost. But then again, I've definitely got a threshold where just spending my money without claiming is worth it for my mental health...
I’ve not used Concur but I’ve never met anyone who hasn’t preferred moving on from antiquated paper processes for expenses. Faster to scan receipts on the go, easier to track, less risk of a lost receipt, bulk of expenses are incurred via a digital process anyway.
In a big company that person who 'just accepts receipts and deals with them' will have to enter the receipt into an expenses system like Concur anyway.
Personally I find it much easier and faster to input it myself than to put everything in an envelope with enough information in the envelope to allow an admin to correctly expense/bill it (e.g. cost centers, an explanation for why the hotel was over the policy allowance, a bank transaction for that thing I lost the reciept for...).
> When companies need photos of their employees they find a photographer who provides that service. They don't ask each employee to learn about lighting technique and do the post processing according to a written set of standards in hopes the photos will look consistent. But with Concur that's what happens.
IMO the better comparison would be hiring an in-house photo team which has to be used any time anyone in the company wants a photo of something.
It's quicker for me to take a photo of this prototype I made than it is for me to arrange for the in-house photographer to come.
Except for one firm of consultants who seemed, to me, to be chaos personified. Their expenses would go on for dozens of pages and never tallied to the amount they were claiming. In the end I resorted to hand-drawing an 'evil eye'[1] and taping it to my in-tray to try and stop the worst of their excesses reaching into my working day. Getting a promotion away from that work also helped.
I think that's precisely it; see my other comment about legibility here: https://news.ycombinator.com/item?id=34294834. That's the conclusion I reached from listening to overworked finance people complaining about their understaffed departments handling the "audit seasons" around expenses.
And I think there's a lot more of it happening, mostly thanks to software - starting with the ubiquitous office suite, that made everyone their own secretary, manager and graphics department.
> When companies need photos of their employees they find a photographer who provides that service.
Do they still? My last experience with this is the company asking every employee to please kindly submit a current photo by themselves.
Obviously, the error in this kind of thinking is assuming that "having each employee of a big organization do a small amount of work" is free, because "it's just a small amount of work". It is, but a) the more iterations of this happen, the more of it adds up, and b) it's disruptive to every employee, as it competes for time and focus with the specific jobs people were actually hired for, slashing productivity across the board.
Numerous large American tech companies are using that for supplier, including Intel, Google or ARM. They used to have their own system, and they moved to that abomination.
Everything is counter-productive, the UX is insane, but to give you an idea: you have a list of PO or invoices, but you cannot click on them to see them, you have a three-dot menu that allows you to get an email to "act on the PO"... They broke the simple idea of an hypertext link... I'm not even talking about changing a bank account...
These companies have revenue in the billions. They have enabled large companies, since before many of us were born. Your payroll and accrued sick leave and benefits deductions and AP/AR doesn’t miss a beat because of these companies. Apple runs on SAP (and Oracle and Workday), mind you. So does Google.
Are they bad or is it the crowd here?
I didn't take it as such. And you really helped, because was a total loss with OP's comment...
Someone thought SAP was bad and messed up an implementation, therefore people who think SAP is bad mess up implementations.
Someone thought snow was white and killed 50m people, therefore people who think snow is white are mass murderers.
Now someone has to come up with something that solves all the other challenges industrial enterprises have. And tell them whatever alternative was developed is actually so much better that an ERP system change is warrented.
On the other hand, we have "beautifully hand-crafted products with love" made in the best languages human can invent (your favorite language only HN crowd dicking around), and those companies are barely paying their developers and bills. But hey, their products are auto-scalable, kubernetes managed, disrupting the industry, powered with green, recycled energy, and vegan food with hand-picked ingredients...
I'll leave it to the astute reader to concur with the pattern here.
Where is this weird (and frankly suspicious) hostility against either of those even rationally coming from?
No need to act as if "crazy HN hipster developers" (apparently your perspective) are the only ones who don't understand that there is a very good reason why SAP is still used at Apple and Google.
I was fortunate to be in touch with renewable energy companies (professionally and indirectly), and you'd be surprised how unhealthy those things are for the environment on a grand scale. But, YMMV...
Otherwise why inject politics and talk about triggering? You are doing the framing.
As someone who has a degree in CS who had to go through "enterprise programming" courses, it's honestly appalling how SAP and Sage are. They're like the Teslas of programming, people who don't know better swear they're great but you can _see_ them coming apart at the seams, and you know that you could do better because you have, back in university.
add to that a healthy dose of anti-establisment and you get the hatred at SAP and Sage, bad products that are viable because they were first to market 30 years ago and now have the necessary connections to appear legit to unknowing decision makers. They have revenues in the billions because the sell the same software as 20 years ago at a premium but switching would be too expensive, so the clients stay.
I have never enjoyed using SAP or Oracle products.
But have used concur and been parallel to some of the other products…
I can see how it gets tiresome reading repetitive arguments, but a solid theory undergirds all the criticism: that one way to beat SAP, oracle, Microsoft, etc. Is to provide a better user experience.
None of that means the software is good: we can recognize the smart business choices those companies made and also abhor their products.
They provide what Alan Cooper calls "Dancing bearware" - it doesn't matter how badly the bear dances, they're just grateful to have a bear that can do it at all.
There was a comical reference to Tesla. If you’ve only experienced the user experience headache that Tesla forces on you every time they have a major software update (at least 2x a year). The muscle memory has to adapt.
I work in this industry, though I work almost exclusively with non SAP related stuff.
On one hand, the problems being solved are complicated and for some products, there are decades of embedded domain knowledge, and they have a lot of products, and can basically do anything any kind of business needs to be able to do, and integrate it somehow with other parts of the system.
On the other hand... large parts of it are absolutely horribly engineered. ABAP is a programming language that makes me recoil in disgust. What it is is the result of polishing the same turd for several decades, seemingly without one person or group ever trying to think about the direction or bigger picture for the whole stack(which is just impossibly large at this point, and like a tentacle monster from another dimension).
It does work, but mostly because people have been hammering out the kinks for thousands of person-years. As for the things said about the consulting industry.. they are definitely true.
(Business success and the craft of software development, or of user interface design, and so forth, are related, but not the same thing.)
"Lidl’s €500 Million SAP Debacle"
https://www.henricodolfing.com/2020/05/case-study-lidl-sap-d...
Anyway, for non-compliant cookie dialogs, I accept and then remove the cookies manually. Track that.
They might enjoy a complaint to the relevant privacy authorities as well
Can anyone remember the name of that awful booking system Oracle sells?
[1] Example of both: The new Berlin airport
In automobiles, industrial equipment, electronics, software, etc. you can sense when Germans were involved. I've accepted that it's something I'll never understand, and that they probably feel the same way about American designs. Same planet, different worlds.
About cars: Do you mean things like the VW scandal? Or do you mean actually shoddy engineering in cars?
I think the reputation for engineering stems from darker times.
VW scandal is different - it's what you get when you set unreasonable and actually conflicting requirements.
The most important factor is to make sure you weight your results by two numbers - one that tries to score the match of the result to the search string and another which is a more absolute weight (which could be population here, simplistically). Without that search will look like this, and these engineers will always say, “this looks good to me? Just finish typing the city and it’ll give you what you want?”
It also inverts incentives in other ways. I'm narurally inclined to be frugal, but that would often involve buying 3 or 4 smaller things to eat rather than a large meal. Since I know that would massively increase my own expense filing hassle, I instead tend to make individual purchases which are larger than I otherwise would.
Point is though, you are right, the majority of users of these systems are very removed from the purchasing decisions. And therefor the UX for them is very often subpar.
So far, and the last time we discussed Concur here, we stop at comments about lines in reports, how bothersome it is to upload receipts and how all of that is taling away minutes of peoples othereise highly productive time on their job.
The fact only UX, which is different from UI, was again adressed, and none of the other points I mentioned, kind of proofs my point.
I get you have some knowledge here and so want to talk about it, but should do so in the knowledge that you're doing so in spite of its relevance, not because of it.
In a similar way if someone criticises the Amazon UI, coming in saying, "But no one's addressed my comments on how hard logistics and warehouse management are" would be a non sequitur.
I haven't convinced you of that though, so I think that's that.
The UX is completely atrocious and the help is even worse. It took far too long to figure out how to make an expense request and they wouldn't accept my valid bank details. Our accounting team even had to send out a company wide message telling everyone to change their password, because apparently you can have both SSO and password logins with the default password set to something like expenses1234.
There are no easily accessible documents. Instead you have to watch videos or wait hours! on hold to speak to someone. We came from Spendesk which was completely intuitive.
On top of all of that, many people had to wait weeks for a reimbursement. I can't garantie that it was because of Concur but I wouldn't be surprised.
The drop-box looks like it does because it's good enough and there there are no business pressures on the market for SAP to fix it. The manager gets to keep their job by buying SAP and the low level staff needs to deal with such stupidities (and much, much worse ones) ad infinitum.
This stuff drives me crazy as there are people who just don’t care. I can’t imagine ever looking at those results and not doing something simple to make it better.
Getting revisions done, let alone done correctly was a struggle. None of us were mobile developers and thus none of us knew Objective-C nor Java so our ability to properly review or modify the code we received was pretty limited. Our app suffered for it, it became picking our battles and small things like the order of a drop-down fell by the wayside.
In a perfect world we would have stopped working with them but in reality we’d signed some sort of agreement I’m not entirely privy to and we were going to use them for a set amount of work.
A stapler, a letter, a pen, an envelope and a secretary.
That is how I always done it were I worked and it worked flawlessly each time.
During Covid WFH I had to send email and take photos of the recipes so it was abit more cumbersome but I managed to cope.
I emailed our “concur people” about it, and they did not even understand what the issue was. I have since forwarded the case to our finance and IT, but am not holding my breath on getting some kind of answer.
Assuming all the dates were set up the same, it could be that the exchange rate are "latest at creation of expense" (or just "latest") and the rates had been updated in the meantime.
I'd usually expect the exchange rate to be the rate at the date the expense was incurred but...
This would be hilarious but totally in line with the rest of the tool. Takeaway would be to do the expense reporting when the conversion rates are advantageous!
And the complexity reminds me of mainframes' UX.
And the UX person, if there was one, didn't raise any red flag, or if they did nothing was done."
...and all of the no doubt uncountable complaints by the poor sods being forced to use this pile of crap by senior executives, who have secretaries to fill out their expense reports for them and get invited to luxury junkets by SAP, were utterly ignored
But these companies do something right. If not software then in sales or marketing.
>Assume that bad technical decisions are made rationally, for reasons that are not apparent.
It could be generalized:
>Assume that acts or behavior you don't understand are made rationally, for reasons that are not apparent.
SAP, JIRA, SalesForce, Oracle, ...
MS Dynamics seems to have some good ideas when it comes to field services (I never worked with that, so). INFOR is rather good when it comes to project management and project driven manufacturing and planning. So, pick your poison, because no system will satisfy all you wishes, but the decent ones will satisfy almost all your needs, one way or another.
One of the big benefits of picking SAP is the huge pool of people with relevant experience, from developers over users (from simple employees over key users to super key users) all the way to those "controversial" consultants. And that across almost all industries, countries, regions and functions.
It takes 2 people client side to hand hold third-parties through the labyrynth, and 1 person at the third party to desperately go back and forth to those 2 people to check if everything is ok and to answer questions about why things aren't progressing.
Literal facepalm.
Have had the same experience with two other clients recently. What is going on with SAP?