SOAP: The 'S' stands for simple (not really)
harmful.cat-v.org
harmful.cat-v.org
It turns out that our framework has SOAP built in, so I had it suck in the WSDL and make a sample request. It was rejected with a generic error. As it turns out, our framework was generating requests that looked like this (simplified with much SOAP nonsense stripped out):
<SOAP-ENV:Envelope>
<Authenticate xmlns="http://bigcorp.example.com/elements/global">…</Authenticate>
</SOAP-ENV:Envelope>
…and the requests in the API documentation look like this: <SOAP-ENV:Envelope xmlns:bigcorp="http://bigcorp.example.com/elements/global">
<bigcorp:Authenticate>…</bigcorp:Authenticate>
</SOAP-ENV:Envelope>
the difference being that our framework declares a "default namespace", while their API expected a "namespace prefix". As far as I could tell from reading the spec (always a bad sign when you're using a standard that both ends support) the difference shouldn't matter.- - -
I went to my boss. He showed me the old code; all the requests it needs to make are hardcoded based on the docs, and substitute XML-encoded variables in the right places.
Save a few minor changes, that's the process we still use today, and it fucking works.
http://wanderingbarque.com/nonintersecting/2006/11/15/the-s-...
I am glad to see it get more attention. It is a classic and it is really about more than SOAP. It is about the way technologies get hyped and over-sold and over-promised, and then made unusably complicated. SOAP is just the example.
In the excellent book "Antipatterns," I recall that this is called a "Wolf Ticket:"
Wolf Ticket is a product that claims openness and conformance to standards that have no enforceable meaning. The products are delivered with proprietary interfaces that may vary significantly from the published standard.
http://sourcemaking.com/antipatterns/wolf-ticket
Wolf tickets are sold as safe choices because of their interoperability, but of course once you have anything non-trivial built upon them, you discover that the interoperability doesn't actually work and you are left locked into your vendor.
Of course, since that times I've alway advised against web services and so far I've succeeded in avoiding them.
The payload is JSON over HTTP.
JSON over REST seems to be flavour of the month FWIW.
It's not (just) a hot new thing used by new faces who've never really known any other way.
I only qualified the statement as technologies change quickly (only 12 months age we would probably have been talking about XML over REST) and things will no doubt change again.
A key point the writer alludes to (and an IMPORTANT one) is that SOAP was hyped by tool vendors because it game them something to SELL you. There's really nothing to SELL with RESTful stuff, it just works and is inherent to the internet, therefore tool vendors aren't exactly pumped up about selling you anything.
Many businesses wind up in this situation by chance, and then have a very painful journey out of the casual schema they created when they were a small business into something which matches their larger business.
Unfortunately, IMO, supplying a good answer to that question is the 'northwest passage' of software development.
I've spent years at it on and off. I can create such a layer, but the solutions I've come up with are unwieldy from the perspective of the business layer programmer, which is an inadequate solution.
--
If I have another attempt, I'll try to create something which would look to the observer quite a lot like visual basic, but where instead of constructing a Windows forms, you're constructing a state, which consists of a set of forms and a variable store.
The datatypes for the fields would be something like this:
Readonly
Boolean
Text (untyped)
SingleSelect (conceptually like html dropdown)
MultiSelect (like checkbox)
ExposeObject (the object is tagged, client
can bookmark these these objects in its
session)
ReferenceSocket (the socket indicates the tags it
accepts, exposes compatible tags, connection point
for bookmarked objects with compatible tags)
Link (like html link)
Action (like html submit button)
A client API then migrates this with a query language that navigates the states. An example of a dumb session that knows nothing of the app to which it's connecting, and which is rendering a UI on the fly: # Remote application knows nothing
list all
# It gets back a description of the current state, 'login'
# Remote application logs in to session
assert form is login
set username wizard_2
set password whatever
action login
list all
# Remote application has control again
# Remote application wants to know what operations are available
assert form is portal
list actions
# Remote application decides it wants to go to person
assert form is portal
link "manage person"
list actions
# Remote app decides to add a person
assert form is "manage person"
link new
list all
# Shows fields and actions
assert mode is new
set firstname Wiz
set lastname Ard
set age 40
select male from singleselect
action save
# Application throws an exception with details
# Client app
assert mode is new
set data_of_birth 08081971
action save
list form
Just as SQL is a flexible language for a relational model, this language is a flexible language for an interactive state graph. A client progrmamer who know the state mapping of the app could write tighter instructions than this.That client app can be anything: desktop app, unit test framework, web app, direct rpc over http.
Conceptually the situation of states, objects and actions is very similar to a text adventure, but I'd been working on it for years before I realised this.
Every time the state progresses from one to the next, it keeps a copy of the previous state in memory.
This logic layer has the database underneath it. The DB is now abstracted away from the user-facing application. Major benefits of this approach:
- You can design the application with almost no concern to user interface. Then you can create a streamlined user interface later that leverages the correct business logic. You can separate the costing for both projects. That's how I came to the problem. It would revolutionise software consulting.
- You can wrap a dumb client interface around the engine. Want your entire business logic exposed to the web, securely? Or RPC? Or mobile? Or for the blind? Done, with literally zero extra effort.
- If we had a relational database that had version numbers for its internal state (think git model), you could write applications that could unwind and rewind. Takes a lot of memory, but you could store the states in a nosql database.
--
OK. I'll try to answer your question now with the best I know of current techniques. Things using current technology that somewhat address the question you asked:
- Filter all chat with the database through stored procedures. Problems with this: (1) non-stateful, you don't have the user's application session; (2) stored procedure languages are a pain to work with. This is the sort of solution you'll find most often in large organisations because you can hire a single known skillset (DBA) who knows both stored procedures and schemas. It's awful but it works.
- Put a HTTP layer between the database and the application. Do all interaction with the database through this. In the payload that users send in they could (1) name the action they want to do such as create person and (2) supply the arguments they should send in for that business case. Basically, here you're doing stored procedures but in language of your choice. Latency is higher than with real stored procedures.
- Have a rule that all application programmers integrate with the application using Apache Cayenne or Hibernate or something similar as an object-relational modeler, in a strongly-typed language. Insist on use of Data Access Object pattern (see Fowler) for any original entrypoint to the object graph. Now when you refactor your database, regenerate your schema, and use the static typing checking in Java to rapidly adjust your application to the changed schema. You'd need to get all client apps to do this, but it's a lot more rapid than if you've got application users using SQL.
I sited Fowler, thinking _Patterns of Enterprise Application Architecture_. I've checked and was mistaken. He does somewat similar stuff in QueryObject, but I'm thinking of a much higher-level concept.
I'll desccribe the relevant ORM patterns that I'd use. Most of my experience is with Apache Cayenne (which I strongly recommend), and some with Webobjects EOF before that.
This has been interesting to review. I haven't done serious work in either a compiled language or application development for years. It's interesting to come back at with fresh eyes.
= Entity
Your ORM will have tooling that will generate these classes for you from a tool or definition document.
There is a class per table in the schema. Each instance of Entity corresponds to a row of data in a table. Methods for foreign keys return objects for the linked class.
Your application would have classes Person, Organisation, etc.
ent.Person would have a method ((ent.Organistion) toOrganisationEntity()). end.Organistation would have a method ((List<ent.Person>) listPersonEntity()). Once you have an entity, the app can find related things via lookup.
Say you had the Organisation object, and wanted to know all the subjects that people for this company were enrolled in. You bounce through the graph. From Organisation you can get to Person, from there to JoinPersonSubject, from there to Subject.
An improvement on this is to have a separate base and application class. Your ORM generates the base class. You generate your app class and have it extend the base class. You put custom logic in the application class.
Example: Cayenne generates com.company.app.ent.base.Person. It has methods corresponding to each column in the person table. You create com.company.app.ent.Person (different package name) and have this extend the base class. You put convenience methods like ((String) getFullname()) in there.
You can then easily regenerate the base class without impacting your custom logic.
= DataAccessObject
(The summary sentence doesn't make much sense until you understand the concept, so I'll explain with example immediately afterwarss.)
Summary: You use a DataAccessObject for getting entrypoints into the graph of data held in the database.
Explanation: In Entity I described an example of bouncing around the graph. Say you had a user in an education system, and they wanted to get a list of all the organistaions associated with their office. You'd use a DAO to do this lookup.
Strangely, these patterns make a powerful coombination. You'd think that you'd need to add a bunch of extern
Your ORM probably won't generate these, but you should abstract them away into distinct classes like this rather than putting static methods in your
DAO functionality is commonly bolted into Entity classes as an afterthought using static methods. Ugly. Awful. Instead, consider DAO to be a distinct pattern, extending from an abstract base classs. The base class constructor will accept the current user as an argument. There will be a method that forms an ORM join between the current user and the
the current user you'd have common functionality such as a join between the
Unlike Entity, there is not a one-to-one mapping between data access objects and tables. You just create them as you need them. There might be an OrganisationDAO that has the query above, and some other queries in it. But then there might be a situation where you're doing a strange mega query across Organisation, Location and SalesTarget that doesn't seem to be specifically associated with any of those tables more than another. Here, you just create a new DAO.
= Session
The user's application session. In this you'll have a single instance of each DataAccessObject. That way your application can do lookups by going session().orgDao().listOrgWithAccountsAtUserOffice().
If you image a pyramid of service types:
Business Process
Business Operation
Component
Infrastructure/Data
Then you want to shoot for building "Business Operation" type services. These give you the maximum reuse in an enterprise as they encapsulate a useful atom of business functionality. They hide the complexity of an enterprise's IT systems from consumers and typically use semantics that are business not IT system specific. For example CreateOrder, CreateCustomer etc...
Component services are the other useful one. Typically used to wrap a system with web services. This eases integration and allows for aggregation of these services into Business Operation services.
For instance the CreateCustomer service above might call CreateCustomer in System A, RegisterMember in System BB and write a record into a the Users table of System C (sometimes its impossible/cheaper not to use web services).
As for as the consumer (the helpdesk app, the self service portal etc.) is concerned they just created a Customer within the business and they don't need to know anything about Systems A, B and C.
Think like this, you need to add the compulsory field of PO Number to your invoice table.
It wasn't compulsory before.
How do you not break everything?
It's impossible.
Or, even worse;
Projects can now have multiple categories. You change your code so all old requests still work, but suddenly half your systems give users the flexibility to add multiple categories and half don't.
Why's that worse? Because you suddenly have people creating projects in one system that only allows one category and then logging into another to add the extra categories because IT doesn't actually have to fix it. Total user nightmare.
And to top it all off all, and this is the actual killer, developers are having to use your never quite finished system which throws weird errors that people can't google like 'unable to add project', instead of 'violation of Foreign Key Constraint, FK_Project_User, on table Project' which tells them exactly the error.
Use standard DB access because there's a wealth of help out there, your inhouse system can't compare unless you're a big company.
Basically you've been asked to fix one of the fundamental problems of why enterprise programming is hard.
As for error codes, it doesn't seem non trivial to explain why your validation failed.
The only useful thing SOAP has done for the entire development community is to give lead developers or tech managers making hiring decisions a huge red-flag. As soon as anyone mentions SOAP with enthusiasm (instead of the disgust it deserves), you know it's 'no-hire'
The alternatives were either that or a custom pipe-separated format.
I'll take serialized objects over XML all day.
Prior to this discussion, I wasn't aware of JSON on REST, I guess I should catch up a bit- I live in an awful legacy-app development world, which we need to expose on a crumbling tower of VB6 and .NET apps that interact with it externally - VB6 is the "new" part btw, the old part is an old SUN technology called FORTE 4GL UDS.
The case of having identical SOAP implementations at either end (eg .NET 2.0 <--> .NET 2.0, Java w/ Axis 1.4 <--> Java w/ Axis 1.4) is the one scenario where SOAP works as you'd hope.
You ended up transferring ~10x the size of the file. I suppose gzip would have helped somewhat.
(sorry if by "gzip would have helped" you meant that)
Step 1: Ignore all tools which supposedly make things easier (very important)
Step 2: Find the web service API documentation
Step 3: Ctrl-C example XML request
Step 4: Ctrl-V example XML request into program, replacing appropriate parts with program variables
Step 5: Parse the response. You're on your own here... Godspeed. :)
Practically: ctrl-C, ctrl-V, and call them when it doesn't work. More often than not it's either A) incorrect documentation, or B) you hit a bug (yes, this means their sample-code doesn't work. It happens frequently). A good chunk of our request-data builders are simply string manipulators, because some of the APIs we call are nitpicky about the order of XML attributes in a tag.
I laugh when people tell me XML works better "because it's a standard".
"Dev: What happens if I GET the endpoint’s URL?
SG: Don’t know. Using GET is undefined.
Dev: Hrrm. And what happens if I move the service to a different endpoint? Do I get a 301 back?
SG: No. SOAP doesn’t really use HTTP response codes.
Dev: So, when you said SOAP uses HTTP, what you meant to say is SOAP tunnels over HTTP."
The beauty of REST is that it is representative of how the web works. With SOAP you're almost always limited to the use of POST methods and when I go to read another developer's code, I see their own defined nouns and verbs for resources they are sending/receiving like "getUsers", "getContacts", and "createPerson". These often are ambiguous or don't match up with the actual resource they are trying to work with.
Endpoints in REST are self documenting, you can read what it is that you're trying to do based on the endpoint and the HTTP verb (put, post, delete, et al)
Thanks for trying SOAP people, it was a noble effort but it's game over I'm afraid.... oh wait, I just saw on Wikipedia that these were Microsoft people. In that case, screw em. SOAP is wank! YOU'VE WASTED YOUR LIVES!!
On the second thought: The same companies made SOAP right after and along with CORBA. Maybe, even the same people. Don't blame Microsoft too much: IBM, Oracle, SAP, BEA - they are all ... equally "good"
I worked on a project recently where we had a team of Java guys who worked on a SOAP implementation for a month. They could not get it to work in any way, shape or form. It was handed to me, and three of us did it a half an hour. All custom SOAP, still worked.
I just don't comprehend why this seems so hard to people on Hacker News. SOAP is just another protocol and as messy as any other.
I think that's where you and most of the other people on the thread are losing it. SOAP is way messier than just good old REST.
(disclaimer: I've never been forced to actually use SOAP.)
I'm not at the office so I don't have the code at hand.
I'm still luke-warm at best on the whole ORB idea as-is, but even thinking from the point of view of someone who thinks object brokers are the best thing since sliced bread, I feel like the SOAP people got it very wrong.
This article is bittersweet because it sums up everybody's feelings pre- and post- implementation.
From my experience back at the beginning -- XML-RPC was slightly underspecified, and would have used a bit more of a firm hand about clarification of what was legal and what wasn't. SOAP was wildly overspecified and complexified to the point that you needed an army of consultants. Which is just perfect for enterprisey stuff, since you conviently have an army of consultants that need to have something to do.
IMHO, XMLRPC inhabited a middle ground that was washed away by REST and ajax/json type interfaces. It's an extra bit of complexity that's just not all that popular now, rather than a stunningly less complicated version of SOAP.
Like people that think you can just replace XML with JSON: you are missing stuff.
REST is an architectural style there is no spec, just a idea! Added to that most folk also only build REST-like services.
Implementing a client for REST based services often requires a bunch of (generally simple) coding which takes time and is error prone.
Also pure REST is really good for building data access/CRUD services but makes it hard to build RPC type services without mangling the semantics.
SOAP has several versions of the spec and a bunch of vendor-specific cruft. The SOAP use-cases from this comment thread are enough indication of that.
> REST is an architectural style there is no spec, just a idea!
Yeah, but you use REST every day in your web browser. A lot of the times it isn't strongly RESTful, but it's still REST. For example, to view this comment thread and post your reply to someone's comment.
In SOAP, if I call the getUserByID(42) method on your end, and you don't have a user with id 42, what do you do? Do you return an empty response? Do you return some kind of SOAPFault? If so, what kind?
In REST, if I GET http://yourservice.com/users/42 and you don't have user 42, I get HTTP 404. That's the only logical response.
> Implementing a client for REST based services often requires a bunch of (generally simple) coding which takes time and is error prone.
This is no different from SOAP, with the exception that with SOAP you have to have your requests stuffed into XML by some libraries which have read some other autogenerated XML to figure out how to stuff your requests.
With REST, you can write a library (that wraps HTTP calls), or you can use curl, or (for simple GET requests) you can use your browser. Your platform might not support SOAP, but it definitely supports REST.
> Also pure REST is really good for building data access/CRUD services but makes it hard to build RPC type services without mangling the semantics.
This has been said multiple times but let's say it again:
REST is not RPC.
If you're looking to remotely call procedures on your app, REST isn't going to provide that.
The only case where REST doesn't work is the case where your client demands SOAP. In any other case, REST works beautifully. Not everyone does REST right, but even when it's done suboptimally (e.g. not providing hyperlinks to related resources) it's still worlds better than SOAP.
Ha ha! I must remember this put-down - it could be used in so many contexts :)
- I spend less money on resources (SOAP, lotion, etc)
- I get clean faster -- SOAP used to slow me down
- Using someone else's shower is easy, I just take my towel. I don't have to worry if their SOAP is compatible with me.