Comment by patio11 | Hacker News Reader

Comment by patio11 | Hacker News Reader

I think this is a signal that you're probably underpriced by a factor of 2X to 10X.

So this is a sales objection. With regards to that justification, I'd want to know a) who in the business is generating it and b) what they expect to feed your answer into.

Is this just somebody who wanted to sound smart in a meeting? Then they don't need your source code; they need ~5 nice PowerPoint slides and you're done. I'd be polite but firm in this case.

Is this e.g. a risk management officer? Then OK, that is a totally reasonable ask. It's going to cost you $20k and some legal wrangling regarding non-disclosure and non-competes, but we will give up to 2 team members of yours up to one business day of access to our code in a clean room on our hardware, under supervision by a technical leader, with appropriate rules of engagement regarding questions.

Is this a business unit head sniffing for new things to build? Then I will not automatically eject out of this conversation, but my price just had another zero added to it, at least.

I’m a security engineer at a large financial organization and we audit all 3rd party products most of these audits include a code review.

This is a compliance process and it’s controlled by the SEC.

All of your suggestions would make the company that I at least work for to simply walk away from table.

This has happened more than once and all of our contracts contain a clause that if the application does not get a pass from our application security team the procurement would not continue.

Thank you for your comment. Can you add, for the benefit of the peanut gallery, what the typical cost of a one year software license you pay for is? I have a good guess but I think HNers need to hear people-other-than-me say it to internalize it.

Not OP, but we were in the same situation and required source review. We negotiated escrow, the whole shebang. License costs for our enterprise was multi-year and ran over $15 million for the project for this one app. It's not an odd or rare request.

Really depends usually in the millions especially if this is a product related to trading.

It’s also important to note that as far as I am aware contracts on this scale have multiple stages and we are paying for PoC/pilot programmes during which we usually perform our review. This usually isn’t software that a pre-sale engineer would come and setup in a few days so we are paying effectively even for a demo.

Overall we audit everything from Xerox software to custom balance sheet management products.

The process is dependent on the level of trust/confidence we have in the vendor and the risk profile of the product.

But in all cases there will be some sort of review.

For vendors that would not give us code access we request permission to pentest the product the pentest would include reverse engineering in most cases and as the vast majority of products in the financial sector are in Java the source code isn’t hard to get through reversing the byte code.

Do you use SaaS products at all? If so, do you audit source for them too, or just products that run on your hardware / network?

SaaS is usually just a pentest some we get/request code access for.

Overall it depends on the risk profile of the product SaaS products tend to have a lower risk profile becuase they don’t deal with trades/contracts directly.

It’s all about managing risks. One of the products we use is Salesforce. Penetesting Salesforce would be a waste of time for us because Salesforce has a good application security team and is a trusted vendor. However we did perform a review of apps/plug-ins that run on the Salesforce platform which we use but have much less confidence in.

Anecdotal - for a large publicly traded financial services who wants to manage their book of record software (i.e. the software that holds the truth of "dear client you hold this many assets"), you're looking at $3-5M/year in license costs.

Depending on the business context of a ML solution for a company like the OP describes, you're likely looking at $500k+, but this is pure speculation and requires more information about the customer situation (e.g. is the ML solution going to save the company money or help them sell more).

> controlled by the FCC

I'm guessing this is a typo?

Or perhaps bad voice-recognition for SEC?

Brain typo :)

Why would a financial product have a compliance procedure by the FCC?

> All of your suggestions would make the company that I at least work for to simply walk away from table.

Unless the company is already a customer and is paying for the development of the software, please do walk away from the table, I'm sure they have better things to do and better customers to sell to

Seems like FUD to me ! Maybe Sarbanes Oxley.

I'm thinking it's a typo for SEC.

That’s correct.

Typo, too many acronyms.

As for the second part I wont comment really I’m not in a place to make decisions on procurement I just brake things. I would just say that size/volume wise as far as clearings go there isn’t a better/bigger client. And everyone is the same.

Ah your other comment makes the procedure and why it's needed clearer

FCC is a typo I'm sure.

>Unless the company is already a customer and is paying for the development of the software, please do walk away from the table, I'm sure they have better things to do and better customers to sell to

This sentiment is similar to things I hear all of the time working in health care / biotech. The fact is you have no domain knowledge and not a clue as to how things work in the real world. If your company just ignores the way things are done and the requirements of the domain you're simply going to go out of business.

When the stakes are high (e.g. death/harm, massive financial losses, etc.) the business process changes accordingly.

I’ve had a financial product audited by an independent firm the client and I agreed on. In fact they still audit it on a regular basis despite the product no longer being maintained.

This also can be acceptable based on the risk; sometimes we get a full pentest report from an independent security vendor.

There is no compliance rule put out by the SEC that requires (or even suggests) source code review.

How specifically did the request for source code signal to you that the vendor's charging too little? You suspect the customer is worried that the product is really nothing because it's priced so low? High prices from a vendor signal they mean business and ward off fiddly objections? (I'd expect the opposite, higher price => higher stakes => more interest in knowing just what you're purchasing.) That an ML product with source that BigCo would want to read probably has some special sauce that deserves a high price? Just that most small vendors looking at selling to very large customers tend to underestimate the optimal price?

I guess I risk inviting you to reply "Yes." or "mostly yes" with that laundry list, because most are theoretically plausible signals and sort of statements of the same idea, but what was your thought process leading to the conclusion they're underpriced? As someone who works for a vendor this is relevant to my interests :)

How specifically did it signal to you that they're underpriced?

Because the poster is surprised by this requirement and doesn't have standardized answers to it yet.

In poker, there is this thing called "assigning a range" to someone based on their actions. You can't see their cards, but their actions might give you signal where you could say "Hmm, playing like they have a middle pocket pair and not totally air nor a monster." If sophistication with regards to enterprise sales is a spectrum like poker hands, surprise about certain parts of the process causes me to adjust my range accordingly, and that plus experience working with software people colors my estimate of pricing sophistication.

Ah, that's very helpful. As you could see from my reply I was reaching for reasons you might think their pricing was low without knowing their pricing, and this makes more sense than the ones I came up with.

No strat talk on hn bro ... ;)

At the flea market they call it "looking at the customer's shoes."

Priceless!

Another possibility is they're worried you'll go out of business. In that case, they don't need to see the code right now. You can set up a code escrow agreement in case you cease to exist.

no, every big enterprise I've worked for we do source audits on everything that is allowed to run. We check for phoning home, we check for license issues, we check for massive security vulnerabilities. It's just a cover your ass kinda thing when 100k people are going to run an app. There's a whole department for that.

Yes, the calculus is completely different when you are a 5k midsize company vs a 50k+ behemoth. There are strict, multi-layered processes for everything in larger companies, especially software procurement (including yearly upgrades!), and legal attends every meeting.

This advice is generally very sound but misses the mark slightly.

> underpriced by a factor of 2X to 10X.

The OP never asked about price, so this is pure speculation. Why is the conclusion that this is a sales objection?

> It's going to cost you $20k and some legal wrangling regarding non-disclosure and non-competes

It sounds like they haven't even gotten through procurement yet, which means they can't even suggest charging $20k for NDA's etc.

> Is this just somebody who wanted to sound smart in a meeting?

This absolutely does happen, but it doesn't happen when the company is discussing POC plans.

> We know that they have lots of resources and are building up internal data science team. And yet it was pointed out to me that their goal might not necessarily be to outright steal our IP, but rather to cover their bases. But we are still worried they might be "inspired" by the parts they see and get their internal teams to replicate across other sites or use cases. And we don't have the resources to litigate, nor any way of knowing they do this.

> Is this a business unit head sniffing for new things to build?

These are both hypotheses based on observations, but they are not conclusions to the situation.

The OP's company needs someone who has experience in this realm (a "grey haired enterprise software sales guy") and help them come to a conclusion before they make decisions to move this forward. There's way too many variables and open questions to offer any real sound advice.

TL;DR - for situations this complex and high stakes, relying on a HN comments will not bode well.

For the record since someone asked in another comment, I have navigated a procurement process with a financial services company with $1T in AUM, working on a $50M/year software project, as an example of experience.

Reply on news.ycombinator.com