A high-performance, zero allocation, dynamic JSON Threat Protection in pure Go | Hacker News Reader

A high-performance, zero allocation, dynamic JSON Threat Protection in pure Go

github.com

40 pointsankuranand17 comments

The author should describe this more clearly, without jargon or the word threat which seems an odd choice in this case. I had to read the whole readme to figure out it does length and depth validation on a json payload. Hopefully it handles streaming data.

All of the validation functions work on []byte. The io.Reader function is unimplemented.

Thanks for feedback, while "JSON Threat Protection" is also a quite used terminology, but yes would provide an description with common word too. Currently Streaming handling is in progress.

What is JSON Threat Protection?

JSON requests are susceptible to attacks characterized by unusual inflation of elements and nesting levels. Attackers use recursive techniques to consume memory resources by using huge json files to overwhelm the parser and eventually crash the service.

JSON threat protection is terms that describe the way to minimize the risk from such attacks by defining few limits on the json structure like length and depth validation on a json, and helps protect your applications from such intrusions.

From the description, I'm guessing that using a specific object json parser (such as https://github.com/mailru/easyjson ) would not be vulnerable to such attacks. They will just skip characters if the field is not found. And if a nested field is found inplace of a non nested one, they'll just error

I don't get it. Are there really parsers that will allocate anything more than O(n) memory for a given maliciously crafted payload? Are there any examples/studies of this?

It seems that limiting your marshaled input size to something sensible for your application should do the trick just fine for most cases.

It doesn't allocate itself, but it effectively forces a big allocation by only supporting []byte inputs. Typically people directly pass the request body to json.NewDecoder, as it implements the Reader interface.

There is `func (v Verify) Verify(reader io.Reader) (bool, error)` function in the api, which will support the streaming part. Currently it's WIP.

what is a 'json threat'? What is the threat actor here? It seems like you do some depth / length checks on arrays and objects but it also 'validate's json somehow? https://github.com/ankur-anand/gojtp/blob/master/jtp.go#L341

JavaScript object notation(JSON) is vulnerable to content level attacks. Such attacks attempt to use huge json files to overwhelm the parser and eventually crash the service.

JSON threat protection is terms that describe the way to minimize the risk from such attacks by defining few limits on the json structure.

Yes It also validates the json.

if the idea is "limit the size of JSON" you already have http.MaxBytesReader or io.LimitReader

This should be the parsers job, not an external validation service. Those threats are very parser specific.

There are situations where you do not want to parse the JSON, but do want to ensure that the JSON is not going to cause a problem. Such as a gateway. It would be a PITA for the gateway to have to know all JSON schema of all services it is protecting. There are XML validators that that perform similar functions.

JSON threat? Like overflow of requests?

JavaScript object notation(JSON) is vulnerable to content level attacks. Such attacks attempt to use huge json files to overwhelm the parser and eventually crash the service.

JSON threat protection is terms that describe the way to minimize the risk from such attacks by defining few limits on the json structure.

I see. Is that really useful? Worst case is a 500?

It validates the structure before fully parsing it and allocating lots of objects you normally wouldn't.

Reply on news.ycombinator.com