331 karma · joined March 14, 2015
Even if you do a content aware hash where you break the file into chunks and hash each chunk, you still wouldn’t be able to magically recreate the hash of a CP file without also producing part of the CP.
In environments where you’re prioritizing performance I’d still argue streams are likely your best bet when the size of the file to be parsed is not a constant. You wouldn’t want to load 50 large files into ram on a server environment let alone a phone.
If your input buffer is a bunch of tiny 10 KB files and you trust them? Sure, load them into memory and access their indices on the stack. Make sure you reuse the buffer to avoid unnecessary allocations.
If you want parallel processing with zero-allocations then streams with an array pool for their backing buffer are the best bet.
Not loading arbitrary files into memory will always be safer than doing so.
As for decoding - I believe the functions for validating if an array of bytes is an image should be far removed from the decoding and presentation of those bytes to the frame buffer. You don’t need to decode a JPG to validate that a file is a JPG. It either conforms to the standard or it doesn’t; the pixel data is irrelevant.
Don’t load them into memory, parse them as a stream byte-by-byte in accordance with the standard for the codec, check every offset before seeking, and reject images that don’t conform to the standard.
And of course, a ton of fuzzing to accompany it.
We're unleashing the future of computing, collaboration, productivity, and development by using real-time video streaming to change the way that desktop software is built and distributed.
We've opened source all of our code challenges, so if you're interested in any of our open positions you can apply by following the instructions on this page: https://rainway.work/Code-Challenges-f1bc61fecf754d06be253ad...
Enjoy your sad, weird life https://andrew.im/
I love the privilege on display here. You can ignore racial inequality issues because "they are too distracting," meanwhile I have to wake up each morning, read yet another one of my brothers and sisters has been killed by police, and I still have to run my business. Your mediocrity exist in your complacently to the status-quo. On a daily basis we innovate, build, ship, and push tech more than you could ever hope -- and we don't need to ban "political discussions" at work.
If a business chooses to condemn something without backing up that solidarity with meaningful resources and capital, they should be criticized.
- The benchmark code is present in the laboratory directory of the repository.
- We don’t compare to Capt’n Proto because it does not have a stable web-based implementation, at least not one that has the features that make it so fast natively, so there is nothing to compare.
- Flatbuffers are fast but have a notoriously awful API to work with while also creating their own non-standard data structures in languages like C++. Bebop generates standard type-safe code.
- Bebop doesn’t try to compress data other than strings. This is because we don’t want to be responsible for compressing trailing zeroes when faster compression algorithms exist that can be down after encoding. Also most data is tiny.
- Bebop supports discriminated unions and has a much more robust type system than Flatbuffers.
- We’re not convincing anyone to use our stuff. It was made for us and open sourced because it was useful; we don’t need people ripping out their current serializers if there’s no pressure to do so.
I’m sorry I’m not a model minority in your eyes, but it’s rich you’re telling a black person to spend more time with black people. These aren’t just my lived experiences, it’s subject matter that comes from thousands of hours of talking with the very people you claim I should spend more time with.
You’re so uncomfortable with race and acknowledging racism you’d rather pretend you are the expert. I’m not “too far gone,” I simply live a life you can’t bring yourself to understand.
If it’s such a ridiculous statement then I invite you to provide anything backing up that claim. Because all those resources you claim are available to Black Americans are often the very things lacking in Black communities: https://www.epi.org/blog/the-racial-wealth-gap-how-african-a...
I mean Christ, have you even heard of a food desert?
Systemic Racism has always slowed the economic progress of those whose family roots are tied back to the first generation of slaves. It’s easier to build wealth as an immigrant than it is to build wealth after generations in a system that churns and spits out Black bodies without a second thought.
This essay talks specifically about the tax of being successful and Black in America: what did your comment aim to do other than try to dismiss that people with a different skin color have difficult lived experiences?
Having experienced actual racism I can infer when someone is making a comment from a place of privilege vs. being malicious. The entire point of that section was to frame things.
You’ll have more job opportunities presented to you, more people willing to take risk. Of course it also helps not to have hundreds of years to systemic racism on your next as well which allows other members of your family to likely not be poor — thus able to support you.
You can’t erase race because you don’t understand it and it makes you uncomfortable to acknowledge that being white has its benefits. That’s why the American Dream is so far out of reach to many: https://andrew.im/essays/a-journey-to-the-american-dream
You actually don’t need to sign up for Discord, emailing support@rainway.com is enough. You can also close your account from the app.
Discord is a convenient medium since our audience largely engages with us through it daily. I’ll update the article to reflect all the valid options.
And to be clear, our business model isn’t in data nor do we share data with others. Outside of basic engagement telemetry we don’t even know what games you own.
I'm also really surprised to see this old blog I wrote posted. I hope everyone enjoys it.