HNHacker News
TopNewBestAskShowJobs

DarkShikari

6,761 karma · joined July 15, 2008

I'm the lead developer of the x264 project and also an ffmpeg developer. I do consulting and contract-coding work related to video compression; more details can be found here, along with a CV: http://x264.nl/developers/Dark_Shikari/CV.pdf .

My blog: http://x264dev.multimedia.cx/

My email: jason@x264.com

submissionscomments
DarkShikari··on How a language can be faster than C
Sadly, from what I recall, gcc has some internal limitations which prevent it from making significant use of "restrict".

mans or astrange can probably elaborate.

DarkShikari··on Ex Apple TV Engineer: New Apple TV UI is actually one Jobs threw away years ago
The current fragmentation of the "smart TV" market is a mess: it's something that a unified environment could easily tear through.

But making the deals with content providers to create that environment is going to be astonishingly difficult. These are companies responsible for messes like http://theoatmeal.com/comics/game_of_thrones , along with the recent removals of films from Netflix by Starz. They love their monopoly on content and would likely be terrified to death by the concept of giving up control to Apple.

DarkShikari··on BB84 Protocol, the first quantum cryptography protocol, explained
RSA cryptography can be broken quickly by quantum computers. There are other public/private key schemes that might avoid that problem, though none are widely used. Some of these rely on NP-complete problems instead of merely BQP problems like factoring.

Quantum cryptography, if correctly implemented, should be theoretically unbreakable, not merely "hard" to break -- much like a one-time pad.

DarkShikari··on Nvidia deeply unhappy with TSMC, claims 22nm essentially worthless
There are many ways to dramatically lower power usage if you're willing to accept slower speeds. Intel has recently demoed some extremely low-voltage transistors that used power in the microwatts. You can make up for this dramatically lower speed by adding far more circuitry using 3D chip-stacking.

Another possible solution is to use 3D chip stacking to add dedicated hardware for a wide variety of tasks, most of which is turned off when not being used.

DarkShikari··on Nvidia deeply unhappy with TSMC, claims 22nm essentially worthless
Intel seems pretty confident that they will be able to get to 11nm in the next 5 years or so. Past that, pure shrinkage of transistors via purely "conventional" methods will mostly hit a wall, but everyone saw that coming.

There's been a lot of work on nanoelectronics. For example, some scientists recently announced at a semiconductor conference that they had found a way to bypass the diffraction limit completely, making 2nm features with a 650nm laser. Similarly, I know an engineer who worked on a massively-parallel electron-beam-based etching device, which could also be applied similarly at very small node sizes. There's a billion other technical challenges, but there's been a lot of progress in making features on the molecular level.

If anything, this is better than expected: EUV has had some serious problems over the past few years, yet its delay hasn't stopped the next few nodes.

Moore's law doesn't require that shrinkage continuing, though, as there are other technologies available. Memristors in particular may allow scaling faster than Moore's law would otherwise predict.

DarkShikari··on Sexism in Tech. Or: CloudMine pulls its sponsorship of API Jam.
But, in my experience, othering isn't going to drive people away, not in any significant numbers.

Are you a straight white male? If so, you probably shouldn't be generalizing your own experience to everyone who isn't.

Try talking with some people who aren't. Neil deGrasse Tyson is one of many examples of people who have spoken out on this topic, and there are many open source advocates who can give you their own experiences. Lydia Pintscher of the KDE project (and author of "Open Advice") has much experience with communities and such problems.

Artificially suppressing occasional acknowledgment of differences

Nobody ever asked you to do that.

only teaches that these differences are a bad thing.

If someone wants to make a point of a difference they're proud of, they'll do it themselves. Don't force it upon them!

DarkShikari··on Sexism in Tech. Or: CloudMine pulls its sponsorship of API Jam.
Othering. Get that word into your head, and keep it there. Othering.

Othering is when you make someone feel as if they don't belong in a community, that they're different from other people there. Making stupid (racist) Asian jokes about the only Chinese-American member of your team. Writing flyers that assume that your entire audience consists of a certain type of heterosexual man. Even something small, like constantly poking at the fact that one of your fellow hackers also has an MBA. The list is endless.

Usually it's best to not call attention to such things at all, even in a positive way. Telling someone how awesome they are for being different is still making a big deal about how different they are, as opposed to helping them fit into your group.

All of these things make those people feel as if they don't belong on your team or in your community. It doesn't matter if you think it's "in good fun" or "just a joke". It doesn't matter if you think "oh, it's not really sexist". It makes people feel unwelcome. It's dumb, pointless, and it's the responsibility of any hacker to just not do it.

Othering can happen in any community that is dominated by a relative homogenous group. It turns off many hackers who might want to work with you. There is nothing inherently "straight upper-middle-class Caucasian male" about wanting to build a working product; stop acting like there is.

DarkShikari··on Why Do The Labels Continue To Insist That 'Your Money Is No Good Here?'
I think what people in some of these countries don't realize is that there is not a big enough financial incentive to do the legal work required to bring certain artists to those regions.

What exactly is this "legal work"? The Berne convention makes copyright worldwide, and your typical software company doesn't need to check to see if their customer is in the US before selling them their product. Your typical independent band doesn't perform a region-check on customers buying their MP3s. What's different here?

From what I can tell, most of the "legal work" is complexity created by the labels themselves.

DarkShikari··on Ask HN: Are we at a disadvantage?
If anything, I'd imagine you'd have an advantage. Silicon Valley is filled with programmers/hackers, but the really interesting people are those that come from other backgrounds with different experiences.

In most cases, making a successful startup is primarily a business problem, not a technical one. Seeing customers' perspectives, knowing the business you're entering, understanding the problem you're trying to solve -- these are all more important than being able to write Ruby quickly.

If your experience outside programming helps you contribute to solving hard problems like these, you're infinitely more valuable than someone who knows only code.

DarkShikari··on Virtual Machines in Your Browser
That's because most existing VNC applications aren't designed for that purpose.

There's no reason you couldn't use a proper video compression algorithm in a VNC application where applicable, or just send the whole image as compressed video.

DarkShikari··on Linus on kernel management style (2004)
Note carefully: though being his usual brand of unfathomably blunt, Linus is talking about the code, not the person who wrote it. He's obviously bothered by the approach the code takes, but he probably doesn't think the person who wrote it is a 'dickhead' -- anyone can write truly awful code, even Linus.

In this case, it looks (from a quick glance) like some Patch X seemed like a good idea, but ended up requiring Patch Y, which was really ugly, and suggested that Patch X maybe wasn't the best idea to begin with.

Of course, this takes getting used to -- part of Linus's style is to be very blunt, call his opinions straightforwardly, and happily insult anyone's code, even his own, when it isn't living up to standards. To work with Linus, you need to have zero ego about your own code -- the goal is to get the best code into the kernel (no matter who wrote it), not to get your code into the kernel.

It's not always perfect, but I generally think it's a good approach, and it seems to largely work.

DarkShikari··on Successful people are successful
How overpriced is education in America? Per year...

For a UC in California (public school): ~$31000 on-campus, $28000 off-campus (see http://www.universityofcalifornia.edu/admissions/paying-for-...).

For a private school in California, ~$55000/year on-campus.

DarkShikari··on There is no point to distributing music in 24-bit/192kHz format.
Many of the groups I listen to do do this -- this is certainly not that rare. Sometimes they go for a bit more money by releasing a separate CD with karaoke tracks, for example, but at least if you want it, it's available.

Of course, you can very easily get just the vocal track by subtracting the two. Sometimes the "non-vocal" track will still include backing vocals or the like in appropriate places, and just pull out the main vocal track.

Some musicians have even released every single track of their work separately; see "Desperate Religion" on http://en.wikipedia.org/wiki/Trilogy_(ATB_album) , intentionally inviting remixes.

For music where the vocal tracks aren't released separately, you can often pull them out nevertheless. The best is if you can get the audio in 5.1 -- vocals are almost always center-panned, which makes extracting them quite easy.

DarkShikari··on Dwolla Being Sued by Bitcoiners for $2 Million
With bitcoin if I hold my coins in a currency exchange (which is basically a bank) they can obey the court order and reverse transactions.

It would be trivial if it was that simple, but the stolen funds were of course withdrawn from Tradehill, and so are no longer in control of an exchange.

It's no different from a criminal running off with a bag of cash; how are you going to "reverse" the transaction of the money being physically picked up, put in a bag, and carted off?

DarkShikari··on Dwolla Being Sued by Bitcoiners for $2 Million
There is a slight relevance of Bitcoin: Bitcoin transactions are irreversible. There is no real technical means to reverse a transaction; it is simply not possible. Even if there was a court order to reverse a transaction, unless the physical recipient could be identified, located, and arrested, the court order would be meaningless.

This is in contrast to almost any other exchange system (besides hard cash), which could, at least technically, force reversal of a transaction. See for example Paypal, Facebook credits, money in an MMORPG, shares of stock, and so forth.

DarkShikari··on Trivial Artificial Neural Network in Assembly Language
Even compiled intrinsic SIMD beats most hand-coded kernels these days

Intrinsics basically are hand-coded assembly: you're specifying every single instruction manually, just in an obfuscated syntax, and without a proper assembler preprocessor.

At that point, it's way easier to just write it straight by hand.

Also, there are still plenty of things compilers are terrible at with intrinsics. This includes things like:

1) Register allocation. Compilers, even "good" ones, will throw things on the stack willy-nilly. The biggest problem here is that the compiler hides register utilization from the programmer, preventing him or her from seeing how many registers are used. If you're aware of register usage, you can tweak the algorithm to avoid the problem.

2) Compilers are truly terrible at making up calling conventions for calls within assembly (e.g. to a subfunction). Humans can make whatever calling convention they like, which can give great performance improvements in functions like FFTs (see: libav split radix fft) that are inherently recursive. In the case of the aforementioned FFT, Loren was able to make a handwritten-asm Altivec version around ~40% faster than intrinsics, IIRC.

3) Compilers are not very good at organizing code; in many cases, it is useful to split up code and reuse blocks to save on code cache (see: x264 trellis asm). This avoids inlining it and creating a massive mess -- which, by the way, trashes the compiler's register allocator too. When using the compiler to do this, you run into the problem of 2); it can't allocate registers efficiently between code blocks, or come up with a decent calling convention, so it's inefficient.

4) There are some things that no compiler in existence is currently able to do, like computed jumps without tables to a templated set of functions (see: x264 cacheline SAD asm).

DarkShikari··on Trivial Artificial Neural Network in Assembly Language
See the most recent versions of NNEDI for a neural network application (in this case, an upscaler/interpolator) that is heavily SIMD-optimized.

During the implementation, if I recall correctly, the main desire was to convert it entirely to integer math, but the biggest problem with that was making a fast integer approximation for the sigmoid function. The final version, IIRC, was part integer and part float for this reason.

NNEDI originally chose from one of 64 neural network contexts for its calculations, but the current version performs a weighted average between a smaller number (context mixing). Since this eliminates the branchiness, it would likely be very suitable for a GPU implementation. Making a shader-based or OpenCL-based GPU implementation that would work in a video renderer (e.g. for realtime video playback) would likely be extremely appreciated.

DarkShikari··on Call yourself a 'brogrammer'? Then get the hell away from me.
From Urban Dictionary:

brogrammer, n.: A programmer who breaks the usual expectations of quiet nerdiness and opts instead for the usual trappings of a frat-boy: popped collars, bad beer, and calling everybody "bro". Despised by everyone, especially other programmers.

Ex: Oh my god, John is talking about football and chicks again. That guy is such a brogrammer.

See also: programmer, frat boy, bro, douchebag, developer

This is not a good stereotype to conjure up, even in some ironic-parody-hipster sense.

It isn't a joke, either; I've definitely seen programmers where I've worked who really do act like that -- and they are like toxin for introverted programmers. Most other programmers probably don't appreciate a "bro" constantly talking about his sexual conquests instead of interesting technical topics or hobbies.

DarkShikari··on C database connection pool library
and certainly impossible if you don't provide at least object files that can be relinked.

This isn't news; this requirement exists on all platforms, not just iOS.

DarkShikari··on Microsoft, please support (at least a tiny bit of) C99
x264 has worked fine in many many MSVC applications, even compiled in minGW with gcc.

There are some minor catches:

1. If you want a .lib instead of a .dll, you have to make one yourself separately using a tool. Not a big deal, it's just that gcc does do it for you.

2. No debug symbols, because MSVC doesn't support DWARF/etc.

3. gcc guarantees 16-byte stack alignment on x86_32, but only with respect to the caller; MSVC does not, so the stack may not be aligned as gcc expects it. You can use -mpreferred-stack-boundary to make gcc expect it, or you can just explicitly align the stack, either with the gcc intrinsic (requires ~gcc 4.2) or a small assembly function.

There are probably a few others, but overall, it does work; cdecl is cdecl.

DarkShikari··on What HN users don't mean to be
It's an old rule that whenever someone says "I'm not racist, but", they are practically always about to say something incredibly racist. I think this can be generalized to all forms of "I'm not X, but" or, "I don't mean to be X, but", and so forth. "No offense, but, <offensive thing>" seems to also be a common variation.

Is there a word for this kind of linguistic construct?

( Possibly related: http://www.notracistbut.com/ )

DarkShikari··on Teacher effectiveness ratings, programmers, and Khan Academy's data
Teaching is a lot like programming in this respect -- if you try to enforce metrics of "performance", you usually end up with optimization for those metrics and dumber, less-educated students. Similarly, students tend to optimize too, just from a different standpoint. Examples of this:

If you rate teachers based on the grades their students get, they will usually award their students higher grades for the same (or worse) performance. This isn't just a joke; many, many professors have reported their deans bearing down on them for awarding too many bad grades, often to students who are so incompetent that they cannot write basic English sentences or perform long division. The solution -- not remedial courses, or letting them drop out (that would hurt financial aid income) -- but letting them pass even though they lack the basic skills to learn in that class, let alone pass. This also leads to the student mentality that they are being "given" grades (and thus can beg for higher grades) as opposed to earning them through their own effort in the class.

If you rate teachers based on the number of their students pass a standardized test, the teachers will focus solely on the contents of that standardized test and put a huge amount of effort into educating the few students who Don't Get It, at the cost of all the students that do. I had this experience in primary and secondary school, where teachers lamented that they had to go over material for the Standards of Learning tests constantly and repeatedly, dramatically lowering the amount they actually had time to teach. The end result is a quick dropping of everyone to the lowest common denominator -- which often just ends up shifting the entire bell curve to a lower point.

If you rate teachers based on student evaluations, your results are nearly useless. Most students don't bother to write in much detail, since they're just glad to be done with the final exam (I did this too!)-- and often they use it for "revenge" on professors they thought were too hard. The teachers who get the best evaluations are often (though not always) the teachers who gave the easiest work and grades, and whose students turn out to be the most woefully unprepared for what comes next. Just like customers of your software are not always good judges of what they really want, students are often terrible judges of what good education actually is.

As far as I can see, nobody has come up with a magical metric for rating teachers. If anything, most of the metrics out there are outright counterproductive -- and you can see it in schools across America, where teachers are being constantly urged to inflate grades for their 'customers', overlook plagiarism, ignore the most talented students in favor of the least talented ones, and help drive down the quality of education for the next generation.

DarkShikari··on Silicon Valley's Humble Billionaire
His children will not inherit a significant proportion of his wealth. This is consistent with statements he has made in the past indicating his opposition to the transfer of great fortunes from one generation to the next. Buffett once commented, "I want to give my kids just enough so that they would feel that they could do anything, but not so much that they would feel like doing nothing".

Since when is Warren Buffet "screwing over his kids"? He wants to give them enough that they can do ANYTHING. That doesn't sound like "screwing over". If that's "screwing over", I'd love to be screwed over!

And if this billionaire had good reason, perhaps, like Warren Buffet, he would have explained it?

Are you his divorced ex-wife?

Don't you love the smell of ad hominem attacks in the morning?

DarkShikari··on Silicon Valley's Humble Billionaire
Just because other people have been jackasses does not give you an excuse to be a jackass. I've seen plenty of divorces that ended quite happily because both people were civil to each other. You just don't hear about those because they don't usually make much noise.

But even if you believe that it does justify it, however misguided that is, since when does a divorce ever justify screwing over your children in addition to your spouse? At that point, it's just pure greed.

DarkShikari··on Yammer pulls down $85 million in venture funding
The company claims that more than 85% of the Fortune 500 now have Yammer networks, including Thomson Reuters, Shell, Chevron, Nationwide and Ford Motor.

This reminds me of claims that "90% of the Fortune 500 use our software", when in reality "90% of the Fortune 500 have had at least one person download our evaluation version". How much is Yammer actually used?

DarkShikari··on Tengine - A variant of Nginx used on #1 Chinese ecommerce site goes open source
There was no talk; it was an unconference.

I'll try to retell x264's story here...

Japan has long had a community of x264 developers and users, but for a long time they largely remained insular and didn't make much active effort to push their patches upstream. This wasn't out of maliciousness; there were many reasons.

The language barrier was a big problem -- less so that they didn't know English, but moreso that they didn't feel confident with it. In reality, we found the Japanese contributors to have way better English than they thought they did. People are often embarrassed or scared to use a language they're not confident with, and they worry about making mistakes and looking bad. Make them comfortable.

Note that this is especially an issue with Japan. In my experience, Japanese tend to be (typically) much less confident in their English despite equal or greater skill. This might be in part because the school system typically doesn't emphasize much 'real-time' conversational English.

This problem is more general than just language; open source IRC channels and mailing lists can be intimidating, and people (often rightly) suspect that they'll be mocked for making mistakes, so they just don't bother.

As is common in many non-English-speaking countries, the Japanese have many of their own tools and methods of communication that are unique to Japan. In this case, they had an ongoing x264 thread on 2ch. A friend of mine from the x264 user community offered to help. He was multilingual, able to speak near-perfect English and Japanese, along with many other languages. Using his translations, I answered questions for a few weeks on the 2ch thread.

Eventually it became somewhat obvious that one of the people posting in the thread was a Western developer; the Japanese jokingly called it the 'Black Boat incident', a reference to the arrival of Commodore Matthew Perry's fleet at Japan in 1853 (see http://en.wikipedia.org/wiki/Black_Ships). We convinced a few to drop by our IRC channel; we invited them, noting that difficulty with English was not a problem, and we had a Japanese speaker who they could converse with directly anyways.

A few came, some contributed patches, and a few stuck around; we now even have a small community of Japanese users who hang out in the main channels. One of them noted later that his (written) conversational English had improved vastly just by being on IRC for a few months and was now much more fluent.

Put simply, cultural and language barriers reduce peoples' confidence in communicating. They can also result in misunderstandings; open source developers, for example, tend to have a very blunt style of communication. In the best cases, this can mean they will ignore their own ego and debate decisions on technical merits, without pleasantries. In the worst cases, this can mean rudeness, intolerance, and general dickishness. Especially coming from a culture more heavily based around politeness, this can be daunting.

If you want to welcome a foreign community of developers, you should try to (list certainly not complete):

1. Have someone who speaks their language, so they can feel confident that they have someone to speak to even if they aren't confident in their English skills.

2. Contact them first; don't rely on them to come to you.

3. Be friendly and welcoming. If you have to, mute That Guy who insists on being rude and obnoxious. Mocking grammar errors or being needlessly blunt are quick ways to make people feel incredibly unwelcome. They probably speak your language much better than you speak theirs!

4. Give them extra help -- don't treat them like Just Another Patch Contributor. Your goal here is not just to integrate their changes, but to gain a connection to their developer community. "Patch rejected, I don't like it" is not a way to gain friends.

5. I really shouldn't have to say this, but apparently (from experience) I have to: seriously -- don't be a racist asshole. Particularly if you're inviting people to an IRC or high-traffic mailing list, there are often people (including devs!) who will make all sorts of insensitive comments. This needs to not happen. Yes, this means no stupid jokes about "roneriness" or Indian tech support.

Yes, this also means no stereotyping. Just because they're Japanese doesn't mean they want to talk about Naruto, and just because they're Chinese doesn't mean they really like General Tso's Chicken. I know you really love Korean culture, but just because they're Korean doesn't mean they want to talk about Girls' Generation and Starcraft. Don't treat someone from a different country as if they're some specimen under a microscope either. In short, avoid othering. Making people feel as if they are different and not wanted is a quick way to make them not want to come back.

DarkShikari··on Dell: No longer a PC company
Apple have very respectable margins on their their PC's.

That's because they don't participate in the aforementioned low-end PC market. Dell makes respectable margins on their high-end brands (Alienware), too.

DarkShikari··on Linus torvalds on security
The worst part is that this kind of thing actually makes security worse.

If you force people to bypass security to do ordinary tasks, and train them to constantly enter the root password for everything, you don't actually have any security. It's like the password policy that's so impractical that everyone sticky-notes passwords to their monitors, or the Vista UAC.

DarkShikari··on Misfortune teller: predicting criminal recidivism
Correlation might be useful, but how about causation?

The most likely "cause" of criminal recidivism is sending millions of people to prison for minor crimes, where they spend months learning from other criminals, and then are released, upon which they find it impossible to find a job (because they're marked as a felon) and are forced to return to crime.

DarkShikari··on Tengine - A variant of Nginx used on #1 Chinese ecommerce site goes open source
There's often a serious communication barrier between Asian open source developers and Western communities; partially a language barrier, but also other issues. The end result is that a lot of projects end up being de-facto forked by Asian open source developers -- this is where outreach can be very important, to help integrate the changes back in as soon as possible and start that necessary contact.

If anyone cares I can talk a bit more about this (as this was an issue with x264). There was also a session (with notes online) on the topic at last year's GSOC Mentor Summit.

← PreviousPage 2 of 29Next →