Redis for win32 and the Microsoft patch
antirez.com
antirez.com
Antirez, if you could try and imagine how Redis could be used in different environments (inside devices, as a component in 'end-user' business software, and so on), you might see why 'deployment' and 'servers' and 'the cloud' is not at all the only area where Redis might shine. It's your project so your choice, but don't neglect industries and areas simply because those involved in it aren't so represented among your twitter followers!
This is how you go from a great, focused project to a buggy and bloated one-size-fits-all project that gets gradually abandoned and replaced by a new fad.
I run Apache on Win32 all the time. Is that bloated and buggy? As well as php (well, no worse than on linux) and mysql. Postgres runs on windows as well. Then of course we have crossplatform darlings like Firefox, Libreoffice, etc.
SQLite is awesome, really really awesome. It is also not a good example -- part of the reason Redis works so well is that it capitalizes on Unix process management (in a way that was apparently intended when Unix process management was evolved, but has for some reason been unpopular and long forgotten). SQLite was never in that position.
Maybe do something similar in spirit to Slashdot's old policy that you can not moderate and comment in the same thread. Perhaps here that policy would ban you from downvoting posts in a thread if you commented in that thread?
So I think if you stack IBM, Oracle, HP, and everyone else Redis runs on without modification, his point stands.
[1] http://redis.io/topics/introduction
And, really, it hasn't, with the possible exception of web startups
Hello!!! You are posting this comment where??? And have you heard of this little web startup named facebook? They use linux apparently. Zynga? Linux. Oh, yeah, Google, they use Windows, right? Amazon. And the largest company in the US, Apple, they use windows too, yeah?
Oh. You mean those other, giant, windows-using companies we're reading about daily in the Wall Street Journal.
2) I am rather interested to hear what you mean by "deeply entrenched in the real world". Are Google's and Amazon's profits somehow not real?
The way my GP alleged that no big companies use Windows as a platform came off as snarky and represents what I think is a skewed, SV/web-startup centric point of view.
I only have two data points: I've contracted for two Fortune 10 companies. Both companies had Windows on the Desktop. But their Big Data / Big Iron was linux and solaris. Indeed the license managers (FlexLM) that authorized software for those desktops? Solaris.
Needless to say both of these companies were outside silicon valley.
So do you have data on that? Just doing some googling it seems that Walmart started out as an IBM shop, more recently Oracle and HP are mentioned. Are walmart buying Microsoft licenses to run Oracle on the HP boxes? I mean, they could be, but I think it would be unreasonable to assume that is the case.
Apparently, posting on Hacker News now means "you're supposed to disregard the entire world except web firms, or else you don't belong here"? Come on, man.
But inside of many big corporate and government environments, Windows rules the roost. I think there's a good reason for that -- Microsoft enterprise stuff usually works. In big orgs, Linux/Unix is associated with "big iron" , "big dollar" nightmares from Oracle, CA, etc.
It's this shortsightedness in the leaders of open source endeavors, of all kinds, that can be a deterrent to outsiders. Not that it's wrong or bad to do something the way you want because it's your project and needs to fit your needs, however that said if it's to see wider adoption you have to look beyond what it is now. If a project is to be more than a personal project then it needs to take into account needs that are more than your personal needs. If the best course of action is to develop two independent projects, then so be it, but it would be foolish to dismiss the whole Windows ecosystem if one wants to see a project more widely adopted (although maybe that isn't any kind of priority, it depends on your perspective and harkens back to the frustration some people have with one personality leading a project).
In fact, the linked article even underlines this: he perceived a user need for a Windows port, took it very much into account, and carefully wrote a blog post about why he chose not to go down that path.
I may believe that some assumptions in that blog post are too strong, but I very much think that even writing the blog post shows that antirez cares very much about stuff he's not personally helped by.
So calling antirez "short sighted" because of this is, well, IMHO, a bit beside the mark.
Redis does one small thing and does it amazingly well -- an in-memory database with persistence and some very useful operations (such as FIFO lists) that do not need roundtrips.
That does not make sense inside a device unless you have ridiculous amounts of RAM to spare (which has never been the case in my experience).
Similarly, it makes very little sense in an "end-user" business software, unless there are _multiple_ processes needing access to the same in-memory data. And that (to me) means a "server" is going to appear sooner or later.
Really, unless you run it on a "server", then sqlite with properly set cache parameters is almost surely a better choice, and works on Win32 since 2002 or so.
Where it makes sense to deploy Redis, looks to me like antirez is right and Unix has won. And even though Redis likely works on iOS and Android out of the box, it makes no sense to do so, at least not at this point.
> It might make sense for end-user business software if that software ever has to be run offline.
I'm sorry, I'm entirely unimaginative -- but I can't think of a use case for an "end-user" offline Redis server (regardless of O/S) that is better served by redis' sweet spot than by a local sqlite or postgres install. Can you help me here?
Not all devices are phone-like end-user devices.
I can imagine a machine that produces something. It is full of sensors and actuators, plus logic that uses sensor input to compute what the actuators most be doing. Typically one would store some recent sensor + actuator decision data in RAM to calculate averages, do all kinds of computations, etc.
Storing the same data in Redis instead gives you all that, but persistence for free. That's great, because it means developers / maintenance people / support people can always get a decent dump of what recently happened, without much overhead or extra work. The memory is the log. Elegant!
Computer systems that control factory processes, such as SCADA and MES systems, often have to deal with large amounts of data coming in from many different machines. Most vendors therefore wrote what they call a "real-time database", even though it is not really real-time, just fast. Redis would be a great substitute for such a database.
A high-performance multifunctional office printer has a large memory of jobs sent to it by many different people; print jobs, scan jobs, and so on. These often incorporate fair amounts of data, but the data processing speed requirements are very high because the printer is so ridiculously fast. I imagine that right now the entire job data structure is effectively held in memory and hand-persisted just in case the printer fails; people wouldn't want to lose their print jobs, would they? Persisting data that must be held in RAM for performance reasons. Hmm, where have I heard that before?
I'm not saying Redis is the ideal choice for these particular cases. I'm just quickly braindumping some examples.
I must admit I can't come up with a great end-user business SW example right now though. Maybe I was off the mark there.
That said, as more business applications are moving to the web too, Redis would be very much applicable there. Many organisations (think offices, services sector) have their entire custom-built internal IT systems based on Windows. Much of this is probably a relic of the vendor lock-in practices that MS was very effective with a decade ago, but they're still there and as MS keeps supplying excellent dev tooling, it's unlikely to go away any time soon. Imagine a team of 10 sysadmins for a 3000 person company all highly skilled in administering Windows systems. Moving such people to Linux "just" to administer a Redis installation is impractical, and the company will rather disregard Redis for other options.
Exactly. You're inventing weird use cases that only half-fit Redis's properties, instead of carefully considering the design and architecture of your application.
Software that attempts to solve all problems becomes Emacs.
a decent response to my post would be to explain why Redis fits web/cloud only and why its design is a misfit for other areas by nature.
Redis may or may not be an appropriate tool for any particular design. I've solved a lot of problems with it, but my employer still has Oracle and MySQL databases that are unlikely to ever go away. I've also solved a lot of problems with Python on Ubuntu, but some of our infrastructure uses Windows and C#.
You've got a hammer and you're searching for nail-like objects. You need to put down the hammer, and figure out whether you even have nails in your project.
When I design a new piece of software, you can be sure that I'll find my nails before I choose my hammer. When discussing whether hammer X fits nail Y, please pardon me that I try and consider that in detail before I dismiss the hammer at all.
This entire thread is about whether it makes sense to support Redis on Windows. This discussion, in it's very essense, is a "we've got a hammer (Redis) that could be changed so it can be used on some more nails (db/persistence problems on Windows)". I list some Windows-heavy situations in which Redis may be an excellent hammer indeed. You respond by lecturing me. What's up with that?
Once more, to avoid the standard anti-MS downvote: I'm not demanding a Redis port. I'm not claiming Windows is superior. I'm not claiming to some "right" to get good software for free on my chosen platform. I'm only pointing out that more than the 0.01% of Redis users that antirez quotes in his blog post could prefer Windows, for good reasons.
I really feel like I've been getting a fair amount of fire here for writing things that I feel aren't that offensive or stupid. I can see that you may disagree, but I can't see where the heat comes from.
I should have asked "inside devices running windows , or as a component in 'end-user' business software running windows " I thought that was implied given the context.
Specifically about printers - they're not that fast that sqlite or postgres can't deal with them. Really. redis is for the 50MB/sec pushing-things-around speed. Fastest printers are at the 10MB/sec, which MSSQL, PosgreSQL and SQLite handle perfectly well. Also, redis is about lots (hundreds of millions) of small items. printers are about few (thousands) of large items.
> Computer systems that control factory processes, such as SCADA and MES systems, often have to deal with large amounts of data coming in from many different machines. Most vendors therefore wrote what they call a "real-time database", even though it is not really real-time, just fast. Redis would be a great substitute for such a database.
They should pay for a "real" in memory databsae if they can pay for windows. TimesTen, Sybase IQ, Kx. Yes virginia, there are consequences to using the proprietary eco-system.
> Imagine a team of 10 sysadmins for a 3000 person company all highly skilled in administering Windows systems. Moving such people to Linux "just" to administer a Redis installation is impractical, and the company will rather disregard Redis for other options.
You're spending $1M/year in sysadmin salary alone here - probably $1M/year more on hardware and other licenses. Shell out $50K for TimesTen and be done with it. Or pay someone (possibly even antirez) who'll produce a windows port. If it's not worth $50K to this company, it's not worth even a single minute of antirez' time.
also, you seem to very much want me to demand that antirez codes a windows port, for free, pronto. i never asked for anything like that. i was, and still am, only referring to antirez's conclusion that POSIX is the 99.99% platform for all places where Redis could the best fitting option. I was hoping for him to slightly adjust that conclusion. What he does with it afterwards is entirely up to him.
I apologize for implying that if I did. That was not my intention.
(offtopic) Really, it's this stuff that keeps me on HN. The ability to shake hands and move on is quite seldom on internet forums.
So when he says "even if you want to run your code under win32 systems what's wrong about installing Linux boxes to run Redis", I can't help but feel he's being a bit narrow-minded. If it were a valid argument, then why port any software to work under win32 at all? Why bother with node.exe? Why bother with Windows implementations of MySQL, MongoDB or PostgreSQL?
In fact, lots of software starts out as Windows software and is ported to Linux. Why not just tell Linux users to get a Windows box if they want to run that software? I would suggest that the author revisit this viewpoint and consider why the majority of products with which he competes have chosen the platform-agnosticism route.
So, you can already run Redis on Windows, if you are so inclined. I too don't like using VMs to develop, and that's why I use Linux for my work environment, but, if you want to have a production-grade OS and be able to run Windows software, VMs are the only way.
But you said, there are other instances of win32 ports of system software like MySQL, MongoDB and so forth. I guess that they are interested in providing more value to their product allowing a larger diffusion, I'm not enough into marketing to value such a thing, in my opinion people running Apache or MongoDB under win32 for deployment are doing it wrong, and I don't want to help.
But again, I just don't want to personally help an effort that is useless in my opinion, but I'm not either against such an effort. Redis is open source, enough people interested in making it working under win32? Perfect, as I said I'm open to help and recognize their efforts in the main project site, I just don't want to be involved. Life is too short to spend it developing something you don't trust.
It does of course make sense as I assume you're developing this because you want to, and not as part of a company's bottom line, which means really you can do whatever you like. No sense developing something if you're not enjoying the process. If however you ever take this to a level where you have a financial incentive in furthering the software's adoption, I'd urge you to do whatever it takes to get an official Windows port in there, which would mean looking at any aspects of the implementation that cause issues on that platform, and doing what you can do to overcome them.
However, even if this was a commercial project, this would still be the correct decision. Innovating new features, polishing and making Redis even better is exactly what @antirez should be doing.
Make an awesome product. Not a half-assed one that runs on every device / os / mobile under the sun. I've seen more than one project dissolve in mediocrity under the pressure of trying to do that.
Curiously, I noticed unix sockets had been officially added in version 2.2.0.
I agree that installing Redis on Windows is kind of "doing it wrong", but the ability to easily access a Redis server with redis-cli even on Windows seems to be pretty important to the continued increase in user base.
Thoughts?
EDIT: I understand there is a redis-cli for Window at https://github.com/dmajkic/redis/downloads , but a separate download that users must know about and keep up to date separately is a barrier to entry, IMHO.
In the first versions of Kiln, we had to ship a Python clone of Redis we called MiniRedis (it only implemented the subset of commands we used) because of the lack of Windows support. We're now shipping the Cygwin build of Redis to run on Windows, which is less than ideal.
I sincerely hope you'll reconsider your position. I believe it is, at best, myopic to think that %nix is the only place people will want to host Redis, or that it has in any way "won" as the only hosting platform.
That's how it should be - you've made a choice of development platform, you get to live with the trade-offs, one of them, for the time being, being "no good native redis".
When you decided to write an application in .NET, meant to be run on Windows, why did you also decide to use an in-memory datastore that doesn't run on Windows? That seems like a really bad design decision for that product.
For convenience you could also run any Linux-specific software in VirtualBox or VMWare on the same machine. You could also install an X-Server on Windows and open Linux apps in their own window on the Windows desktop. I've done it, works great. And you can always find somebody to help out, you only need to ask. Haven't you heard? Virtualization is the future ;)
It is an author's right to devote his time to whatever the fuck he feels like doing. If he feels like there are more important things to work on, rather than Win32 support, then so be it.
If you can't stand this situation you can always port it yourself, send a patch, etc, etc... You can also be hard-core, like the Mono people that reimplemented .NET from scratch because Microsoft doesn't think of Linux as a useful target.
Isn't the whole point of this article that Microsoft sent a patch, and it is being rejected, just to avoid supporting win32?
That's why they are rejecting the patch. Because it doesn't provide anything new while adding complexity. Not because they want to avoid supporting Win32.
So you are seeing that anyone who wants to see win32 support should stop moaning send a patch.
But then, any such patch would add extra code and an abstraction layer, so therefore must be rejected.
So, sending a patch isn't actually useful at all then.
The patch will not make Redis behave well on Win32, Redis will still remain unusable for production environments and Redis already works well enough on Windows for development purposes.
Really, dismissing this patch is a no-brainer - and even though the author said that he prefers to keep it POSIX-only anyway, I'm sure they'll reconsider if they receive a proper patch that fixes the problem, but for the moment there is no such patch.
If there would be a patch that fixed the problem, this whole discussion would be a non-issue, even if the patch wouldn't get integrated in the main branch - somebody (if there indeed is interest) will create another branch specific for Win32 and everybody will be happy. But we can't even have a discussion of branching or not, because there is no such patch available.
So I don't see what's with all this arguing.
BINGO.
Hence, he isn't.
But when he follows your own logic, you call him "narrow minded".
On the contrary, porting front end software between platforms makes a lot of sense, but that's a completely different story.
If I have a desktop app which talks to Redis (for whatever reason), what this means is that I can't realistically produce a version which works offline. Depending on the market, that could be huge.
Instead of running "redis.exe" (which you installed), you run "virtualbox redis.vdx". You could even have the inner redis use the outer disk with a little more work, running the internal vm read only. Configuration is slightly more complicated, as you have to both configure the inside redis and the outside virtualbox -- but it's all just a few short text files, and you only need to do that once.
You can produce a version that works offline. You can even bundle it inside your product (virtualbox and redis both being free). And you can do it today without antirez's help. Really!
EDIT: online -> offline; typo.
This being a BSD product, they can just maintain and release their own version, and they're not even obliged to share their modifications with the community; they haven't in the past (e.g. BSD sockets, zlib) so I wouldn't expect them in this case either.
And as a paying microsoft custoemr, you should be getting such a service for your money.
Really, this is all it is about -- You, yread and microsoft are trying to get Antirez to provide valuable (to yourselves) service for free.
That is a great philosophical question, and different people will have different answers, all valid; however:
> "Installing Linux boxes to run Redis", I can't help but feel he's being a bit narrow-minded.
..
> Why not just tell Linux users to get a Windows box if they want to run that software?
This is not a symmetric win32<->linux case. If you're a win user, and you need Redis, you can run it today, for free, starting 10 minutes from now (30 if you've never done this before and need to download stuff):
download oracle virtualbox; download ubuntu server install, install inside virtualbox inside ubuntu, sudo apt-get install redis-server
And really, that's all there is to it; if you are an experienced windows developer with no Linux experience whatsoever, this will take you all of 30 minutes (25 of which just waiting for downloads). It actually performs quite well; not as well as running directly on iron, but probably not more than 10% slower than a native port that does everything right (The current native ports are 100000% slower during syncing to disk)
If you're a linux user and you want to run a windows-only server, that costs your $100 or so for a retail win7 license, a few hours of downloading and 20-30 minutes of active setup last time I did that. Want to fire that up on 3 developer machines? That would be $200 more, thank you. Reconfigured your virtual box more than 5 times? You have to talk to Microsoft before you can proceed.
So, really, what he offers IS a viable solution for windows developers. Really. Try it before you consider it "narrow minded". You might find the narrow mindedness was not where you expected.
> In fact, lots of software starts out as Windows software and is ported to Linux.
The few I am aware of are picasa and 7zip, of which I only ever use the latter. Do you have more examples?
It's actually significantly harder to port Windows programs to Linux than the other way around, because the Win32 API has about 100 times more functions, and the standard practice (cheerfully supported and encouraged by Microsoft) is to use all of them.
I believe that thought is narrow-minded because you are not thinking in terms of enterprise. This would probably work in a startup, or maybe even in a mid-size business. But in an enterprise, going to your manager and asking to fire up a linux VM in a Microsoft shop will probably get you reprimanded (or laughed at). I would love to use linux all day, but we can't do it as there is no freedom to. With a Windows port, I could create a business case for it and get a VM provisioned with Windows Server on it within a short period of time. Honestly, as bad as this sounds, money for small things like Windows licenses is not an issue when it comes to enterprise.
And it's been very long since I've seen an "enterprise" that didn't already have linux deployments. Can you name a few?
Whoa whoa whoa. It's narrow minded to not dance around the narrow mindedness of others?
There is certainly a problem that needs solving here, but it's not Redis on windows.
Roughly 60% of our customers run Windows servers in production (2003 or 2008) which is why we support Windows as a first citizen platform. These customers have no desire to run Linux boxes for various reasons and we would simply lose deals if we didn't support their environment.
We've managed to deliver the same level of performance for all platforms we support, and although our NoSQL product is very different from Redis I think you could fully support Windows if you wanted to.
Additionally, I know from experience that supporting multi-platform greatly increases the code quality and robustness. Obscure race conditions or memory problems may be revealed by such support.
Nevertheless, adding support for a platform a posteriori is a huge task and you have the right to consider it's not worth it.
In other words, we change the code so that it fixes the platform specific bug while adding no platform-specific code.
(Don't know if I made myself clear).
MS coming out the message "but...but...but...you should support everybody!" is a bit of a pot/kettle situation and I hope they are treated with the same passive-aggressive apathy as they treated Linux and Mac users for all those years.
I know I'm being petty but I can't seem to make myself stop smiling.
Are still doing.
> What comes around goes around and I really don't want to be on the receiving end again.
You're still on the receiving end. And "what comes around goes around" is simply what's happening to Microsoft in this case.
I'd reserve judgment about this specific case until we see how things actually turn out.
I've been dealing with Microsoft for more than twenty years, and I'm not holding my breath for their decent behavior, although it happen occasionally.
I really have no idea where you are coming from with this statement. My observation of the world is that "give the other cheek" gets you slapped twice and doesn't eventually put you in a better place (unless it is after you get to heaven, which I haven't yet experienced)
This is the exact same situation Windows users are in with Redis. Except that, instead of just "coming out the message 'but...but...but...you should support everybody!'", Microsoft actually did something very helpful. They didn't just complain, they took it upon themselves to help improve the situation with Redis by providing the patch.
Let's also be clear that when comparing Microsoft and antirez (or open source software in general), we're comparing two very different things. We all know that Microsoft is a for-profit company, so expecting them to go out of their way to improve their competitors isn't really reasonable. Open source software, however, is about making software freely accessible to everyone, so an expectation, or hope, that antirez would embrace Windows support is quite reasonable.
Of course, that's not to say he has to. It is his project, and there for his right to choose not to support it. But I personally think it's a mistake. He is encouraging a fork, and forks generally lead to problems. Client libraries will have multiple versions to target. Developers will have more to learn, and more time will be spent trying to keep feature parity between the forks.
- Half-assed port for windows appears, with a big "NOT SUITABLE FOR PRODUCTION";
- Clueless programmers will simple google "redis windows", hit the first compiled version "redis.exe" and use it, eventually shipping it to production;
- They'll then hit the official support forums and SO with "Redis doesn't work!!"
How about a port of IE to Linux so I can test websites with it? How about the ability to install multiple versions of IE on Windows? I could go on....
http://blogs.msdn.com/b/ie/archive/2011/02/04/testing-multip...
As you might expect it's using virtualization, but nothing that isn't included in Windows 7 Professional and up.
Microsoft does have a third option, you know. One many of us would love to see happen: real, first-class POSIX support in Windows.
Here's the script that will install them for you (works on my Ubuntu): https://github.com/xdissent/ievms
Someone has reported this a couple days ago on their forum http://answers.microsoft.com/en-us/ie/forum/ie8-windows_xp/w... but there's been no response.
Other questions on that forum indicate that there was exactly the same issue in 2010 and in 2009, so it looks like in practice, you can test with old versions of IE for free, but only in the summer months. It's certainly better than nothing, but it's not something I would depend on.
It would be cool to have a VMWare Virtual Appliance Ready to go. Shouldn't be too hard since Redis is sponsored by VMware ;)
This is simply missing the point of cross-platform software...
@antirez wants to take advantage of the underlying POSIX platform and as such Redis takes advantage of unique features only available on *nix. His point is merging the patch increases the size and complexity of the project (whilst reducing stability) - detracts from his mission of making Redis run exceptionally well on POSIX platforms.
He also highlights that Windows/.NET shops like StackOverflow have no problem setting aside linux servers to run host Redis servers in production.
Redis's performance (compared to traditional databases) comes from the fact that the data structures are maintained in memory, which turns out to be a really good tradeoff when your data set is small. That has nothing to do with POSIX, or unix. The point of contention is the fact that Windows does not have an efficient implementation of fork, which enables copy-on-write semantics. Redis uses that to periodically dump data to disk without blocking the original process. This is a really clever use of a feature unique to unix, but one man's cleverness is another man's limitation.
Building cross platform software really forces you to examine your architectural decisions, learn about other platforms (and by extension, the limitations of your own), and your software always ends up having a better architecture when you're done with the porting process (in addition to being usable by more people).
@antirez's arguments regarding complexity, additional testing, etc. make total sense, but your performance/stability argument doesn't hold water.
Also it's not that hard to #ifdef 42 graphic backends to a dead code base. It obviously becomes a nightmare as soon as you want to extend and evolve the core code.
What a wonderful argument in support of @antirez's suggestion that other people port and maintain it. In every case each port was done after the fact, into its own project. They may since have been reintegrated back as part of the source release (5 years later). But the core team certainly didn't do it. Carmack ported Quake to GL in '96 and IIRC that was used for a linux port. Again, not on main branch, not being committed while the team was working Quake 2.
I've made cross platform video games and I don't find your opinion remotely credible. I've also ported video games from one platform to another. A game that was highly optimized for one platform required a complete rewrite of the graphics engine for the other. Companies like Naughty Dog and Insomniac, that can focus on a single platform, have a much easier time of it.
Building cross platform software really forces
you to examine your architectural decisions
Quake has different implementations for different platforms.
If people are signing up to use it because it's fast, what's the point of deploying it somewhere that slows it way down?
The interesting part in antirez argument is that he seems to be a proponent of mixed-platform deployments, so that interoperability can be achieved on a different level.
I spent the first year of my Rails development on Windows and it was so painful - slower, strange bugs, many unsupported tools/libraries.
This seems like a reasonable decision, but one in a long list of reasons you can't easily develop on Windows unless you're targeting a Microsoft stack for your product.
It's an ideal setup, really- when I'm done with development and watch to kick back and watch a movie, I just put the VM into sleep mode and kill off those background tasks.
Let's say BigCorp really wants to use Redis, but only supports Windows.
Why is it OK for them to ask antirez to support Windows, but not OK for antirez to ask them to support Linux/POSIX?
Don't go with a kneejerk response, think about it.
Neither party in this relationship has any special obligation to the other, there's no contract, no money is changing hands.
BigCorp wants antirez to support their preferred software (Windows), antirez wants BigCorp to support his (Linux and/or POSIX).
It will cost BigCorp more to support POSIX, you might say. But cost them more relative to what? Of their tens of thousands of employees, billions in revenue, BigCorp would be dedicating an infinitesimal fraction of its resources to supporting a POSIX-compatible OS.
Meanwhile, a huge fraction of antirez's time will be spent dealing with the effects of trying to maintain a high-performance, low-level datastore on two utterly incompatible platforms, and the time he spends on that is time taken away from work on other bugs and features, which also amounts to time taken away from all the people who already use Redis on POSIX.
There is no incentive at all for antirez, his employer VMware, or the Redis community at large to go through the pain of integrating Windows support into Redis mainline. There is some incentive for BigCorp to support Linux, since they want to use Redis effectively.
The incentivization is exactly backwards, so it should surprise absolutely no one that antirez isn't prepared to deal with supporting Windows.
Why is it OK for them to ask antirez to support Windows, but not OK for antirez to ask them to support Linux/POSIX?
Because antirez has a tool whereas BigCorp presumably has a problem. Because all of us could benefit from this, while only some of us might benefit from BigCorp switching to Linux.
Asking somebody to make their tool more available is, I posit, much more reasonable than asking somebody to change their problems.
BigCorp has potentially hundreds of thousands tied up in existing equipment, IT infrastructure, and training for employees. You and I might not flinch at spinning up a VM to run a database, but we (and the rest of the HN community!) should not fool ourselves into thinking that this is in any way always appropriate for a company.
Meanwhile, a huge fraction of antirez's time will be spent dealing with the effects of trying to maintain a high-performance, low-level datastore on two utterly incompatible platforms, and the time he spends on that is time taken away from work on other bugs and features, which also amounts to time taken away from all the people who already use Redis on POSIX.
Or, you know, he might talk with the Microsoft development team and try to get them to take over maintenance of the Microsoft codebase, and clean up the platform-specific architecture that gets in the way--and in so doing, write better code.
If you can't/won't write platform-independent code, you are either lazy or your problem domain is heavily dependent on your platform. As far as I know, in-memory key/value stores aren't a Linux-specific problem, so...
<rant> There is no incentive at all for antirez, his employer VMware, or the Redis community at large to go through the pain of integrating Windows support into Redis mainline.
Let's look past incentives here. Let's talk about Doing the Right Thing.
A huge number of people here make daily use of software packages and stacks that are open source. We owe our livelihoods to the folks that have been kind enough to release the source to their programs and, more importantly, work with the community to improve their code.
We owe it to ourselves and our community to call out people being lazy and selfish, and letting politics overrule sound engineering and decency.
This is an unreasonable decision by antirez. He's got a cool framework. He's got a bunch of code and (presumably) people that are happy to work with him on making it even more useful.
He's also not going for it, because "use linux lol". Not because it's hard, not because he doesn't have folks that can add it, not because the problem extends beyond some refactoring and #ifdefs and makefiles.
Because he's lazy. Because Linux is the One True Way. Because helping out anyone who can't just change their infrastructure on a whim is too much to ask.
Really folks? You don't see a problem with this? </rant>
If I have misclassified the situation, by all means, enlighten me.
There is a dollar amount that would get Salvatore to do a first-class WinAPI Redis implementation (he won't advertise it, and may not even believe it exists, but I assure you that there is one). Pitch numbers to him until you find it, then, pay him. You may or may not choose to share your newly purchases WinAPI Redis with the rest of the world --- bear in mind that there are enterprise products with mid-5-figure per unit price tags that do nothing but provide WinAPI-friendly .NET-compatible memcached.
That's roughly your argument here; I've just done you the favor of adding concision to it. The motivations of all the parties involved, enterprise IT costs, all that jazz, none of that matters. Fundamentally you're saying "if something can be made cross-platform, it's wrong not to make it cross- platform".
That is, to say the least, a comical argument coming from people who want Linux code deployed on Windows.
Not quite.
Guy writes code for Linux. Code could theoretically be made to work on Windows. Microsoft team submits patch to achieve this functionality. Users can reasonably expect a Windows port. Guy decides using submitted code is too much effort (from article says will get harder and harder) and that besides, there is no demand for it (from article: "I don't think Redis running under win32 is a very important feature."). Users have been denied functioning code because of arbitrary decision by guy.
(1) Guy writes code for Linux. (2) Code could theoretically be made to work on Windows. (3) Microsoft team submits patch to achieve this functionality. (4) Users can reasonably expect a Windows port. (5) Guy decides using submitted code is too much effort (from article says will get harder and harder) and that besides, there is no demand for it (from article: "I don't think Redis running under win32 is a very important feature."). (6) Users have been denied functioning code because of arbitrary decision by guy.
(4) simply does not follow from (1), (2), and (3), derailing the whole rest of your argument.
Salvatore doesn't owe anyone a damn thing. He has the opposite of set the expectation that there will ever be a Windows port, so any user who thinks they can "reasonably expect one" is delusional.
Anyone who says otherwise is arguing for the sake of arguing. Writing a program specifically for Unix is obviously a totally reasonable thing to do, just as it makes zero sense to demand Panic provide a Windows port for Transmit, or that Microsoft provide IIS7 for FreeBSD.
The code is brought into existence. Code is observed to be capable of being modified into working on Windows (i.e., nothing radically platform-centric about it). Microsoft provides the grunt work to accomplish a patch that proves this observation true.
Very much, by definition, users can reasonably expect a Windows port. The capability exists. The code exists. The patch functions. If Windows User Clippy says "I expect that redis will be ported to Windows, given these facts", Clippy is not being unreasonable.
I'm sorry, but to claim otherwise is merely being obtuse.
Your points about IIS7 and Transmit are correct, as those rely on specific OS functionality that is unique--the constructs Redis uses are not, and again code has been written to show this fact.
Now, there very much is something to be said for Salvatore (antirez?) not having to do this work for free. Indeed, VMWare may be spending a good chunk of change paying for his services ( http://antirez.com/post/vmware-the-new-redis-home.html ). I do not wish to suggest that he go uncompensated.
What I do wish to suggest is this:
VMWare: this dude you are paying to develop a widely-used and respected project has (seemingly arbitrarily) turned down code submitted from other practitioners in the field. He has suggested they fork the project (diluting the quality of the Redis brand and potentially removing resources that could improve mainline Redis). He has suggested that the project is too difficult to make cross-platform; I'm sure that you folks might have a different opinion on how important cross-platform support is.
Pay him more to make a better project (with help from people in the community!). Or acknowledge that he is correct in his assessment that this is somehow a Linux-only problem. Or do something else to help him figure out how his goals align with the VMWare mission.
* If someone writes a patch for your project, the burden of proof is on the maintainer not to accept it, rather than on the submitter.
* If someone domiciles their open source project at VMware, they assume an obligation to all of VMware's customers.
Neither statement has any truth to it. You're just throwing stuff at the wall to see what will stick, which is why your comments are turning light grey and mine aren't.
Can I ask you an orthogonal question?
What's the last project you used Redis on? How'd it turn out for you? What do you like about Redis?
Then you should go back to logic school. Let me make it clearer for others reading this, as yourself appears to be a lost cause:
(1) Microsoft develops web browser (2) rendering could theoretically be made standard compliant with minor changes (3) people submit style sheets and javascript files that, if included in a web page (or embedded into IE) would achieve that functionality. (4) Users can reasonably expect Microsoft to do that.
Wait, what?
Furthermore, your logic breaks in other places:
(5) ... there is no demand for it (from article: "I don't think Redis running under win32 is a very important feature.").
How does "not important" imply "no demand"? Let me give you an example that will make it clear:
"Apple just said that getting Win7 apps running on the iPad is not important". You bet there's demand for that, and yet that does not make Salvatore's or this (fictional, but realistic) Apple quote wrong in any way. "important", unless otherwise qualified, is to the speaker.
(6) Users have been denied functioning code because of arbitrary decision by guy.
Wrong. They could just build it from source. Less convenient? Well, it takes just one person to package it and make it into an .msi.
Furthermore, the decision is not arbitrary -- you may disagree with the reasoning, but he gave detailed reasoning.
You sir, are either an idiot or a masterful troll -- I would pick the first, if only because it makes me feel better to think I haven't been trolled :)
When the other person consistently ignores logic and invents their own facts and reality, I occasionally adopt the Linus strategy; but I still take care to address the argument itself.
Point well taken. HN is a better place thanks to people like yourself.
This is a direct patch applied to an open codebase, which in turn allows users to reasonably expect that, there being no technical or resource obstacles to implementation, a platform port could be expected.
Your critique of my 5th statement is not unfair--I should've elaborate on the connection between "not important" and "no demand". I was in error. Your analogy using the iPad simply muddies the waters.
Your critique of my 6th statment is also unfair. The decision was arbitrary (we know that technical constraints don't exist, and that it isn't "free work" being asked for unreasonably). Asking people to just build it from source is not good for users, and forcing the creation of a fork of a codebase is not good for developers. You are being unreasonable, I believe.
The reasoning given for the decision is heavily based in arbitrary judgements not backed by valid engineering concerns (a lot of that low-level OS stuff? Yeah, actually pretty standard across operating systems, though the incantations may be different).
You sir, are either an idiot or a masterful troll -- I would pick the first, if only because it makes me feel better to think I haven't been trolled :)
I wasn't attempting to troll here, and I prefer not to think of myself as an idiot--then again, I also prefer to debate with people who can structure arguments without excessive namecalling or misleading strawmen.
EXPLICITLY BECAUSE microsoft refused to patch internally, which is what you think antirez is doing wrongly!
> Your example with the Microsoft web browser is disingenuous, as it involves a closed, proprietary system, and
So? Where is the contract, written or social, that antirez signed, which predisposes him to accept patches that go against his philosophy of keeping the code base lean, simple and efficient?
> This is a direct patch applied to an open codebase, which in turn allows users to reasonably expect that, there being no technical or resource obstacles to implementation, a platform port could be expected.
THERE ARE TECHNICAL OBSTACLES, CLEARLY DETAILED BY ANTIREZ. You might disagree with them, but they are not empty words. And furthermore: WHY? WHY SHOULD IT BE EXPECTED?
> The decision was arbitrary (we know that technical constraints don't exist,
PLEASE READ ANTIREZ AGAIN (AND AGAIN (AND AGAIN)) THERE ARE LEGITIMATE CONCERNS WHICH YOU FAIL TO ACKNOWLEDGE
> The reasoning given for the decision is heavily based in arbitrary judgements not backed by valid engineering concerns (a lot of that low-level OS stuff? Yeah, actually pretty standard across operating systems, though the incantations may be different).
So please explain to everyone how easy it is to do copy-on-write from memory to disk without pausing and without fork() -- because THE GUYS AT MICROSOFT WHO SUBMITTED THE PATCH DON'T KNOW HOW TO DO IT. THEY THINK THEY MIGHT HAVE A SOLUTION BUT HAVEN'T POSTED ONE YET.
> You are being unreasonable, I believe.
The only unreasonable thing I am still doing is replying to you. We apparently live on different planets. ANTIREZ HAS NO COMMITMENT TO THE GOOD OF USERS OR DEVELOPERS. Furhtermore, ANTIREZ BELIEVES AT THIS POINT SUPPORTING WINDOWS WITHOUT FORKING IS AGAINST THE GOOD OF USERS, DEVELOPERS AND HIMSELF, as he has clearly stated.
> I also prefer to debate with people who can structure arguments
Funny ! I'll come back to this post when I need another laugh!
If you can't write your OWN in-memory key/value store, you are either lazy or a parasite.
> Let's talk about Doing the Right Thing.
The right thing for people like you is to, you know, just maintain and release their own Redis port. It is BSD licensed - you don't even have to release the source code if you don't want to. And in fact, a port already exists. So you are either lazy, a parasite, or a troll.
> Because helping out anyone who can't just change their infrastructure on a whim is too much to ask.
Money talks, bullshit walk.
> Really folks? You don't see a problem with this?
Yes, I do see a problem with people like you who feel entitled.
I haven't seen any complaints about Microsoft dropping POSIX support (which, if it was actually usable, would have supported redis out of the box).
If you can't write your own green threading library, you are either lazy or a parasite. Please let's be reasonable about what is worth rewriting and what is not.
The right thing for people like you is to, you know, just maintain and release their own Redis port. It is BSD licensed - you don't even have to release the source code if you don't want to. And in fact, a port already exists. So you are either lazy, a parasite, or a troll.
This attitude is why so many man-hours are lost in the community, and why so many pointless forks exist. Come now, do you honestly believe this is a more efficient/sound/useful solution than trying to get antirez to just add support in mainline Redis (without even having to start from scratch)?
Money talks, bullshit walk.
Care to elaborate? I'm not fluent in "tired cliche".
I haven't seen any complaints about Microsoft dropping POSIX support (which, if it was actually usable, would have supported redis out of the box).
I imagine that there were, in certain circles, a lot of annoyance at this. My team and I do a lot of cross-platform C code, and this would've made our lives a lot easier. Then again, if people complained (as I did here!) perhaps things may have gone better?
I was mocking your statement, in case you missed it. I don't think that if you don't write cross platform code you are lazy -- especially not high performance code that derives some of its performance from platform specific features!
And no one needs to rewrite anything! Just use your own fork with your PATCHES. Read my lips again: NO REWRITE NEEDED!
> This attitude is why so many man-hours are lost in the community, and why so many pointless forks exist.
For some reason, you believe the community owes you something. And what exactly are those "so many pointless forks that exist"? GCC has had 5 significant forks through its life, non of them pointless.
> Come now, do you honestly believe this is a more efficient/sound/useful solution than trying to get antirez to just add support in mainline Redis (without even having to start from scratch)?
Of course it is efficient for parasites like you when antirez spends the time -- especially since you don't care about development efficiency and speed on Linux which are essentially guaranteed to be harmed, at least in the short term and possibly for a long time to come.
The only question is: why should antirez care?
> Care to elaborate? I'm not fluent in "tired cliche".
SPEND TIME (which translates to SPEND YOUR INCOME which translates to SPEND MONEY) MAINTAINING A WINDOWS FORK, TO ANTIREZ STANDARD OF PERFORMANCE ON LINUX AND CLEANLINESS OF IMPLEMENTATION, and if you manage to achieve that, I can almost guarantee he'll merge it in. The problem is, it might not actually be doable -- but TALKING ABOUT IT IS CHEAP FOR YOU, so that's what you do -- which is what others tend to call BULLSHIT. Is this elaboration enough?
> My team and I do a lot of cross-platform C code, and this would've made our lives a lot easier.
Translation: We would have gotten something for nothing.
Question: Does your team release any of its code? If so, can I have a look? If not, why not -- and why do you feel entitled to someone else's source code?
> Then again, if people complained (as I did here!) perhaps things may have gone better?
Oh, you are so naive. The only reason the posix subsystem existed in the first place was so that NT would be eligible to compete in Government tenders (predominantly, but not only, US). It was always a 3rd class citizen, hardly supported or usable, although it was very successful in the business sense of letting MS compete in those tenders.
Microsoft only fixed browser standard compliance (and not fully) when they realized they're starting to lose the browser war again. That's 10 years of very vocal complaints that went unheard, and for a very simple reason: It conflicts with their lock-in strategy. Same thing about the posix extensions.
You might be a very good developer for all I know, but you really fail in understanding how the world works.
Actually, we would've been able to focus on mainline development instead of having to write some supporting libs. That said, having written that code, we now are going to share it with the rest of the world, so they don't have to waste time doing what we did. If you'd like to leave an email address here, I'll cheerfully send you notice of our release personally.
Question: Does your team release any of its code? If so, can I have a look? If not, why not -- and why do you feel entitled to someone else's source code?
We're releasing a cross-platform low-level OS, threading, and math library for C under either the WTFPL or BSD license around New Years. This library will have a public bug tracker, open source, and excellent documentation for usage and code commenting as well overall package design so that others may extend it easily if they wish.
~
Sir or madam, call me whatever you wish, but at least be accurate. I am no parasite.
My email shall remain hidden for now.
> We're releasing a cross-platform low-level OS, threading, and math library for C under either the WTFPL or BSD license around New Years.
Great. I will keenly be following your responses to contributed code to see how you believe it should be done.
> He's also not going for it, because "use linux lol". Not
> because it's hard, not because he doesn't have folks that
> can add it, not because the problem extends beyond some
> refactoring and #ifdefs and makefiles.
> Because he's lazy. Because Linux is the One True Way.
> Because helping out anyone who can't just change their
> infrastructure on a whim is too much to ask.
What a load of entitled bullshit.
If I choose to run my infrastructure on Linux I can't complain that I don't get Microsoft's Active Directory. That's one of the consequences of the decision I made. When you choose Windows as your platform, these are the consequences.
You don't get to later yell at the developer of an open-source project you like and call him names because he won't take the time and effort to make his project run under an OS that's significantly different from the one he writes for. Especially after he's said that even though he won't support Windows, he will bless a Redis-win32 project run by someone else.
Yes, let's talk about "Doing the Right Thing." Take the patch Microsoft's team provided and start maintaining it. Write tests and documentation, fix bugs, make it work. Antirez says he won't support Windows himself? There's the code, make it happen. That's why open-source software is awesome.
> We owe it to ourselves and our community to call out people being lazy and
> selfish, and letting politics overrule sound engineering and decency.
However, if he does support Windows, it might take antirez some extra time to support this patch, mostly front loaded. If it is too much for him personally, he could ask just one of the BigCorps who care about it for some help. (I'll bet Microsoft would be willing to dedicate a dev or two.) The number of hours spent may be in the tens to hundreds, but will be much more efficient overall.
The latter of which, at least, has done exactly what you say you don't want to have happen. Also, neither of those companies are even remotely BigCorp, they're quite small and nimble.
> has to spin up %nix boxes just to run Redis, there will be tens of thousands of hours wasted as each company gets this new environment up and secures and maintains it going forward.
Perhaps an incentive for them to find a way to make an unobtrusive patch that might be accepted into Redis mainline. Or an incentive to press Microsoft for real POSIX support.
None of this is an incentive for antirez, VMware, or anyone else, other than the people who want to run Redis in Windows.
At best, you're arguing the good of society, but getting agreement on what that means requires someone to have made a lot of moral and ethical judgements similar to yours.
It's not a moral and ethical calculation, it is a simple calculation of time. One of our main goals in the software industry is to reduce redundancy. Antirez's decision is increasing it.
Arguing simply based on incentives is also a fallacy. If we were to look at everything from that perspective, no one would donate to charity and far fewer open source projects would exist.
The code is right there. Any of your BigCorps are free to grab it and make it work. They are not free to demand the time of an open-source dev.
> Forking the project is less than ideal.
> Client libraries will have to spend time making sure they
> comply with both versions
> development in general will slow as each project will have fewer total
> devs.
There are some OSS projects that maintain first-class Windows support, but nobody should be surprised that when they find out that Windows is poorly supported by an OSS ecosystem that is predominately focused on producing software for UNIX descendent systems.
As for Reddis, the cost of supporting another operating system -- especially one as different as Windows -- is quite high. If you don't use Windows, there's little cost justification for doing so.
This being a BSD product, BigCorp can do their own port. They can decide to release it or not, based on whether it is more economical for them to share the support load (but also provide the "profit" to their competitors), or not.
Microsoft chose not to with their BSD Socket modification, for example. They can just as well do the same for Redis.
Pure and simple, this is Microsoft tried to get something (remove incentive for people to use competitor) for almost nothing.
When Microsoft NoSQL server arrives (and I suspect it would, sooner or later), if it is even remotely competitive with Redis, you can be sure they'll speak badly of Redis -- they do that to their own products when a new version is out.
So, no, the "subsystem for unix-based applications" is a no go.