You have basically two strategies - being what you are or being a karma-whore. First one is better in the long run.
-24 karma · joined June 10, 2011
contact me at schiptsov@gmail.com
You have basically two strategies - being what you are or being a karma-whore. First one is better in the long run.
And everyone should be happy - without BS projects jobless ratio among mediocre coders and ctrl-c-ctrl-v sysadmins will be much higher.)
I appreciate your innovative idea and amount of work you have done, so this small efficiency issue does not really matter.
btw, who cares about resources when hardware is so cheap and purchased in ocean containers? ^_^
Or, at least, in more suitable Erlang? ^_^
Isn't it an obvious startup-idea?
Using HTTP[S] protocol allows you to pass through almost every heterogeneous network. That means any highly restricted and protected (I prefer the term 'misconfigured') network of a cell phone provider, who blocks everything else, but http.
You can survive in that crazy hotel's or airport's wi-fi network, especially located in developing countries, which is a mess of cheap Chinese ADSL-modems and no-name Wi-Fi access-points with settings no one cares about.
It also means that you can pass through all these almost useless corporate firewalls made by idiots (why firewall if you have all those windows with IE6 boxes running by users with admin rights?).
The second big thing is about statelessness and using URI as a namespace - representing data as a file-system hierarchy and using one standard protocol everywhere - it is almost the same ideas which lays on the foundation of this enlightening Plan9 system. No xml, no BS, just names and simple standard protocol - 9P.
Together, those two ideas are enough to forget about all that proprietary crap, not because it is proprietary, but because REST is more general, flexible and easy to implement concept.
First, it is useless to store data in memory if you want them be committed into disk storage. The general idea here isn't about switching to some new version of mysql or SSD disks, it is about to realize that you have a data-flow inadequate for your one-server architecture.
Second - check-point intervals should be adjusted to your actual data-flow, which means they should be executed often enough. If there is situation of almost constant checkpoint - non-stop data writes, that is the sign that you need to consider sharding/multi-server solution.
The hints that there must be no other disk activity on the same hardware volume or any swapping in OS, I suppose, are obvious. People who have a /var/log and /var/db on the same volume are idiots.
There are also good idea to use one file per table storage and put a data and physical logs on a separate hardware volumes (links are your friends). One raid-X volume that fits all is a quite naive solution. Raid isn't a guaranty of reliability. Replications to a back-up servers are.
Third, when you test your configuration before put it into production, you should tune-up your servers to perform with data and log syncing, and then figure out appropriate buffer sizes and checkpoint intervals. Then, in production, when you're experiencing an increasing flow of queries, you may choice to switch into different syncing strategy and/or more often but a little bit faster checkpoints.
Configuring mysql with huge buffers and no sync means lack of understanding the basic concepts, self-delusion and misuse of software and hardware. ^_^
The vision should be a little bit broader (and scarier to those invested in it and its clones) - OK, every teenager in the world already has a FB page and uploaded some photos of herself and exchanged some stupid messages with so-called friend. Wow-impulse has faded away. Now what? ^_^
Don't even try to say 'the next facebook' - it will be 'just another social site', af FB was for MySpace or Livejournal's addicts. ^_^
What it could be? Ok, something like a cam on your clothes to broadcast 24/7 via some G5 GSM directly to some datacity which sends it back to your personal 3D movie hall (forget Youtube with a that crappy flash player)?
No, I don't think so. The time of the mass-exhibitionism in the net is, it seems, over. ^_^
Data Warehousing is all about JOINS, big JOINS. Or that thing they call it data-cubes. In that case you need query optimizer, lot and lots of buffers, data partitioning and several layers of caches. You also should use stored procedures, because it is a good way to structure and manage your code, same way modules work for other languages. So, you know, that old lovely DB2.
Even in old times, people who claims that there is a solutions that fits both cases were considered crazy. That is why no sane person considered MSSQL (leave alone MySQL) as something other that a taste-less joke.
Nowadays people forgot about designing in terms of data flows. Everything starts with installing some framework, such as Spring+Hibernate, Rails or some PHP crap. They forgot that not tables itself, but queries (which type, how often) to them is what matters, that indexes optimized for actual query flow is what performance of a server is all about, and that actual structure of tables (and corresponding indexes) must be adapted/redesigned for that particular production flow of data. That was a DBA's job.
Today some people believe that they can eliminate smart but costly engineers (DBAs or sysadmins) by some software which is marketed to them as Easy, Smart, Fast, Zero-thinking or whatever - ready meals for a mediocre crowds. OK, if you're building a 20 pages web-site for a 100 visitors per day, that might work - you can save some money and time, but, if it is a industrial or internet-scale solution, there is no chance that you can run Rails or say Code Igniter crap in production without huge changes or total redesign. No fucking way.
So, all those specialized solutions, such as memcached, redis, membase, mongoDB are about dealing with flows of technical data, such as AJAX queries from UI, logs, authorization requests, chats, photos and other unimportant things, OR about building a huge distributed cache layer above actual data store. But, of course, you cannot build a Data Warehouse out of it. (Or invent a complete different approach to dealing with data, such as map/reduce).
So, ORM is anti-pattern? It is not efficient? Ridiculous. It is just broken by design. ^_^
So, that rumors about miserable average level of intelligence in US aren't just rumors? ^_^
And Power (can't tell how I hate that stupid primitive branding for idiots by idiots) is an attempt to be like AWS - disk-image based hosting.
What is really interesting, is that some people have invested money in such projects at this time, and believes that it will be profitable. ^_^
One who wants to learn how to sell fruits or vegetables should visit some big Asian bazar (local word for a marketplace) and take a look. Most of sellers are gurus of merchandising, which in this context means how to place fruits, which ones to put together, which ones to put aside, which ones in customer's reach, which one only to display, aren't some kind of WF innovations, but quite old ideas. And instead of flowers they put fresh tiny branches with leafs, as if they were accidentally cut in harvesting. And of course, the ideas about showing boxes, as if it was just fresh delivery, or putting drops of water on fruits, or making a fresh cuts, giving you some fruit to touch or to smell, and so on. The best markets I have seen was inside and around Kashmir valley, and, of course, street vendors in Nepal's capital Kathmandu. So, this article is something like, I don't know, an amateur take to the subject. ^_^
And all that Whole Foods thing is just for people who know no better. Fresh means when it comes morning time directly from a tree or from a field by people who brought stuff to sellers.))
I had a lot of experience with Solaris (x86 only - people tried to run Informix/Oracle on a cheap hardware) starting from Solaris 7 and onwards. It was always a problem even to install it. And lack of working compiler makes things even worse.
Starting from Solaris 10 they did a lot of work to improve the overall experience, but too late - everyone migrated to Linux to run the same crap.
And I must say, that after it was installed and tuned it was running quite stable as a database server and it can deal with heavy loads, while similar Linux instances failed now and then. But it was 5 years ago. Modern Linux kernels can handle everything quite well.
So, if it OpenSolaris based, then nothing to see here. Community is too small. Who will write and test up-to-date drivers for all new hardware that vendors are pushing to the market each half-of-year?
Update: Oh, come on. I've clicked to the TECHNOLOGY link and what I saw instead of technology review is load of BS. SmartMachines? OK, even I know that one should market any crap with Smart or Easy or Eco or LowFat prefix in it, because, you now, I have a SmartMachine... OK, fine. But what about technology?
Instead of writing that you're building a solution based on fast, light-weight, low-overhead, in-kernel, native virtualization system and RedHat supported libvirt stack, that you're co-sponsoring and actively participating in development process, and here is our contribution and so on, I see loads of BS. KVM is faster than Xen? I know that, thank you. I have a KVM instances running on my Laptop. ^_^
You're using ZFS? Linux native port? FUSE? So, you're active developer and tester? Co-sponsor? You have hired or supporting active developers? Providing a feedback to community? No? You just trying to sell me something you called a SmartMachine, OK, fine. I don't buy it. ^_^
The ability to run an old win32 desktop crap.exe is what Windows is all about. Only a complete idiot will choice it as a platform for a new, build from the ground up project, or, god forbid, a server.
And there is enough ways to run a web-browser, especially plug-in-less one. It is called Android. ^_^
Repeat after me: Javascript is just an in-browser scripting language. Period.
Yes. I know, it is possible to use JS on a server side, but you also can hack a VB run-time to send you some files via http. Wait, isn't that crap already exist and called Azure?
Most of really good ideas have been discovered long long ago. ^_^
2. Assumption that when a large project with lots of coders involved grows (and matures) the number of bugs (per line of code) decreases, is very naive one. (see pp 1)
3. Of course, all those useless layers and piles of poorly-designed abstractions we used to see in a typical Java projects are the results of automated memory management and Moore's law. ^_^
If one neglects what is under the hood one will eventually run into a trouble. Memory and its management are still here, like another processes, flows of data and other dynamics. Not thinking about them does not eliminate them from existence. JVM is a user-level process, one of many.
So, it is not a risk homeostasis it is a mere ignorance. ^_^
As a consequence, less code means less consumption of resources, for Java it is very true.))
Just by looking at the code one could see that Clojure makes the development process for JVM less annoying and frustrating while Scala makes it even more verbose and complicated. ^_^