srini1234

a new disturbing trend http://us.classifieds.sulekha.com/business-analyst-proxy-int...

I think they are gold/uranium/diamond mines. these minerals are usually found in channels. depending on the concentration of the minerals, they dug out the sand/mud and half-filled them with chalk, so that they can take an aerial picture and analyze the area for coverage, further prospecting etc.

I am a recent convert too (< 1yr). If you are moving over from Ubuntu,the transition will be less traumatic. First thing to figure out is the "state" of dock icons. There is a little halo under the dock icons that tells you have an active window. Then figuring out the Apple+Key convention as opposed to Ctrl+Key convention. Also figure out the ease of installation and uninstalling apps, which blew my mind.

As for the apps, get MacPorts and get meld for file compares. SmartGit client is great. TextWrangler is great too. Figuring out how to get all vim extensions. Actually figuring out the "package" architecture of an installed app is also important.

There is only one company I would love to see Apple acquire - Nintendo

Your application spans several major areas of computing technology. First, you would need a highly efficient data capture system - most epidemic information (flu etc.) come from non-structured inputs such as web search trends, texting, hospital emergency call patterns, Hospital staff reports, CDC tracking, PCP tracking etc. These inputs are like fire-hoses. So, you must have an efficient input interface. Keywords here are XMPP, eJabberD, SMS Gateway, etc.

Next the data must be stored and sorted efficiently, so that the analytical engines can easily produce intelligent outcomes. Here the keywords are: NoSQL, Hadoop, Map-Reduce, BigTable, Mongo, etc. Read various High Scalability case studies such as Twitter, Facebook, Flicker, etc.

Next you need to deploy an analytical engine so that the visitors can run their own decision making queries. You probably have to prepare some standard/canned reports also. Here the Keywords are: Machine Learning (ML), Weka, Bayesian Partitioning, Markov Chains, PMML, etc.

Then you have to actually write the web app and put it in a hosting facility for the world to access. Here there are many options. Keywords are: Python, Rails, Heroku, AWS, EC2, Rackspace, Azure etc.

Yes, it's a good idea to hire a hacker at least part-time or even find a Tech Co-founder. In your case, it is better if the person has some domain knowledge (Social Health issues).

Firstly, write scaling is a hard problem. Secondly, secret to write scaling is in read scaling. That is, cache reads as much as possible, so that the writes get all the throughput.

Having said that, there are a bunch of things one can do:

1) Start with the data model. Design a data model so that you do not ever delete or update a table. Only operation is an insert. Use Effective Date, Effective Sequence and Effective Status to implement Insert, Update and Delete operations using just the Insert Command. This concept is called Append Only model. Checkout RethinkDB..

2) Set the Concurrent Insert flag to 1. This makes sure that the tables keep inserting while reads are in progress.

3) When you have only Inserts at the tail, you may not need row-level locks. So, use MyISAM (this is not to take anything away from InnoDB, which I will come to later).

4) If all this does not do much, create a replica table in Memory Engine. If you have a table called MY_DATA, create a table called MY_DATA_MEM in memory table.

5) Redirect all Inserts to the MEM table. Create a View that UNIONS both tables and use that view as your Read Source.

6) Write a daemon that periodically moves MEM contents to the Main table and deletes from the Mem table. It may be ideal to implement the MOVE operation as a Delete trigger on the Mem table (I am hoping triggers are possible on Memory Engine, not entirely sure).

7) Do not do any deletes or Updates on the MEM table (they are slow) also pay attention to the cardinality of the keys in your table (HASH vs B-Tree : Low Card -> Hash, High Card-> B-Tree)

8) Even if all the above does not work, ditch jdbc/odbc. Move to InnoDB and use Handler Socket interface to do the direct inserts (Google for Yoshinori-San MySQL)

I have not used the HS myself, but the benchmarks are impressive. There is a even Java HS Project on Google Code.

Hope that helps..