siganakis

There are a huge number of genes which are known to be transcribed (as mRNA) and turned into proteins, but we have no real idea as to what these proteins actually looks like, or what their function may be, for example [1].

If we knew their final protein structure, it would be much easier to work out what biological processes they may be involved in.

Also, taking known protein structures and learning more about how they may interact with other proteins would be extremely useful. For example, a large number of molecular pathways in cancer are fairly well described in a general sense, but lack specific information on the actual reactions.

Finally, being able to put it all together in the context of mutated genes (which we can now screen for fairly easily) and being able to determine what impact a mutation may have on the protein (can it still be formed, how will it interact in a pathway?). For example, some mutations break a protein completely, while others may impact "switch" genes, leaving them in a permanently "on" state preventing them for reacting to external signals [2].

This kind of thing is really important in cancer research (and one day treatment), where the function of specific proteins may be the difference between life and death.

[1] http://www.genecards.org/cgi-bin/carddisp.pl?gene=C1orf186 [2] http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1891745/

+1 for the Pentel P205, it's by far the nicest pencil I have written with, and for well under $10.

Writing with pencil, especially when working through a problem, lets me work more effectively. Knowing you can erase something gives you a sense of freedom to explore ideas that I don't get when writing with ink. Kind of like having an "undo" feature in an application lets you be a little more cavalier with your experimentation.

But then you have the new problems of what plugins to include by default, how users discover plugins and how they are configured.

Plugins, while valuable for people willing to take the time to learn about your product's eco-system are unlikely to help users who are willing to spend only 5 minutes evaluating your product.

> Murder is banned. He committed more than twenty murders. Why do people think that you can ban mental illness? The focus on guns as opposed to the person is similar to the TSA's focus on weapons vs. terrorists.

Of course you can't ban mental illness, however you can make it more difficult for people to access the tools that enable them to kill many people.

Guns are very much are a part of the problem. I'd also think that the culture of gun worship is also a big part of the problem.

If competence is absolute, then there should be some way that you can objectively measure it (otherwise how do you know its absolute?).

What are the ways to objectively measure the competence of a programmer?

If that was the case, then it would be much, much easier to hire and fairly compensate programmers than it is in reality.

He wanted to study "pure" maths, not applied:

"Moscow had many schools, but there was only one place to study pure math: Moscow State University, known by its Russian abbreviation MGU, Moskovskiy Gosudarstvenny Universitet. Its famous Mekh-Mat, the Department of Mechanics and Mathematics, was the flagship mathematics program of the USSR. Since I wanted to study pure math, I had no choice but to apply there."

I think that the bigger problem is when people begin using "social networking" while they are actually doing the intimate, stimulating, exciting things in real life - either looking for validation or because its easier to bury your head in your phone than it is to participate in social interactions.

For example, checking Facebook while out at dinner with friends, or uploading photos of a party while you are at the party. What could possibly be happening on the internet that is more important than being out with friends?

Fair enough if you want to check facebook while you are waiting on a train killing time, but when you are actually doing the things you hope to be reading about on Facebook/Twitter/Intagram, something seems very wrong.

But to prove that you need to show that there exists a neotenous trait that improves sexual selection.

Then you need to prove that the lactate gene is in linkage disequilibrium with that gene (thus capable of being "coming along for the ride".

Given the impact of a genetic mutation that improves metabolism, which has its own evolutionary benefits (improved ability to access energy) there seems little reason to invoke this third trait.

I used it for a complicated round the world itinerary.

My first competition didn't get any takers ending with no flights, but I contacted support and they re-ran it and I got about 8 different itineraries - all cheaper than expected.

Funnily, there was $1000 to be saved by flying from Melbourne to St Lucia (Carribean) via London (~15,000 miles) rather than via LA (~11,000 miles). That made no sense to me at all!

Data should not be organised based around retrieval or insert / update patterns but organised according to the model that best captures the essence of what the data is. That may sound fluffy, but most of the time data captured is captured following something real happening that caused data to be generated. Your data model needs to make sense in the context of the thing that happened in the real world, not in the context of what is inserted or how it is read.

The issue being that youre methods of collection and retrieval will change over time and your data model needs to support that and still make sense for existing data.

Also of interest is the observation that Ketamine (and PCP) are associated with NMDA Antagonist Neurotoxicity (Olney's Lesions), a form of brain damage.

Not really something you should want to self medicate with at this point until more research is done.

http://www.dextroverse.org/txt/olney.txt

http://en.wikipedia.org/wiki/Olneys_lesions

Essentially they end up rebuilding the "transaction log" that most RDBMS's use to commit data safely.

Before any data is written to the "data" files of the database, it is written to a "log" file sequentially. Once that has succeeded, it will write to the data file, then write to the log file again to say that the write was successful.

That way if the database breaks during the write, the system knows about it. It's also useful because it allows you to restore the database to a point in time, as you have a full log of all the transactions run (and you know how to un-run them).

In this case, they just make the transaction log a little more accessible.

For your example, erroneous historical data would remain in the system but with a new record being added indicating that at a certain date and time the old record was replaced with the new one.

Basically if you think of a store processing a purchase for $100 and refund of $100 as 2 transactions (for $100 and -$100) rather than simply deleting the first transaction.

This is important because things may have happened between the deposit of money and its refunding (e.g. interest payments).

It does seem like a lot of work to me.

One of the problems is servicing two workloads, one for "transactional" processing and another for analysis.

For transactional systems you need to be able to change things quickly and consistently. For analysis you need to be able to query lots of data quickly.

For decades people have realized that these are two separate workloads, so have built 2 systems, a transactional system (on an RDBMS) and a data warehouse (generally on an RDBMS). Data is then shipped between the 2 in batch jobs.

The transactional system is normalized, and the data warehouse is normalized. Within the data warehouse you make denormalized copies of the data that fit the reporting workload required so that as much of the workload is pre-computed as possible.

The problem is that as reporting requirements change, you need to modify these pre-computed stores, as they are very heavily tuned for the particular reporting requirement. Building pre-computed stores is generally done in SQL and can be challenging as you are generally shifting a lot of data and you are trusting the RDBMS to get its optimizations right.

There is a trend now to use Hadoop for the building of these pre-computed stores (and even to use Hadoop for the entire data warehouse). However, writing map-reduce jobs for queries is cumbersome compared to SQL so your productivity suffers. But you don't have to pay Oracle or IBM for licenses.

The key problem is that you need to pre-compute stuff to do fast aggregation, but you can't pre-compute everything. So what you pre-compute is dependent on what your users want, and that changes all the time.

So what you want is a system that lets you change what is pre-computed easily and efficiently.

There is a world of difference between a physical working prototype and product that you can deliver in commercial quantities.

I guess Kickstarter is aiming to fund physical projects that already have a working prototype and require funding to deliver in large quantities.

It seems fair to me, as the person requesting funding should be required to have put in a fair amount of effort beyond a flashy 3 minute video promising the world. A working prototype seems to be a fair bar.

The simulation/rendering rule is only for "Hardware and Product Design Projects", not software.

I think that there is a tension between "creepiness" and "effective marketing". This I feel is one of Facebook's core problems, where for them to maximize the value of their dataset, their ad targeting becomes incredibly creepy.

One issue is that from an end users perspective it makes it obvious how much information is being captured about them. While most people are aware that their information is being captured, seeing it plastered all over their facebook feed makes them confront it.

Worse than that are the questions that come with these ads - "Why am I seeing ads for baldness cures?" Is it because I'm a 30+ male, or is it because they have analysed photos I'm tagged in and detected my thinning hair? Sometimes it just feels mean!

This is primarily a challenge of data science working in a marketing environment and doesn't really permeate through all areas of data science, however it is the form of data science that is most visible. Therefore much of data science and the big data we work with gets lumped in with sleazy marketing.

Most of these points rely on the belief that evolution is always made up of slow, small changes in the genetic code. While this is the case most of the time (and enables the fine tuning that we see in many species), evolution can happen very quickly.

Population bottlenecks cause species that were once very fit to no longer be fit for an environment, enabling vastly different organisms to become dominant very quickly.

Then there are phenomena like Chromothripsis[1] (only discovered last year), whereby the genetic code of a cell is literally smashed into thousands of pieces and seemingly randomly reassembled. No one is saying that Chromothripsis is a major driver of evolution, but it is interesting in that massive rearrangements of this nature can still provide cells capable of reproducing. In the case of cancer, where it was first discovered, these cells are even able to reproduce faster than normal cells.

The other thing is that even advanced life like mammals and humans have more than 10 times as many bacterial cells than human cells[2]. Since bacteria can evolve very quickly, they play a very important role in evolution.

The eyes from no eyes makes no sense to me, since the entire array of light sensing capabilities are seen in nature. From eagles to plants. The same is true for many of the examples of emergent complexity that are thrown out there.

[1] http://en.wikipedia.org/wiki/Chromothripsis

[2] http://www.scientificamerican.com/article.cfm?id=strange-but...

I think that you have confused mutation with gene in your initial assertion.

There is probably only a few percent of variation between all of your 2^100 ancestors genomes (the majority of whom will be related), so protein coding regions of your genome and your MRCA are actually likely to be very similar.

Now if that MRCA had a specific mutation, the chances of your inheriting that specific mutation are probably low, but depend on that mutations prevalence in the population as well as whether the mutation is deleterious or not.

100 generations is very few in terms of changes within the genetic make up of a population under normal circumstances.

Isn't RNASeq more like looking at really verbose log messages? Where you only sample 1% of the log messages and may be looking for an event that happens 0.01% of the time?

Hello, I was in a similar position the year before last. Essentially I was well paid, but realized that my career was either going to go into management, or had reached a plateau. Basically, much of the challenge of programming had been lost and every project started looking the same.

I decided to do a Masters, but in Bioinformatics rather than CS. Bioinformatics is basically the application of computing to biological data sets, and in my case whole genome sequences.

I won't end up a bioinformatician as the money is no good, but the background the course has provided in statistics looks like it will be very important in my career development.

The most important thing about any Masters is the quality of research projects. You will learn much more in conducting your own research than through coursework, and it will put you in contact with really smart, interesting people.

The good:

- More free time to work on side projects / start a consulting, contracting business

- If your research is interesting, it will open up doors (my research is in cancer at a world class cancer hospital, which seems to give my resume much more credibility)

- It gives you perspective on life, a chance to question what you really want out of it

- A different class of job opportunity becomes available, although this seems to be related to your research (e.g. if your research project was in driverless cars, I would expect that just about any tech company would at least interview you just so they can chat about it!)

- You'll experiment with new technologies. Before starting my studies I was very much focussed on the Microsoft stack, but most of my research has now been on the nix stack which means I now have a much more diverse skill set.

The bad:

- Much less money. Even with contracting, there will be times when cash is tight and expenses crop up (like needing to fix your car!)

- Pressure to jump into a phd. Worse of all, you'll probably end up wanting to do a phd, which is cool if you are young, but as you approach 30 the thought of tying up another 4 years studying without making much money is scary.

- Exams, they don't suck any less now your older

Basically I think that going back to study has been one of my best decisions. Its completely changed my career path for the better and given me a new sense of excitement for my work.

[Edit: formatting]

Thanks for your comment. My email is terence (at) siganakis [dot] com if you would like to chat.

The R integration stuff is more for the data science crowd. I feel it would be useful to have a streamlined way of writing a query to return a subset of a large database and then be able to manipulate it in R as if it were a data frame / matrix. I am interested in seeing how Julia plays out too.

I think that there is a feeling in the community among many people that all genetic information should be freely available, which at this stage I feel is wrong. When people go to a doctor, they expect answers, and the reality is that in many cases the researchers just don't know. This technology is so new, relatively unproven and potentially dangerous that circumstances such as the one you describe are likely to continue to arise.

An interesting thought experiment would be, what if Mr Lyons had been wrong, and had claimed that the fetus was carrying the mutation when he was not? What if the test had been compromised and he had told the Mrs Grondahl that her child was going to be sick when she wasn't?

In this type of new research, the chances of the researcher getting it wrong are very high. It takes years for people to truly understand the complex nature of disease and its interaction with DNA.

With this type of research, I feel that we are going to see a number of incidences where the researchers do in fact get it wrong, and do a lot of damage by being cavalier with their findings.

Medical research moves slowly for a reason.

I too have spent quite a few years working in this area. I have primarily been involved in building tools to make this kind of analysis easier - but it is very difficult.

As you mention, the primary issue is that almost all business data is very complex. There are very few areas (in my experience), where simple queries offer much value, because there are so many niggling business rules that lead to exceptions and counter exceptions to rules.

This means that "self service" analytics are pretty much impossible. There are often very few people in an organization who actually know how to define even seemingly simple measures like profit or revenue.

This means you need consultants who can learn the business rules, and who also understand the data platform. This makes these projects expensive, and makes it hard for platforms to win in the market.

For the most part in my experience, the best way forward is to choose a database engine (MySQL, Postgress, Oracle, DB2, SQL Server), get the data into it (itself difficult!), then write a bunch of SQL to incorporate the business rules. Then if you are feeling fancy (and still have budget!), built a nice HTML front end on it.

Its not very sexy, but it does work. I would love it if there was an easier way, but I am yet to find it.

From the pledge section:

$50 gets you access in May 2013 $100 gets you access in December 2012 $200 gets you access in October 2012 $500 gets you access in August 2012 $1000+ gets you access in May

So the product will be open source, but you need a license to download it? So its open source, but not "free". Are there any well known licenses that support this?

I take it that it also means that the license will not allow for you to copy / redistribute Light Table? Does it still really qualify as open source then?

Will they allow for contributors to the core of the product and if so, what rights do contributors need to sign away?

That said, I think its an awesome project!

Cancer survival rates are a mixed bag with many barely moving at all. The statistics below are based on that for Victoria, Australia (population of about 5.5m).

5 year survival rates for the following cancers has barely changed at all since 1985: Pancreatic, Larynx, Lung, Mesothelioma, Ovarian, Cervix, Central Nervous System.

Although the following cancers have seen much better improvement:

Pharynx, Colorectal, Stomach, Oesophagus, Gallbladder, Breast, Prostate, Kidney, Thyroid, HL, NHL, MM, ALL (basically all the blood cancers).

Overall the 5 year survival rates for cancers have improved from 47% to 64% now. Definitely better, but unfortunately not "spectacularly better".

Of course, these are survival rates, so do not reflect improvements in public health that prevent people from getting cancer (e.g. survival rates for lung cancer have have actually decreased, but less people are acquiring it due to reductions in smoking meaning deaths have reduced by about 50%).

http://www.cancervic.org.au/downloads/cec/cancer-in-vic/CCV-...

The trouble is that there are many, many different ways that a cancerous cell is able to be cancerous.

A malignant cancer cell has generally had upwards of 7 mutational events that directly enable it to replicate itself enough times to form a tumor and become damaging.

These mutations includes things like breaking programmed cell death (apoptosis) pathways (e.g. P53 mutations), increases in anaerobic glycolysis (due to lack of oxygen within the micro-environment of a tumor), increasing production of angiogenesis related proteins (to create new blood vessels to the tumor so that it can continue to grow).

Cancer cells even acquire mutations to the pathways that regulate DNA damage repair and replication so that further mutations happen more easily, providing more variation in the clonal colony of cancerous cells making the "cancer" more able to withstand changes in the tumors environment (e.g. the introduction of chemotherapy, hypoxia, etc). Essentially the cancer evolves over relatively short periods of time.

In other words, this is just one pathway that there is a chance we may be able to disrupt, but it may very well be a crucial one. If we can build up enough drugs to target the specific mutations that make some cancers so aggressive, then we stand a good chance at significantly reducing the mortality of cancer.

Excellent research, but we have a long way to go still.

Bitmaps are very good at "Seek" operations (where I try to find a value), but poor at "Access" (where I try to find the value at an index). This is because "Access" requires that I check all the bitmaps to see which one has the set value.

Also the "Word Aligned Hybrid" method of RLE compression is patented. http://www.freepatentsonline.com/6831575.html

My bad, I got that wrong.

The translated array should be:

1 0 2 3 0 3.

I am correcting the article now. For a non-rushed, accurate description please look at:

http://www.alexbowe.com/wavelet-trees

Edit: Got it wrong again... I guess this is why we have computers and debuggers!

Thank you, this is exactly why I wrote the article - knew I was missing something!

I'll do a follow up with the rank / select optimizations.

Thanks!