As someone who practices bioinformatics, it doesn’t seem appealing. Bioinformatics is like 0.1% dealing with FASTQ files and the rest is using the ecosystem of libraries for statistics and plotting. Many of them in R, by the way.
I'm a microbiologist though, for stuff like human RNA-Seq I understand that it's often plug and play to get a gene counts table at this point.
I'm a microbiologist too, but the kind that uses mostly off-the-shelf tools to do the taxonomic/functional assignment on metagenomes, and then stats/data science on the features. I kinda didn't know what you mean by "99% of the scientifically important stuff happens before the stats and the plotting".
I mean, give me a 500x2.6x10^6 sparse matrix of gene function abundances and tell me that you've done anything scientifically meaningful. Or on the other side, let me hand you a fastq file from sequencing a poorly extracted DNA sample, and you give me the best algorithm in the world, and there's nothing scientifically meaningful that's going to come out of that.
I got my start at a NGS facility, so handling FASTQ was closer to 80% of my time, so any speedups would have been greatly appreciated.
Agreed. I know people in my department who just ran Galaxy pipelines and R scripts to make pretty plots. I was on the other side of the spectrum and needed fast parsers, so the SAM and VCF specifications were my bible.
If this is not the way to remove workflow friction, what is?
1) The fellows writing papers - usually these guys have PhDs. Usually a science-focused PhD. 2) Analysts - often have a background in mathematics, biology, or big-data. Success here can lead to an onramp to camp 1. Much of your time here is spent in interactive programming environments, like Jupyter notebooks. 3) Programmers - writing novel or faster bioinformatic tools, often in low-level languages like C++ or Rust. Sometimes you can get a paper out of these, especially if you have a CS background. There's increasingly room for higher-level tools though here too, so it starts to overlap with 2. 4) Pipeline programmers - people gluing analysis workflows together out of the tools written in low-level languages, often with a liberal helping of Unix command-fu. Often sort of an ad-hoc role, containing people from diverse backgrounds, from biology to sysadmin. (This is my current role). 5) Biology/wetlab - people running experiments in the lab, and want to analyze their own work, especially for QC purposes. Wild-west ad-hoc development practices.
I was quite helpful to him and the other students (who mostly struggled with packaging: conda, pip, apt, etc). In turn, they were quite patient with my lack of biochemistry background. It was nice to get a taste without having to take what would've been 2.5 years worth of prerequisites.
But bioinformatics is an umbrella term. There’re so many different things people do. I started by identifying field I’m interested in (ageing and immunology) and backtracked from there.
I’m wary of software engineers coming over the bioinformatics because they never have the domain expertise required to make meaningful contributions, and yet many think they know everything.
For me, the pain points are often the same as in business. Biologists with no data analysis experience want something done without understanding constraints. Requirements are often not understood and there isn’t a good plan.
Some people do indeed suffer from code being slow and this can be solved with better tools. I works with large datasets in single-cell genomics (over a million cells) and the model takes ~12 hrs to train on an entry-level GPU. So, most o my time is spent at trying to understand the results.