The Ph.D Grind: A Ph.D Student Memoir

Wow, I can't believe I just read that whole thing. Maybe it was the excellent writing quality or maybe I just find things like this interesting.

As someone who just graduated with a B.S. in chemical engineering and will start work toward my PhD in the fall (also in chemical engineering, but will actually involve a TON of programming), this strikes a chord with me.

A few takeaways from the article:

-The author does not give himself nearly enough credit. "Self-deprecating" is probably the right term. He considers himself average, but he is far above average. I mean this as a compliment of course.

-I think some of those tools he wrote could actually help me with my work (particularly CDE) -- I'll have to go take a look.

-I am going to have to learn how to focus, and very quickly. I consider myself very motivated/stubborn but I get distracted so easily that it's ridiculous, and this is not going to work for grad school unless I alter my behavior substantially before I get there.

-Before reading his book, I too had the idealized notion of "brilliant idea, years of writing one paper about it". The reality isn't quite as fantastic, but I appreciate learning the truth over belated disillusionment.

-Just realized the author is pgbovine -- excellent work! I currently have a completely free summer to do whatever -- would you have any tips on how I should prepare before I get to grad school? I already know which professor I will be working with, so I'm trying to figure out what level of "initiative" I should take before I arrive.

[ Short on sleep, so the post may sleep incoherent. Excusing myself from any egregious offenses against the English language ]

I read this in its entirety, unable to go to bed before finishing. This is amazing. I'll admit that I started out with skepticism towards Philip's initial research goals.

Instead what I found is an application of immense talent , culminating in software projects that are nothing short of amazing. For what it's worth, something like CDE seems like a holy grail in many scenarios. I've spent some time time thinking about distibuted cluster management software (Google's borg, MS auto-pilot, Hadoop YARN) and the problems these system tend to run into when encountering stateful services. Naturally, I'll spend a bit of time reading up CDE after this.

As Xcelerate said it's quite obvious that the author is _nowhere_ close to average, even in a top-tier program like Stanford. I also admire is that the author was able to maintain this healthy degree of self-deprecation while still doing the inherently (to a research or developer) distasteful but vital work of selling.

Many of my team mates at my current and previous jobs hold PhD. I come from an academic family: my dad was a full professor (e.g., habitulation et al) in our home country, my mom completed her PhD dissertation but wasn't allowed to defend for reasons best answered by a Google search[1]. However, this writeup still shocked me in many ways (the sheer grit and tenacity required on top of extreme mental horsepower). Thank you for sharing this.

[1] https://www.google.com/search?q=jews%20in%20soviet%20univers...

Edit: one _tiny_ suggestion -- pgbovine, I like how you've planned for the book to be accessible to those outside of computer science/software engineering fields (e.g., explaining what device drivers, Python, et al were). That said, I still think there are a few pieces that could be improved in terms of the paper's accessibility to non-CS folks: going into more details into potential sources of bugs (to explain the motivation behind your initial research), the full pain of systems' environment management, etc... You should definitely publish this for a wide audience (at the very least gearing this for students or professionals in all natural sciences and other quantitative fields).

The part that hit home for me was his description of the Prof's frustration with the speed at which he was implementing the enhancement to Klee and the fact that most grad students are not as smart as their advisors.

Most grad students are exhilarated to work with a brilliant researcher but it can crushing when one learns his or her own limitations.

Like the author says, probably 1 in 100 are the equal of their advisor and capable of becoming a professor at a comparable university.

This is great, and thanks for writing it up. I don't think many graduate students would consider doing this right after graduation...I certainly wasn't in a mood to write another detailed document after getting done with my dissertation (also a couple of months ago, and in computer science).

I wanted to add a little about the end-game. For me, the decision of what to do next followed a slightly different path. In the fall of my 5th year, while planning to graduate, I had to really think about what to do next. Academia sounded interesting, so I went ahead and applied to tenure track jobs, and did interviews. Come winter, I had an offer from an Ivy League school. But I had also wanted to do startups for a while.

From my experience, working on projects that were not provided/sanctioned by advisors was somewhat like working on a startup. You have to build things that others find useful/interesting, sell the crap out of it in publications, presentations etc, and I also had to fund it.(my research was building systems software that improves user experience in poor network environments, like those common in developing regions. One of the definite perks was travelling lots and deploying stuff I built.)

In other words, I have no idea why most PhDs are not be expected to go work on a startup when they finish. They are domain experts, and are expected to be good communicators when it is all said and done.

So, when it came time to make the call---an assistant professor, or a startup founder---I chose the later (and disappointed my advisor). Some friends tell me I could have done that without a PhD, and it was a waste of time. I think a PhD naturally leads to a start up. The Grind, as the op calls it, is something that teaches you a lot about yourself, and surrounds you with some of the smartest people you'll find. Your job, every single day, is to innovate [with a more flexible runway]...and you can make (and learn) a lot out of it.

Wonderfully written and a page turner. Bookmarked it but ended up devouring the thing in one sitting. Deserves widespread publication. Guo is a remarkable dude, and brutally honest. It's a tale of perseverance and canny self-promotion from a guy who is spectacularly good at learning from failure. I felt an almost parental joy upon reading the outcome. Publish this on Kindle, Mr. Guo!

As a current graduate student skeptically considering working toward a Ph.D., this has helped me confirm my suspicion that it's not for me. I started doing research as an undergrad, so I have become somewhat familiar with academia's publication game... and I find it distasteful.

Knowing what goes into a Ph.D. is invaluable in making such a decision, and this book was quite helpful in that regard. Thank you!

Pretty amazing memoir. Does a nice job of capturing the ups and downs of the grad student experience. Phil - consider distilling this down as a blog post or two - lots of good lessons here, but given the length they might be lost on most or your potential readers.

whoa i'm astonished people have the attention span to read this on a friday night :)

here's an HTML version too ... http://pgbovine.net/PhD-memoir.htm

I'm only halfway through this, but it rings true to my experience. The circumstances are different, but the emotions and lessons I had to learn are much the same. Near the end of my PhD, when I knew that graduating was just a matter of time, I had difficulty explaining to people what the process had been like, and how to answer people when they asked if they should do it, too. In the future, I'll point them to this.

Once I started reading the memoir I couldn't put it down. As an entrepreneur three points stood out for me: i) Give and ask for help: Asking for help is crucial for the success of any venture. But providing help opens many unseens doors. ii) Give and attend talks: By giving talks you expose your work to a wider audience, more importantly you will get real-time feedback and find a potential customer or two for your product. iii) Network with amazingly talented people: Just a half hour or an hour meeting with smart, highly experienced people can light up possibilities that you may have never considered. Then there are chances that they will introduce you to the next investor or big customer.

"One possible reason I dreaded was that nobody had previously built something like CDE because it was impossible to get the details right to make it work effectively in practice. Maybe it was one of those ideas that looked good on paper but wasn't practically feasible"

No, there was autopackage, alien and any numerous other programs and battles/discussion long email threads that worked not just for python but for any language. Technically speaking as I see CDE, it is useful for python and standard installs in very select cases. For large research projects where the variables and runtimes will differ with no rhyme or reason other than "That's what we started with". It's probably a lot more of an issue to get running properly.

I'm sure there is no easy answer, it's really a human problem. What would really be highly useful is for standardization on a linux distro specifically for research but even then that would only be useful for at most an organization/institution/research lab. Getting researchers to stick to such a distro as a general rule is impossible without weapons.

I had no idea you could get faculty from another institution to be on your thesis committee. More importantly, I had no idea PhD students could do summer internships to supplement their measly stipends. If I knew that I might have applied to grad school already. It seemed like the only people in my undergrad program who had a good idea of how to plan out their academic careers were the ones whose parents were university professors. I felt alienated and left to fend for myself and I still don't know how to break back into that world.

Philip, I first read your blog 2 years ago when I was starting to think about a PhD and I had a decent academic admit season in Fall 2011 when I finally applied (headed to CMU later this year). Recently, I was going through a slump since a paper out of my undergrad thesis I submitted to a conference was tossed out with very harsh reviews. This manuscript once again has come at a very opportune moment. You are an extremely gifted writer. Congratulations on finishing.

This rings true for Humanities Phd's as well. It's difficult to describe the cycle of guilt, drive and passion somehow all wrapped together. This is a great overview.

I just read the first chapter. Very insightful.

Especially early on, I was so personally attached to my work that any problem (eg, an upper bound not working out right) was devastating. But then, when it did work, ecstasy. And then, uncovering bad algebra in the derivation caused despair. And then, error corrected, euphoria. Round and round.

The Google Tech Talk mentioned in the story: http://www.youtube.com/watch?v=6XdwHo1BWwY

"He was an ardent pragmatist who cared more about achieving compelling results than demonstrating theoretical 'interestingness'..."

Are those people actually rare in academia?

I must confess, while the grad school parts resonated for me, the part that was most engaging was the vein of applied provenance tools.

Having recently left my program (but enjoying research), I've actually been spending a huge amount of time planning out and slowly executing building a better tool chain for data analysis, and one of the really exciting details that has popped out of my design is a very nIce support for incremental computation and computational provenance, while having all of the constituent parts be libraries (rather than needing to mod a run time)

Point being I was really intrigued to see the thinking behind other folks getting drawn into that space

Haven't read the memoir yet, but after reading John's review http://blog.regehr.org/archives/743 I'm looking forward to it. Good luck at Google Phil.

This is absolutely brilliant. Extremely well written, I read the whole book without stoppigng... Thanks for sharing your story with the world.

Nice read. Congrats on finishing pgbovine! Did you ever encounter other PhD students taking a similar "entrepreneurial"/independent approach to getting a doctorate?

Off topic, but I chuckled as I'm interning at Microsoft this summer and actually came across a paper on Klee this last week as I was doing research to guide my own testing project. I did a double-take when I saw the name.

As I read I thought, wow this dude sounds just like my friend Phillip from high school. Glad to hear you're doing well.

-Frank

The IncPy idea is very neat. A cursory look at the github repository indicates that it is released as a modified version of CPython 2.6.x. This is unfortunate, because it guarantees that it will soon be obsolete. Ideally it should become a PEP and be implemented by multiple Python implementations; how difficult would that be? From the repo I cannot see how much extra code is involved here.

Another thought I had is whether you could implement this in pure Python using the introspection features of the language. Finally I was reminded how with certain languages such as Lisp, there is this notion of an "image", which is a dump of the state of the interpreter. Does that achieve the same end as this? If so, wouldn't that be a much more general solution?

Amazing piece of writing, can't praise it enough. I am at my second year in the process and already sympathize with most of what you write. Thank you for writing it, hope I will be able to learn from your experience.

Thanks for sharing, I find your writing style very engrossing. As someone who doesn't have much experience in academia, I have a new level of respect for Professors and Ph.D.'s.

While not a Clancy thriller, I appreciate the well-written insights from this paper. I don't know the author yet, but thanks to him for taking the time to write this.

This is a really good read! I'm a first year CS student and I'm interested in going into research later on.

Thanks for writing this.

"involved some of the grimiest programming grind that I had ever done."

More info on CDE (Code, Data, and Environment) without having to read the memoir: http://www.pgbovine.net/cde.html

Awesome read, thanks for taking the time to write it :)

Amazing post! Thanks for sharing.

Just read this in one sitting. In many ways, it parallels an entrepreneur's journey from other stories I read, except at the end the rainbow, there is only graduation.

I'm glad I'm not an academic

Ten mins on the website and this strikes me as all being very "needy". I'm still struggling to understand why everyone feels these type of "little ole me" stories are so fantastic. (It's not the author's first either...)

Just wanted to offer the counter opinion. Each to his own...