The PhD Path: Experience and Lessons Learned

For my first (two-part) blog post, i'm going to discuss my own path through graduate school and try to generalize the experience where I can in order to give general advice that can help students currently (or thinking about) pursuing a PhD in Science/Engineering.  The views expressed here at strictly my own and do not reflect the position of EECS or the computer science division here at UC Berkeley.
***Getting in***
My acceptance into graduate school was not standard.  I worked for 4 years after graduating college, enrolled in Berkeley as a non-matriculated student, engaged in research with the faculty in the computer science department, and finally published some papers, sealing my acceptance.  The whole process took 2 years, from 2005-2007.  Mid-way through the PhD program, I participated (twice) in the computer science division's admissions committee.  
From my own experience, I can say that the main thing the department is looking for is demonstrated ability to do excellent research.  Why is this the case?  In a very general sense, professors in the computer science department have a budget for their graduate students.  They have a vision for a set of multi-year projects they want to invest time (and money) into, and they're looking to build a team.  The time-frame for that vision is typically 5-7 years and they want to make sure they're getting the very best student to help that vision come to fruition.
The best way to demonstrate that you can do excellent research is to, well, do excellent research.  This is much harder than getting good grades.  For your application packet you, ideally, you want a mix of good grades and co-authored papers (even if you're not the lead author).  This is especially true in the CS department where the acceptance rate is about 2-3%.  You have to stand out from the majority of applications and doing research is the best way to do that.
If your grades are not that great, then you MUST have done research and your position in the author list needs to rise.  The better your grades, the less they expect your impact on a previous research projects to be.  For systems research in computer science, papers tend to have multiple authors -- since systems typically take a team to build -- so becoming a co-author is not as difficult as, say, computer science theory.  Still, you have to demonstrate initiative and ability.  You won't be "hired" for a job you've never done.
Getting research done as an undergraduate is difficult.  Places like MIT, Stanford, and Berkeley make that much easier.  In most cases, the best way is to engage is a summer research opportunity somewhere and work hard.  Show your graduate mentor that you really, really want to publish something.  They'll almost always respond positively to this attitude -- graduate students are HIGHLY incentivized to publish.  It is THE metric used to evaluate your career as a researcher and it starts as soon as you start publishing!  Furthermore, and most importantly, if you publish you'll get great recommendations.  The best recommendations in computer science come from students who have research exeperience.  Optimize for this.
The bottom line is that you need to demonstrate that you're going to be an asset.  They examine your entire "score card" (grades, GREs, previous research) in order to make that assessment.  GRE's matter only if they're bad.  Nobody cares if they're good.  Everyone who gets in has good GRE scores.  That said, if you write a good paper or two, it (and the recommendation it induces) can trump everything.  It's the metric that carries the most weight.
***The Beginning***
The beginning of the program is when you try to transition into the role of a researcher from an undergraduate.  Classes generally matter a lot less than they did in undergrad.  In computer science, they end up not mattering at all -- except to attain a passing grade.  In the computer science department, you take some classes mix it with some research.  The classes you take are more of a tool-building activity, to give you the right set of tools to engage in cutting-edge research.  In my case, I took a machine learning course and a few advanced systems courses.
At Berkeley, the advanced systems courses consist of reading many (dozens) of systems papers.  You must become trained in reading, writing, and distilling those papers into their core idea.  You read a mix of good and bad papers, the bad are really so you can identify them, and the good are to teach you how good papers are written.  At the end of each course, you're expected to do a research project.  In some cases (mine included), those research projects are the beginning of publishable work.
As you build your tool box and engage in some research, you start to get a sense for what your're really interested in and you start to get a sense for which faculty and student teams you fit best in.  In other departments, this whole process is more structured, with explicit matching processes and time frames.  In computer science at Berkeley, it's more ad-hoc.  You attend  few research meetings, take classes with future colleagues interested in the same area, and you eventually settle with a research advisor that will guide you at least through your masters.
Decide what you're really interested in here, but like most choices, it's important to look around and try your hands at doing a few things.  Most classes will give you that opportunity.  If I had to do it again, I would not only work with my peers, but also look to do some work with more senior graduate students.  They can teach you a lot about the culture of the lab your happen to join to about how to do research in general.  They can also make you a co-author on a paper early in your graduate career, which will give you the confidence that you could go ahead and start leading a team after doing that a couple of times.  DO NOT think in terms of the PhD here.  Think in terms of the masters project-horizon.  If you find something you're really into, explore it and think about how a set of 1-3 projects are going to come together in a common theme for writing your masters report at the end of your 2 year (or so).
***The Masters (Should I stay or should i go?)***
After a few years (typically 2-3 in computer science), you have the option to write a masters report and decide if you're going to opt out to go into industry or continue in the program.  It's a hard choice for a lot of people in computer science, since what's waiting is typically a nice cushy job with a pretty decent paycheck (to say the least).  But if you've really enjoyed your time and feel like research is the right professional venue for you, then you stay.  Another thing you have the option to do here, is break away from your current advisor (in computer science, you can do this more easily than in other departments).  If you're pivoting to a new area or decide that the direction your advisor is headed in is not the direction you want to head in, make the hard choice to switch.
Your master's project is a nice breaking point, professionally.  It can either serve is an indicator to jobs that you know how to engage in the practice of your field and apply what you learned as an undergraduate or it can set a good foundation for your PhD research.  In either case, use it as a driver for exploring a topical interest you have, rather than something that will lead to your PhD research.
***Dissertation work (the beginning)***
In some cases, your master's research leads into your PhD research.  In either case, you need to start thinking more broadly about the next set of 3-4 projects your're going to work on and their general theme.  That could be in a completely new area, in which case, it'll be a redux of the beginning, where you're checking out new project areas you want to dive into.  In other cases, that means you should but the master's results in a broader context and go from there.
Read, read, read.  Read as much as you can about a topic you're interested in.  Even if you're pretty close to an expert in that area, you should read and start writing down ideas what for something you want to explore.  Think of it scientifically and think about the experiments you're going to run to dig through those results.  Where possible, be systematic.  I was not as systematic as i should have been and it only leads to more confusion as you eventually manage to put all the pieces together.  For any big questions you have, run small experiments that can help make you more confident that it's the right direction.  FAIL FAST!  That's the most important thing here.  Run as small an experiment as you can run that tells you whether you should explore further.  Otherwise, you run the risk of going down a rat-hole.  It won't make you feel good and it's going to extend your time in the program.  In the worst case, you'll quit from discouragement.   So be sure to distill your questions and ideas into the simplest set of experiments and use those results to guide forward.
For the next blog post i'll dive more into the dissertation process and how to close it out.  I'll also discuss what I've learned about diving into the job market and when you should start thinking about what you want to do after you're finished with your PhD.