Christopher W. Fletcher

[Graduate School] First Year Experiences

Created: November 3, 2011

As of starting to write this post, I am officially one year and one month into grad school. I figured that people might be interested in a "first impressions" type of post, so I have tried to distill the most important lessons learned in the past year. I am a computer systems student, so the stories I use are related to computer systems. That said, I have tried to make them as generally applicable as possible.

Lessons and advice

Broadly speaking, I made a lot of mistakes in my first year. You may read and nod to the lessons below. Unfortunately, I think that these lessons can't be learned unless you hit your head against them once or twice in the field. In the off chance that you believe (and take home) what I have to say, however, this post might just save you some time!

Move through ideas quickly

The absolute most important lesson I learned as a first year was to move through ideas quickly. Do not get attached to your ideas. Most of them will fail. This is a surprisingly deep issue that I think takes a deal of maturity to come to grips with.

Depending on who you are, it may be easier or harder to come up with ideas. In both cases, though, coming up with ideas is a lot of fun and morale boosting. Letting ideas go is hardly ever fun. For some people, it just means that a path you were taking has run dry. Others might blame themselves. "Am I not able to find a solution to this problem because I am just not clever enough?" In both cases, if you are not careful, you might think that by dropping an idea you have wasted your time on it.

Dropping an idea should be thought of as a stepping stone to the next idea. (That is how I look at it now, anyway). Every idea will teach you something. You'll read papers related to it, or perhaps build an experiment to test it out. At the very least you will bout it out on a whiteboard for a while. From my own experience, when I first get a new idea, I am really excited about it and am most productive in the time that follows as I sketch it out. If it starts to run dry, I start working slower and slower. Trying to figure out when to stop and move onto the next idea will help you keep at your most productive working levels.

So how to do you determine when an idea is running dry? My adviser passed on this great way of thinking about ideas, and it goes like this: Take an idea X. What is the best and worst outcome for idea X on a low to high scale? ("Low" means bad result and "high" means great result.) If the best and worst outcome are both on the high end, you have a great idea and all you have to do is run with it. If both are on the low end, drop the idea. If the best outcome is on the high end and the worst outcome is on the low end, you don't have enough information and need to do experiments/etc to close the gap.

In all cases, talk to your peers about your research. One situation that you want to avoid is backing off of an idea because you just aren't thinking of the solution. The best way to see if this is happening is to sanity check yourself with different people. I've found that once I have been beating up some tree long enough, I fail to see simple solutions that I would have seen quickly if I was looking at the problem fresh. Avoid this by talking with people, or by putting the idea on the shelf for a while and working on something else.

Paper reading

Reading technical papers is a central activity for any grad student. As you grow as a researcher, your cumulative number-of-read-papers increases, so you become more aware of your field at large. When you are a first year, however, you are a blank slate. When you are starting out, how can you expect to cover enough of the field to find whether someone has already done work on idea X that you might have? Here are some pointers:

  • People are smart. Assume your idea has been done until you can reason about why it hasn't. If your idea is too intuitive, it most certainly has been done before. Count on it. This may just say something about some of my ideas, but I was surprised. Aside: to people in computer systems, this is especially true and for a particular reason. Our field is very sensitive to technology. If you have some great idea that is based on a technology change, count on either a.) it being done before, b.) it being in the works by someone else.

  • Don't read papers using brute force. Try to find the key (highly cited) papers related to your question, and read the abstracts of papers that cite the highly cited papers. To find the highly cited papers (drum roll please...), first talk to people who you work with and use the internet to find keywords related to your topic. Why keywords? If you describe or search for your work, you will probably not find it directly, as you will have probably invented new terms for old-hat techniques, etc. A keyword/buzzword is a way to relate your idea to what is already in the literature. Once you have found a keyword, the second step is to look for some paper that is related to the keyword. When you find a paper related to your keyword, read its cites. If the paper is good research, it will almost certainly cite a central/highly-cited paper. As soon as you have found the highly cited paper, you are off to the races. Your task now is to look over all of the papers that cited the central paper. I've recently converted to google scholar for tracking forward cites. Google scholar has really matured in the past year and gives pretty good coverage (definitely better than any one of ACM, IEEE, or even citeseer). To pass over each forward citation, just read the abstract. If it doesn't seem relevant, skip it. Otherwise, read more. At the end of this process, you should feel much better about paper coverage.

  • Iterate between reading papers and doing actual work. There are two sides to this. The first side is: don't read papers all day every day and get no "real work" done. In computer systems especially, you will likely get demoralized when you realize that every idea has been done before :-). In all seriousness, if you blindly read papers, it will be harder to retain what you have read, and you probably won't get much insight beyond how to extend those papers in the ways that they suggest. The second side is: read papers! If you don't read papers, you will have obvious problems. So what is a good middle ground? What I have found works best is take a literature pass at consistent intervals. My routine is to first do work for five days or so. That work will have hopefully bred a few new ideas which usually poke at a new frontier in the research. For every new frontier, count on reading between five and ten papers. This can vary dramatically depending on the frontier, but that has been my experience on average. Remember, reading papers is not just about checking to see if your idea has been taken! It is also so that you can use other peoples' ideas (you cite them, of course) to get a better working result.

  • When reading a new paper, constantly ask yourself: is my work different from this work? If you have don't have a good sense of this, you will think a paper is exactly the same as your work when it is really not. Why is this bad? It causes you to get demoralized and possibly halt your project when it really is worthwhile research. Developing good intuition about what sets your work apart really comes with experience. As you learn more about your field you learn about details that make X fundamentally different from Y. So, my advice here is to work hard early on to nail down those details: what in your field makes X not equal to Y?

  • Don't get glued to the idea of working on a completely novel, grandiose project. The point here is similar to the last bullet: sometimes you can tell that your project is different from something you read in a paper, but you still stop working on your project. Why? Because if anyone has ever written a paper related to your work, your work isn't "new" enough, and is therefore not worth spending time on. This is dangerous thinking and can lead to you getting nothing done. My advice is to start modest. The more you dig into what seems to be an incremental project, the more you will learn. The more you learn, the more position you will be in to make a larger advance. In my experience, it is very difficult to come up with a "larger advance" unless you dig into a smaller one first.

Good vs. bad questions

Research is all about asking questions. It behooves you to frame your questions in a way that will spur your creativity. This may sound fluffy, but it really has made a difference for me. Here are two questions I asked recently:

  • What are the implications of changing X in a computer system? This question is blind (but hopefully informed) exploration. When you change a single thing in a computer system, more things will probably have to change and at the end of the day you will get a different looking system. This is a good question because you will probably learn things about the system that you hadn't considered before. This is a bad question because it is sometimes hard to sell the results of your work. In the end, you need to publish your findings and convince people that they should care about changing X. Unless there is a big benefit in changing X, the final system that you get may not spark peoples' interests. This type of question is also dangerous because by constraining yourself to change X, you may be tying your hands behind your back. Make sure to keep track of cool things "Y" that change in the system because of changing X. If Y is cooler than X, is there an easier way to get to Y than by changing X? Don't go in obsessed with your original idea X. Be flexible and adapt as you dig deeper.

  • How do I build a computer system of type Y that minimizes power consumption? (To those outside of the field, minimizing power consumption is a big deal.) This is a more directed question where, if we get a good result, we know exactly how to sell it. In some ways, this is the opposite of the previous question in that you are working backwards. You start with your sales pitch and just have to fill in how to do it. This is a good question because it is easier to sell and is less likely to bottleneck your creativity because you are not constraining yourself to "change X" as was the case with the first question. This is a bad question because it doesn't really get you anywhere. There is no insight or trick. It merely points you in some direction.

Given the two types of questions, how do you get the best of both worlds? What seems to happen to me is that I start with the first type of question "what if I change X" because it involves some cool trick X. I wallow for a while and eventually realize that "hey, if you change X, there are some opportunities to reduce power." At that point, I change gears and ask the second type of question: "how to minimize power," given the exploration I had already done by changing X." Speaking from some experience, my idea X never ends up panning out. To come full circle on this post, this is the time to drop idea X. In my experience, dropping X is always difficult because it was the start of the whole research direction. What I have come to respect is that the outcome is the exploration that leads to being able to ask "how do I minimize power, given some ticks I have accumulated."

Aside: The insight vs. effort ratio. Another way to judge a good question is to constantly ask yourself: what will it take to answer this question? Will it take some insight, infrastructure, or both? Fortunately, this tends to be less of a problem for people who have done more work in a field. You build infrastructure as you go, and become more knowledgeable in general. So, the amount of effort that you will have to put into getting an answer may decrease. As a researcher, try to maximize the insight that your work puts back into the community, while minimizing effort.