Skip to content

Designing Research Projects

I never assign projects—I don't think it works effectively.

We should invest more time designing and killing projects, but avoid premature termination. Early-stage ideas are fragile and often hard to fully recognize.

Key principle: Difficulty and importance are uncorrelated. Strive for projects that are both easy (that is, have an angle of attack) and impactful. Often 10x more impact isn't 10x harder.

The Project Management Paradox

Designing projects is difficult because we must make the most important decisions when we have the least knowledge—at the project's start.

In knowledge work... the task is not given; it has to be determined. "What are the expected results from this work?" is the key question in making knowledge workers productive. And it is a question that demands risky decisions. There is usually no right answer; there are choices instead.

Peter Drucker

Cognitive Challenges

When dealing with complex problems, several cognitive reactions impair our judgment:

  • Emergency reaction reduces our ability to step back and assess objectively
  • Irrationality increases failure potential through strong influence of context and background knowledge

Due to these cognitive biases and the project management paradox, we must actively set aside sufficient time to think about what problems to work on and how to address them.

Project Selection Factors

When choosing projects, consider multiple factors:

  • Lab's research interests
  • Potential knowledge gain and question importance
  • Task difficulty
  • Our experience level
  • Available time

Different projects suit different career stages. For PhD students, the first project should be a "second master's thesis"—clearly scoped work leading to publishable results quickly.

Consider this framework from How to Choose a Good Research Problem:

After this foundation, longer-term projects become more suitable.

Research Velocity

For many research questions in our field, getting rapid signals proves valuable. Build small prototypes (simpler/smaller datasets, smaller/simpler models) to quickly gain intuition about direction and identify the most important variables. Look at this slide deck for ideas about research velocity.

Remember: Ideas are cheap, evidence is expensive. Science is the art of the soluble—can you test the idea?

Project Lifecycle

Not everything needs to become a full project immediately, but once something becomes a real commitment, it should have an owner, a brief, a milestone, and a review date.

The main idea is: explore cheaply, commit explicitly, review regularly, and sunset things deliberately when needed.

Explore

A question, hunch, reading path, or small technical tryout. This can stay informal. No paperwork needed.

Incubate

Something promising enough for 1–4 weeks of more deliberate exploration. This should have a short note with:

  • Title
  • Owner
  • Question in one sentence
  • Why it might matter
  • 2–4 week test
  • Next decision date

Committed / Active

Once we invest more seriously, the project should have a brief with:

  • Title
  • Owner
  • Question in one sentence
  • Why it matters
  • Why we are well-positioned to do it
  • 3-month milestone
  • Key bottleneck
  • What would make us pivot or stop
  • Next decision date

Maintenance / Sunset

For tools, benchmarks, datasets, or paused work, we should be explicit about who owns them and whether they are actively maintained.

Review Questions

Reviews happen in pod sessions and quarterly portfolio reviews. The point of reviews is not paperwork. The point is to make sure our effort stays thoughtful, visible, and directed.

For incubating work

  • What is the question in one sentence?
  • Why are we well-positioned to work on it?
  • What would success look like in 3 months?
  • What evidence in the next month would make us pivot or stop?
  • What is the real bottleneck?
  • If this works, does it create understanding, adoption, infrastructure, or just another paper-shaped output?

For active work

  • What changed since the last review?
  • Are we still solving the right problem?
  • What is the current narrowest bottleneck?
  • What is the next decision, not just the next task?
  • Is this becoming more distinctive or more generic?

For tools and benchmarks

  • Is this still a research project, or is it now a maintenance product?
  • Who maintains it?
  • What usage or adoption signal matters?
  • What would make us sunset it?

Additional Resources