/blog   ·   /Scitech   ·   /Shorts

Building Intermediate Dependency Graphs

This post was sparked by a recent exchange I had over Twitter. I’m writing this brief follow up to further clarify this line of thought; something hard to do within Twitter’s length constraints. This post is written in the spirit of a short, hence I won’t be fleshing out ideas in comprehensive detail.

@ctabrizi_ There’s some part of me that is unreasonably satisfied seeing additive advantage in practice: e.g. “Building X helps me do Y”, X + Y helps us build Z”

@bhaprayan I wonder if there’s a good way to figure out intermediate steps like these for hard problems. Sort of like a dependency graph.

@bhaprayan It seems easy enough to do, but maybe we reach roadblocks at certain points in the graph (the only way past which are fundamental breakthroughs).

@bhaprayan Assuming the number of breakthroughs if randomly distributed across all fields (we might have to do a weighted sampling over resources e.g. capital + talent, but that seems like a more involved process), we could imagine creating a map of progress.

Rapidly Exploring Random Trees. Image credit: Wikipedia

The original thread referred to building a dependency graph of steps to complete in the process of solving hard technological problems.

I’ve seen a related idea1 in the past that attempted (successfully), to build a dependency graph to guide learners through areas of machine learning. The learning context is simpler since teaching is an iterative process that is tested and improved every year, with new batches of students. Outside of a learning context, it appears the hard part is figuring out what the intermediate steps should be. Once they’ve been generated, figuring out what order to complete them in appears to be a simpler problem2.

So how should we figure out what the intermediate steps are? Could a generative model help? The generative model at use in the present is relying on experience to think about what these steps could look like. However, this is suboptimal since we’re not thinking about the possible space of solutions that aren’t related to our past experiences. A more automated approach could be to train a model over Wikipedia3, that can then be queried to find the shortest path between any two states (start state to goal state)4.

A further question is whether these steps are subject to change once decided or if they’re fixed targets to execute on? My present view on this would be the former. I think the sequential approach to innovation that Tesla or SpaceX have followed, to solve hard technological problems is admirable. Yet, even for these examples, I wonder which steps were identified at the start and have been fixed throughout, as opposed to ones that changed along the way.

Would explicitly building this graph and releasing it to the broader public result in a faster iteration towards surmounting important scientific and technological challenges? At present maybe we’re just reinventing the wheel in the process of innovating across different sectors (without crosstalk). Perhaps building such a graph could help us better allocate future resources, and create a shared “API” to utilize innovations across fields. TBD.


  1. Metacademy claims “All the information is already out there…Every time you encounter a concept you don’t recognize, you need to go back and learn it first. Pretty soon you’re deep in dependency hell, switching between twenty tabs, trying to juggle all the prerequisites of prerequisites, wondering if any of this will help you towards your original goal. It would be so much easier if you had a guide who could tell you exactly what you had to learn, why it’s important, and where to learn it.” [return]
  2. Suggests connections to using some variant of topological sorting, since this is something that package managers use anyway. However, the “packages” to be “installed” here would be breakthroughs to hard engineering (and science) problems. [return]
  3. Just a rough approximation at this point for the totality of human knowledge. [return]
  4. This makes an important assumption that future breakthroughs are generated only through currently available knowledge. It’s appeared to hold up so far? The process of creativity however remains a mystery. [return]

Written November 10, 2020. Send feedback to @bhaprayan.

← Amorphous Text, A Socratic Exploration