Building Intermediate Dependency Graphs

This post was sparked by a recent exchange I had over Twitter. I’m writing this brief follow up to further clarify this line of thought; something hard to do within Twitter’s length constraints. This post is written in the spirit of a short, hence I won’t be fleshing out ideas in comprehensive detail.

---

@ctabrizi_ There’s some part of me that is unreasonably satisfied seeing additive advantage in practice: e.g. “Building X helps me do Y”, X + Y helps us build Z"

@bhaprayan I wonder if there’s a good way to figure out intermediate steps like these for hard problems. Sort of like a dependency graph.

@bhaprayan It seems easy enough to do, but maybe we reach roadblocks at certain points in the graph (the only way past which are fundamental breakthroughs).

@bhaprayan Assuming the number of breakthroughs if randomly distributed across all fields (we might have to do a weighted sampling over resources e.g. capital + talent, but that seems like a more involved process), we could imagine creating a map of progress.

---

The original thread referred to building a dependency graph of steps to complete in the process of solving hard technological problems.

I’ve seen a related idea¹ in the past that attempted (successfully), to build a dependency graph to guide learners through areas of machine learning. The learning context is simpler since teaching is an iterative process that is tested and improved every year, with new batches of students. Outside of a learning context, it appears the hard part is figuring out what the intermediate steps should be. Once they’ve been generated, figuring out what order to complete them in appears to be a simpler problem².

So how should we figure out what the intermediate steps are? Could a generative model help? The generative model at use in the present is relying on experience to think about what these steps could look like. However, this is suboptimal since we’re not thinking about the possible space of solutions that aren’t related to our past experiences. A more automated approach could be to train a model over Wikipedia³, that can then be queried to find the shortest path between any two states (start state to goal state)⁴.

A further question is whether these steps are subject to change once decided or if they’re fixed targets to execute on? My present view on this would be the former. I think the sequential approach to innovation that Tesla or SpaceX have followed, to solve hard technological problems is admirable. Yet, even for these examples, I wonder which steps were identified at the start and have been fixed throughout, as opposed to ones that changed along the way.

Would explicitly building this graph and releasing it to the broader public result in a faster iteration towards surmounting important scientific and technological challenges? At present maybe we’re just reinventing the wheel in the process of innovating across different sectors (without crosstalk). Perhaps building such a graph could help us better allocate future resources, and create a shared “API” to utilize innovations across fields. TBD.

Footnotes