Big Opens to Explore

[jonfroehlich]

Just wanted to jot down some thoughts here so I don't forget:

• Cross-city training. We began exploring this in our 2019 ASSETS paper but this is like the big open question: can we take a model trained on prior city data and characterize/classify a city's accessibility? If not, how much per-city training do we need before the model begins to perform reasonably?

• Performance as a function of training data. I'd like to see some graphs that show how our models perform as a function of training set size (a standard experiment in ML papers). I'd also like to see us get to ~1 million validated labels in our training set size.

• Using the tag data for predictions. Do we have enough tag data to begin training models to classify not just the label type but also the tag or condition. Perhaps we could start with a subset of tags here (e.g., poles, garbage cans, bikes, etc.

• Integration of model into Project Sidewalk. As a start, I could see us visualizing the model's prediction on a validation card or prioritizing validations (e.g., prioritize gallery cards based on predicted accuracies + manually validated data)

[michaelduan8]

From our meeting, also want to jot down the possibility of exploring severity prediction alongside tag data/prediction.