I didn't get any work done

 So, I wanted to implement a pipeline with MLFlow today, but before I knew it, I was googling and got into reading research papers and watching lectures. 

This paper is great https://www.nature.com/articles/s41467-018-07471-9 There are several good computational ideas here. I liked the portion where they made saliency maps by averaging convnet activations first within the layers (and resizing the feature maps) and then across the layers. The reason I'm stymied by this task of classification is that I'm overwhelmed. There are so many convnet layers and I don't know what to do with them. There are also a lot of neurons. I am afraid of fitting a large model and getting some number back. I feel like I need to understand the data first. 

For my career it would be nice to experiment with a bunch of deep nets. Algonauts has voxel partitions into place, words, visual pathways etc. In principle I could work within this partitions. Null spaces are cool. PLSR is cool. Spatial priority maps are cool. PCA. Gaze prediction... I am overwhelmed. It's only been 2 days since I started working on this. 

Figure from Medium(https://ravivaishnav20.medium.com/visualizing-feature-maps-using-pytorch-12a48cd1e573)

So look. For the first pass it would be cool to investigate these averaged feature maps. Not over the whole convnet, but within layers. Of course, PCA is more powerful than averaging, but in the example notebook they did it over all the neurons and images and I find that difficult to interpret. These spatial salience maps seem much more intelligible. 

Okay so now we have activation maps. Essentially we have a new image space, except we have more data for each presented image. Can we try to do regression on this new image space? Different dl models will differ in their saliency maps. These saliency maps are an embedding. The authors didn't do it this way in the end. They used PCA. Both on convnets and on fmri voxels.  

I need to think about this stuff. Watching this https://www.youtube.com/watch?v=jobQmEJpbhY&ab_channel=CognitiveComputationalNeuroscience