So I would like to know if there is an easier way to find the culprit.
I really don't want to turn off all my mods one by one to see which is causing the performance drops. This leads me to believe that I'm hitting some kind of bottleneck (possibly the hard drive - the game is installed on my HDD, not enough space on the SSD) when some of the HD textures from my mods get loaded or something similar. Also, I don't have permanent low fps, it's more like microstutters every few seconds or so. I've watched the performance monitor closely while playing my memory is at about 40%, CPU at 20% utilization. But even though I can play a heavily modded regular Skyrim without problems, I'm now noticing some performance drops while outside.
So I've just installed the Skyrim Special Edition and customized it with about 100 mods using Nexus Mod Manager.