My guess is that many people who spent time around Epstein were probably not pedophiles. Maybe even the majority of them. His "day job" was a guy who knew all the powerful people and could introduce anybody to anybody else. If he had been a woman, we would have called him a socialite.
Some of the people were probably in his inner circle and participated in the child abuse. But, others were probably just using him as a way to meet royalty, or as a way to connect to VC money.
But, you know what? I'm ok if plutocrats who weren't actually pedophiles get caught up as collateral damage.
I imagine most of them didn't know that child sex abuse was going on. Even among the ultra rich, that's not seen as acceptable behaviour. Epstein was probably careful about who he let into the inner circle. They were probably fully vetted, and/or Epstein had compromat on them so they couldn't expose him without exposing themselves. The rest of them probably thought he liked women who were barely legal, but still legal.