And yet another drawback is how the S&P 500 (as the index most invest in) is far less diversified that you'd think.

Sure, it includes 500 companies (a handful with more than a single share class), but the top 6 stocks constitute over 25% of the index.