Youtube videos are served from Youtube, not from your server. So deleting those won't help.
Probably the best thing to do is to enable browser caching for your site in .htaccess. This MIGHT work for you, but it depends on how your server is configured.
Code:
<FilesMatch "\.(jpg|jpeg|png|gif|ico|js|css)$">
ExpiresDefault "access plus 1 month"
</FilesMatch>
The other thing to look for is if some IP address is constantly hitting your server and ban that (those) ip addresses if you can.