This is a good topic, and I don't see why not to keep digging deeper. The vb4 board is slower compared to any other version for sure and anyone can test that. The main reason of that is the fact that there are many JavaScripts and CSS files on approximately each page while other versions of vb say vb3 has a lot less than that; at least half of the way.
Now let me be honest with you, I have spent almost 2 years trying to figure out an excellent way to make vb4 works fast yet no complete solution I found. There are many things you need to compromise from vb4 features so that it works faster.
1- reduce used features that are based Js files and inline blocks such as javascripts codes on main pages.
2- combine all css styles in one; this is something VB Co should have done themselves.
3- changing all static contents to Data URI instead of images to reduce the page load. however, the page size will increase yet that doesn't matter if you use some cache software or plugin.
4- Using CDN is a great solution for many things. if you have your own VPS or dedicated server, then CDN is great to save a lot of bandwidth. also it can protect your static content and site as well from DDOS injections either through static contents, SSL 'if you have one" and database management system such as MySql or Mariadb
I believe some people could do better solution if they have excellent programming skills using either .Net or Java.
my advise to do it yourself if you know what you are doing. if not, then hire someone skillful programmer to do it for you, but that would cost you arms and legs.
Cheers