Mods allowing specific sets of ads for specific forums
 

Apparently, this is a rarity: a banner mod where selected ads are displayed only in selected subforums.

phpadsnew integration: well, apparently it's possible with some hacking of the code, but I think there needs to be a different template for each forum. There's also a lot of criticism of phpadsnew: too bloated, too difficult to set up and configure, and so on.

vB Ad Management: no longer supported. Also, it doesn't allow specific ads for specific forums; a number of people asked for the feature, though.

Geek Advertising Banner: apparently there's nothing available on whether it can do this or not.

Does such a mod exist?

To be honest with the introduction of default ad locations in 3.7 I wouldn't bother with most of the ad mods that currently exist. Unless of course you need them in different locations to the default locations.

Within the templates you can use conditionals to decide which ad code to display. As an example on my forum I have the following:

Logged Out/Guest Users
Text or images header banner
Text or images banner after first post (x 2)

Logged In
Text only header banner
Small text only banner after first post

Subscriber
No header
No first post ad

All of this is controlled via conditionals, and could easily be extended to perform checks on forum id's. For example this is a summary of my code for the header banner:

A verbose description of this is, if the user is not in usergroups 25,26,6,7,5 and the current page is not my CMPS page or my searchresults (custom) page. Then check if the user is logged in, if yes then display the "LOGGED IN BANNER MARKUP" otherwise display "GUEST BANNER MARKUP"

You could extended the conditional statements to include checks on forum ID.