Users Keep Threads "On Topic" Manager
 

This hack allows members to keep threads on topic by tagging posts. A post can be tagged as "starting to go off topic" & "off topic". If the "starting to go off topic" threshold is met the post gets a light red background as a visual warning. If the post hits the "off topic" threshold the thread background becomes red. Users can also set in their preferences to ignore this modification or use the modification and even ignore posts that go "off topic" making those posts disappear from view. Threshold levels can be adjustable (possibly even by the user). I am requesting this because I have a large forum and many annoyed users stating mods are not doing so good keeping threads on topic. This would be a new level of user controlled moderation to help forum moderators keep topics on track.

Sounds like, with a little template tweaking, NuRatings would do what you're looking for.