[snakes1100]

If your the only one that logs into the server via ssh or ftp or thru a host panel, then you can simply block all access to those ports and lock them down to your IP, simply host.deny them or set up some rules via iptables.

If this is still going on after all this time, it may be time for you to wipe and redo your OS on your puter as well, if there is still and exploit in your system, he very may well be using your own puter to do the accessing of your server.