Since when has freedom of speech been a right? It certainly isn't in the UK. You can't say what you want, nor does society want people to say what they want. Can you imagine if we had loads of right-wing fanatics spouting out racist comments, paedophiles talking about children and murderers talking about their next victims?
Freedom of speech is a myth, as ultimately you are not free to say things that are morally wrong. And what is morally wrong, is defined in the law. This Mr. Wilders is a racist, and being racist is illegal in the UK. Why would we want to let him in

?
Mind you, I think it's hypocritical considering other people we
do let in the country, but that's another story.