Ask how many lives he has saved. Press this subject, and the mob turns on him, dispersing as the preacher shrinks under your verbal barrage or tell him you'll shut him up, not being a man of combat, the priest will shy away, and the crowd disperses, mocking the priest.
There is also option to ignore him and walk away. The preachers are as common as syphilis in Crippled Kate's, and just as difficult to get rid of. He will latter call a Witch Hunters for that dishonor and you will have to deal with them.