SO fiddle

Test Fiddle mainly for SO Questions

HTML

<link rel="stylesheet" href="http://ajax.googleapis.com/ajax/libs/jqueryui/1.8/themes/dot-luv/jquery-ui.css">
Blocking using keypress: <input type="text" id="in1"><br/>
Removing using keyup: <input type="text" id="in2"><br/><br/>

Tip: hold the comma key down to really notice the difference.

JavaScript

$("#in1").keypress(function (evt) {
    if (String.fromCharCode(evt.which) == ",")
        return false;
});

$("#in2").keyup(function (evt) {
    this.value = this.value.replace(/,/g, "");
});