I'm in love with a sex offender, but my family doesn't accept him

I started dating a guy I met online. He told me that he registered sex offender but he didn't do anything. He took a plea to always register as sex offender and he did two years. That was more than two decades ago. Nothing on his record since.I always believe the past should be left in past. But my roommates, their family and my son are unforgiving. Should I forgive this man that I'm falling in love with or should I dump him? If I stay with him I will never see my family and granddaughter again.