Should I let my boyfriend go?

I'm an early 20 something year-old virgin, and my boyfriend is in his early 20s with kids. He's cheated on me twice: once in the beginning and then four months ago. He wasn't use to dating a virgin and broke up with me three times because he wasn't sure. We both work at the same job (where we met) and it's with the girl that works with us. I've been trying to get over it, but now my confidence and self-esteem are low, and sometimes I feel like he'll do it again or that he's not ready to be with me. I think about it all over again when I see her. He says he's in love with me, and with the last time he cheated, it didn't feel right, and that's when he knew he had to stop because his feelings were for me. He's contacted her and told her he will not communicate with her because he's in a relationship with me. How do I gain confidence? Should I let him go? If I let him go, I don't want to be sad about it being that we work together. I try to tell myself I'm beautiful every day, and I think I am a pretty attractive female with a great personality, but I don't like the fact that she works there.