What are my rights?? offender lives in the Condo Next Door - Now nobody will BUY ours!!!!?

WE bought a townhouse three years ago, there are only two so it is more like a duplex. A few months later a couple (elderly) and their son knocked on our door asking about the other unit for sale. They talked with us a bit and said they werent sure if they, the couple, or thier son was going to move in. They bought the unit and thier son moved in. Since then we have been informed by family and friends he is a offender, who had just gotten out of prison for ually aulting and physically abusing an 11 year old girl for many years as a teacher. We were never informed of this, but most people know his name, we are young, mid 20's with two boys and a little girl on the way. We have never been informed of this, and they give us constant complaints about our cars and parking etc. Well, recently we decided to put the house up for sale by owner. Everyone who comes here to look at it, asks who our neighbor is, we tell them and they run. We can't sell the place, and it isn't fair. The couple who bought it knew he was going to be living there, and never told us to this day he is a offender who had just been released. He lives in paranoia, I can't find a babysitter that will watch my kids, because when someone pulls in the driveway, he runs to the windows and stares out at them. Our mailboxes have been smashed dozens of times, I get eggs thrown at my house. This is our home, we paid alot for and now we are suffering and can't get out! What are our rights? We feel cheated, and his parents protect him by making us go through them when we have issues, and I want to move now. My children and I live in the same building as a offender, and I feel like we are never going to be able to sell this place!