How can I protect myself from noise complaints when I live in an 2nd floor apartment?

I am really frustrated. We moved into our apartment about three months ago and about two months ago a new neighbor moved in down stairs. He compains constantly about our noise level and today two of our apartment office staff came to my apt and asked me to keep it down. We have a two year old and a 6 year old and it is summer so there is no school. The second day that this guy moved in he beat on his ceiling while my son was throwing a tantrum (about 530pm). Our apt rules our a 1000pm quite time and my kids are in bed my 7pm. I realize that kids are loud but I can't control every noise that they make (ie tantrums). Today I was vacuuming and of course kids freak with vacuums and run away from them. They had just finished dinner and right before that they had been playing video games (the youngest was watching my oldest play). About 5 minutes after I started vaccuuming the apt people came and beat on my door. So in a at least a two hour time frame we had been virtually silent and the second we make any noise this guy calls the office on us. I have talked to him and he even stated himself that the noise is always between 1000am and 7pm. He also said he was trying to sleep and that was why the noise was bothering him. I don't know what to do. Im sorry that he wants to sleep at 5 in the afternoon but most of America does not get to have that schedule. I also can't keep my kids silent ALL day. He plays his music and has friends over on the weekend and I have never complained about him. Its an apt and people make noise. I expect it and don't make a big deal about it. I am afraid that the apts will eventually try and kick us out. THey said they have a three strikes rule and they have already been by twice. They even encouraged me to think about if we really needed a garage and if we might think about a first floor apt. Yes I want the garage and does that even matter? I don't want to move again. I told them to put some one with kids underneath us but they said they can't make him move... wow. as Im writing this my kids are in bed (its 715) and he has started up with some god awful loud music. I don't want to start a "war" with my neighbor but I am wondering if they can kick us out? We dont' make noise after curfew and even then if the police were ever called, we wouldn't be issued a citation because its kids and not a tv or radio or party or anything... What are the laws about this sort of thing?