What should I do with my relationship with my son's father?

Im in a situation where me and my son's father have been together off and on, mostly on, for almost ten years and he has 3 kids before my son! I have loved him since I first met him but we went through our ups and downs in the past,we split for like 2 years from 2000 to 2002. He was in relationship with a girl for 3 years before we met and we had been talking towards the end of there relationship. The same day they broke up we got together. Well 2 months into our relationship she tells him she's 2months pregnant and she's keeping it. I tried to stay with him but I couldn't take seeing her walk around carrying this mans baby who loved so much, so I ended our relationship and moved. Then in 2002, we met up again and exchanged numbers and started where we left off. But then he sits me down and tell me during the time we were apart, him and his ex had a kid on the way. He let me know they wasn't together, so I said I wasn't gonna run this time and stick by his side. I've accepted and now love his kids like they were my own. We've had few conversations about kids and I wanted some of my own but he does not want to have anymore. Well, in feb 2008 I found out I was pregnant and I told him I was keeping it. Now that my son is here, our relationship has gone downward! I feel like he don't feel the same about me anymore and its saddens me because I want to be with him and to have my son grow up with 2 parents in the house but when I try to talk to him to see how he feels about me, he say he still love me and wants to be with me.We argue all the time, he don't spend time with me or my son like he should, I do all the physical work when it come to my son, we stop communicating like we use to, I have to keep reminding him when our son need things and have to wait for it. He always have his other 2 sons on the weekend and never offer to get my son too and have to spend time with them at his mothers house because his son's mother don't want them at my house, he hang out ALL the time and im left with all the responsibilities as parent. All she have to do is pick the phone and say their sons needs something and he there but my son has to wait. He feels like the only thing he has to do is go to work and provide for his family. Well, I work also and still come home and take care of my son and everything else. Im so tired of dealing with this and my son is only 16 months. I love him so much but I don't know what to do. Sorry for the long story but Im have so much bothering me and I need advice.