I think that you are doing the right thing by not making too big a deal of the times he's clumsy. He is hard enough on himself. It is great that you have him in Judo. Martial arts definitely increase a child's self-esteem. Make sure that when you talk about him to other adults, that he is not within ear shot. Overhearing his parents talking about his left-handed clumsiness and that his 4 year old bro is more co-ordinated would shatter his self-esteem. I'm not saying that you do, but it's a common thing for all parents to be guilty of on occasion!
You may think I am nutty, but I recommend putting him in ballet classes. Ballet focuses on strength, coordination, balance, and flexibility. If he balks and says that the other kids are going to tease him, then tell him that many football players, boxers, and other macho athletes have ballet training.
Good luck. I am left-handed too, so I feel for your kid!