We're working with 2 branches and performing builds on each one of them. Today we've noticed that in the "Associated changests" list we see changsets from both branches. A little investigation of the "Microsoft.TeamFoundation.Build.targets" file we found that you can override a property named "LabelQueryScope" to make the build process only changesets from a specific branch. So in this case if you have a project with 2 sub branches than you need to set the "LabelQueryScope" property with "@$/$(TeamProject)/Branch".
By changing this property you also change the labeled branch of the build.