You could chalk it up to him holding the hair pinned against his arm. One thing you could do to get the hair more gravity friendly, Use 2 hair props. Prop one would be parented like you have here. Keep the bandana visible. Prop 2, Hide the bandana and rotate it to looks like gravity has taken it.
I am with Star here. Really a terrific piece although the hair removes some of it's reality. In future, you might consider making hair material invisible and adding it in later in postwork in such a situation.
You're right my friend. I tried in every way to move the hair, but they are all one with the bandana, and if moved around too much hair, the bandana is deformed.