Why can’t we say dans la justice instead of en justice?

Dans is used for being "in a place physically". En can be used for abstract things like in this case "justice". So using dans will be incorrect

Anish, how about 'les femmes dans le besoin'? here dans is not used for being in a place physically.

Eldra: Great question! I know it's been a year since you asked, but I want to give it a try. 

The use of "en" which is used in "Je crois en la justice..." is determined by the verb "croire".  "Croire en qn or qch" ie. "Croire en Dieu" 

Larousse gives as one of the definitions of "besoin" as:

"État qui résulte de la privation du nécessaire ; indigence, dénuement : C'est dans le besoin qu'on connaît le véritable ami."

I agree with you that the state of deprivation of life's necessities is not a physical place, so why "dans"? The only answer I can come up with is that it is perhaps an idiomatic expression.

Maybe Celine or another native speaker can speak more clearly to this question.

I'm nowhere near qualified to answer, but it my head it kind of makes sense as an idiom... because, if we imagine les femmes as what they are, people, animate people, they are in a "place" of need. So, dans le besoin is an image of being in that space/place, even if it's not a literal place, it's a figurative one.