They're probably getting tastier food staying in there.

[His voice is barely above a mumble, and he squints in the bright daylight as he looks up at a sign post. He hasn't been to the zoo very often. Excursions were still few when Shinji was very small, and Byakuya may as well have been invisible all through his own childhood. He doesn't know his way around, and he's a little nervous that they'll get lost and end up in trouble.]

Why risk getting killed when you can hang around in a comfortable zoo?