The crest is already in The Kid's pack at the start of the game.
"An identification badge from the old days.
All those on official business for the City bear the Star of Caelondia.
It is said to represent warmth, labor, and sacrifice."
To Rucks : "Weren't these only meant to be for uniform and identification?"
To Zulf: "What was it like living in the City? Didn't you get homesick?"
- "For Zulf, Caelondia was like a second home. He's real worried about his first home too, far to the east."
According to The Stranger, the City Crest contains a bit of The Bastion's power. As such, the crest serves two functions; it draws some pieces and particles under The Kid's feet, allowing him to explore ruins, and also leads The Kid towards the Cores.