My son is 3 – I haven’t measured his chest recently, but he also has a 21 inch waist. The 3T school bus tee is fitting him nicely now – plenty of ease but not baggy, and the right length.
If you were to size down in width, I’d still keep the length of the 3 or 4 – I often add a little extra length when I make this shirt, because my son almost always outgrows his shirts length-wise (or head/neckline wise!) before they’re too tight.
I would not make the 18-24 month size for a four-year-old, though – I think the armscye would be uncomfortably tight, and she’d struggle to get the neck hole over her head. I think the smallest I’d go is a 2T, with some extra length. But if this were my kid, I think I’d go with 3T with the length of 4T, and at worst the shirt will be roomy.