I've been meaning to post about this issue myself, but have not found the energy to do a proper visual aid yet. I was planning on simple indicators in the corner of the unit icons to mark which unit was from which building, inspired by unit insignia found on real military uniforms, just a shield with a number on it to indicate the building, or no number to indicate unattached units.
I don't have nearly as many pages of units as the above posters, but I still consider this the single worst design flaw in the game. The units on the army management screen are sorted based on the order they were built, which is completely useless, and for most players the order might as well be random. Even worse, all units are listed newest first and units of specific types are listed oldest first, adding ridiculous inconsistency to the whole mess. On top of everything else, oldest first is possibly the worst way the units could be organized, since it results in the least used and unattached units being first, and the most used units which die and are rebuilt frequently being last.
I would welcome any improvements to the current army mismanagement screen. Even if we can't get indicators for unattached units or which building the unit is from, the bare minimum should be that the units are sorted in a sensible way. On the all units tab, units should be sorted by age, latest age first, earliest age last, and within each age by unit type, and finally unattached units last within each type. On the tabs for specific unit types, units should likewise be sorted first by age, then unattached units last.
This is of course not the only instance where the developers have used an odd way of sorting things. On the blueprint inventory screen, the great buildings are sorted by age, but the ages are in alphabetical rather than sequential order. It would be ridiculous if it were not such an embarrassment.
It seems like when you move units in or out there is no hard and fast rule as to where they end up on the list.