We theoretically already have Child Protective Services to take children away in that case. Child abuse includes "[failing to] provide adequate food, clothing, shelter, medical care or supervision."

On the other hand, there's the darwinist approach...