but factory farms don't hurt anybody, do they?

Umm, yeah. According to the Pew Commission (part of the Pew Charitable Trusts), CAFOS (Concentrated Animal Feeding Operations, where most of the animals that are eaten in this country spends their time before they're slaughtered) pose an unacceptable risk to public health and the environment. Read more about it here.

Not that it's really news, but it's nice to have a large, respected organization say it publicly. Oh, and don't think you're avoiding CAFOs because you are eating organic or "free range." Unless you know the farmer and have seen the way the animals you are eating live, assume the worst. You're most likely right.