Challenges...very briefly: Keeping "on schedule," because it's way easy to take days off here and there and suddenly you haven't done a lesson in six months. (Ok, not really that bad, but if FEELS like it!)
Finding a program that works for your kiddo can be hard; you need to understand HOW your child learns. Some kids learn best by HEARING things and watching other people do them. Those kids generally do GREAT in regular school, because the system is set up that way and their natural learning style fits right in. Other kids to best when they get to work with a concept hands-on for a few hours at a time, or they only seem to remember something if it's connected to a physical action. Those learners do quite badly at school, because they're required to sit quietly and passively and take in information without touching and experiencing it. Your idea of geography (for younger kids) might start out with you pointing out places on the map, much like a teacher in front of the class. Ultimately, you may end up putting pieces of tape on an inflatable beach ball printed up like a globe, and tossing it to your kid, asking them to identify which continent is under the tape.
For a teenager, you might start with a Science Biology textbook, but find out that doesn't work, and end up with a kid who is working as a lab assistant to a grad student, taking a frog census at the local nature preserve or getting up at 2 a.m. to turn over the turtle eggs in the lab's incubator.
Other than that, putting up with your friends, neighbors and family howling about how you'll ruin their lives is often a pain in the butt. For some people, getting out of the school mode of thought is nearly impossible.