I never played soccer as a kid and when I watch a game I am often baffled about when the ball is in or out of bounds. When exactly is the ball out of bounds in soccer?

This is a common misunderstanding of the rules, especially for the soccer novice. Unlike basketball and football, in soccer, the ball is not out just because it touches the line (either the end line or the touchline/sideline). Nor is a goal scored just because the ball touches the goal line.
    The rule in soccer is this:  The entire ball must cross the entire line in order for it to be out of bounds or score a goal. This holds true even if a player is skilled enough to "bend" the kick, or get it to turn in the air. If the ball is kicked and the entire ball crosses the entire end line or touchline/sideline, even while in the air, it is out of bounds even if the ball bends back inbounds. And for goal scoring, if the entire ball crosses the entire goal line, it is a goal, even if the ball subsequently bounces back out of the goal into the field of play.