In-form forward Willian opened the scoring for the west Londoners but they were pegged back by Romelu Lukaku and then Jesse Lingard struck the winner after coming off the bench.
The Blues are now fifth after 28 games played, and Conte reckons they face a tough task finishing in the top four.
The Italian said, as quoted by BBC Sport: "There is this risk. We must be ready to be concentrated, and to understand this target is an important target for us.
"In this league, getting a place in the Champions League is not simple. We are disappointed with the final result. We had a great chance to get three points against a competitor for a place in the Champions League, but in the end we are talking about another defeat.
"This country has the only championship where you can find six teams who start the season by fighting to win the title or getting a place in the Champions League."
The former Juventus boss has a job on his hands to make a successful finish to the season at Stamford Bridge, with the FA Cup and Champions League their only remaining sources of possible silverware.