David Luiz punished Simon Mignolet by giving the visitors the lead from a 25 yard free-kick as the Reds keeper was still organising his wall.
The home side responded in the second half when Georginio Wijnaldum, who had earlier been denied by Thibaut Courtois’ diving save, headed home from close range after James Milner had nodded the ball into his path.
Mignolet then made made up for his earlier error by keeping out Costa’s penalty which was awarded after the Spain striker was fouled by Joel Matip.
Liverpool, who trail Chelsea by 10 points, sent on Sadio Mane for the last 15 minutes.
The forward had arrived back in Liverpool less than 24 hours earlier having played for Senegal at the Africa Cup of Nations – but was unable to inspire Jurgen Klopp’s side to victory.
Who was your man of match?