Slaven Bilic's men created chances to score further goals in difficult conditions at The Den but battled against an improved second-half performance from Millwall to take full advantage of second-placed Leeds United's 2-0 defeat at Nottingham Forest on Sunday (AEDT).
Matheus Pereira hit the base of the post in the fourth minute and then drew a good save from Bartosz Bialkowski with a dipping drive from outside the box moments before West Brom took the lead with 42 minutes played.
Krovinovic received the ball 25 yards from goal and beat Mahlon Romeo with his first touch before burying an effort into the corner of the net from the edge of the penalty area.
Hal Robson-Kanu should have doubled the visitors' lead on the cusp of half-time but was denied by a timely block by Jake Cooper, and the Millwall defender was on hand to again deter the Wales international at point-blank range three minutes after the restart.
Callum Robinson curled narrowly wide moments later, but the second finally arrived as O'Shea lost his marker to head in a Kamil Grosicki corner at the near post and seal victory six minutes from time.