Roma got off to a blistering start with former PSG player Pastore scoring minutes into his first match at the Stadio Olimpico with a spectacular back-heel after a pinpoint Cengiz Under cross.
"It's certainly harder in Serie A than Ligue 1, the first two games have been such a struggle," said Argentine Pastore.
"I think after our goal, we got a bit too relaxed and allowed them to have the ball.
"We came out for the second half with more intensity and managed to get it back on track. We could even have won it at the end."
Atalanta -- revamped ahead of their crunch Europa League playoff game this week -- proved dangerous,taking advantage of sloppy Roman defending to pull ahead 3-1 at half-time.
Belgian defender Timothy Castagne tapped in the equaliser after 19 minutes and within three minutes Argentine Rigoni got his first goal with the second on 38 minutes.
Roma coach Eusebio Di Francesco shuffled his side at the break with World Cup winner Steven Nzonzi coming on for his debut after arriving from Sevilla, along with Justin Kluivert.
Alessandro Florenzi pulled a goal back on the hour mark before limping off injured with Patrik Schick thrown into the attack in his place.
But it was Greek defender Kostas Manolas who pulled the hosts level seven minutes from time, tapping in a Pastore free kick.
Roma, who finished third last season, have four points from two games and face a difficult task on Friday at AC Milan, who are reeling after throwing away a two-goal lead to fall 3-2 to Napoli.
"This leaves us with a very bitter taste in the mouth, we went very close to winning," said Atalanta coach Gian Piero Gasperini, whose side are also on four points after winning their opening match.
Champions Juventus' 2-0 win against Lazio and Napoli's comeback against AC Milan meant both start their campaigns with the maximum six points.
Highlights Javier Pastore Roma Serie A Atalanta