The first of those two occasions came last year in Cardiff. Madrid finished second in their group behind Borussia Dortmund, before facing Napoli in the last 16, and beat them with an aggregate score of 6-2.
The other occasion was in the 1999-2000 season. There, Los Blancos finish as runners up in the second group stage behind Bayern Munich. Back then, the Champions League had two group stages and Madrid won their first group, but came second in the next one. They then defeated Manchester United 3-2 on aggregate in the quarter final and eventually beat Valencia in the final.
Their other four victories in the Champions League, since it was played in the current format, came after topping their group.
So it's unlikely that Real Madrid will be too worried about finishing second or first in their group.