実際上はどちらもあり得ます。
https://www.google.co.jp/search?newwindow=1&safe=off&hl=ja&complete=1&ei=QBV4WrKjNsOT8wWrprBo&q=%22had+realized+that+I+didn%27t%22&oq=%22had+realized+that+I+didn%27t%22&gs_l=psy-ab.12...0.0.1.19335.0.0.0.0.0.0.0.0..0.0....0...1c..64.psy-ab..0.0.0....0.gWvIN2BXNgk
https://www.google.co.jp/search?newwindow=1&safe=off&hl=ja&complete=1&ei=bBV4WunNDIy18QX4g46QAw&q=%22had+realized+that+I+hadn%27t%22&oq=%22had+realized+that+I+hadn%27t%22&gs_l=psy-ab.12...123094.123124.0.126707.3.2.0.0.0.0.187.187.0j1.1.0....0...1c.1.64.psy-ab..2.0.0....0.A8-IaingiHg