The son of Horikawa and Fujiwara no Jishi, Toba ruled from 1107 until 1123, when he abdicated in favor of his son Sutoku. Due to his youth, and the declining fortunes of the Fujiwara regents, the first part of his reign was marked by the increased political involvement of abdicated sovereign Shirakawa. Despite the fact that Shirakawa protected the interests of Toba from the Fujiwara, the two men did not have the same sort of relationship that existed between the In and his son. First, there was the matter of Toba's consort. After much difficulty finding a suitable woman to give to the Tennô, Shirakawa finally decided upon Fujiwara no Shôshi, a woman who was surrounded by "widespread rumor about her conduct" (Hurst, 1976, p 156). She would later give birth to Sutoku, amidst controversy over whether he was the son of Toba, or Shirakawa. Whoever was the father or the prince, Toba was forced to abdicate by the In in favor of the five year old Sutoku. This decision, made by Shirakawa, was most likely an attempt to "increase the honor and glory of the empress Shôshi" (Hurst, 1976, p 157), a favorite of Shirakawa's. The question of succession, and the subsequent existence of a junior and senior abdicated sovereign (which reduced Toba's influence significantly for quite some time) was clearly a point of contention between Toba and his grandfather.
After Toba's abdication, Shirakawa continued his involvement in politics, and, as senior member of the imperial household, held far more influence than the junior In. It was 1129, the year of Shirakawa's death, before Toba was able to exercise the influence that he must have been hoping to enjoy after his abdication. He spent a great deal of time increasing the privatized landholdings of the imperial house, and did not attempt to control such holdings to the extent that Shirakawa did. It is interesting to note that Toba made no effort to "abolish or circumvent the regular channels of imperial govenrment but, like his grandfather, made full use of existing institutions" (Hurst, 1976, p 162). Here again it can be seen that the In did not weild much of the actual power in the government, but instead exercised influence over the true governing body, the Kugyô council, affected a great deal by the Tennô and the Fujiwara household.
Towards the end of Sutoku's reign, the problem of imperial succession again became a problem. Toba, perhaps as a result of the controversy concerning Sutoku's true father, and perhaps because he was originally forced to abdicate in favor of Sutoku, did not have a good relationship with his son. When his favorite consort gave birth to a prince, Toba engineered the accession of this son, who would be later known as Konoe. This action widened the gulf between the In and Sutoku. When Konoe's health began to fail, Toba snubbed Sutoku's son, putting future Tennô Go-Shirakawa on the throne in order to promote the future accession of Bifukumon'in's adopted son, who was fathered by Go-Shirakawa. This final insult was one of the factors that led to the Hôgen disturbance following Toba's death in 1156.