The Song minister Jia Sidao made a secret approach to Kublai to propose terms and asked whether the Song paid an annual tribute of 200,000 taels of silver and 200,000 bolts of silk, in exchange for the Mongols agreeing that the Yangtze should be the frontier between the states. Kublai first declined but reached a peace agreement with Jia Sidao and returned north to the Mongolian plains because he learned in a message from his wife that Ariq Boke had been raising troops.

He soon received news that his younger brother Ariq Boke had held a kurultai at the Mongolian imperial capital of Karakorum and was pronounced Great Khan by Mongke's old officials. Most of Genghis Khan's descendants favored Ariq Boke as Great Khan; however, his two brothers Kublai and Hulegu were in opposition. Kublai's Chinese staff encouraged him to ascend the throne, and virtually all the senior princes in North China and Manchuria supported his candidacy. Upon returning to his own territories, Kublai summoned a kurultai of his own. Only a small number of the royal family supported Kublai's claims to the title, though the small number of attendees, included representatives of all the Borjigin lines except that of Jochi, still proclaimed him Great Khan, on April 15, 1260, despite his younger brother Ariq Boke's apparently legal claim.

This subsequently led to warfare between Kublai and his younger brother Ariq Boke, which resulted in the eventual destruction of the Mongolian capital at Karakorum. In Shaanxi and Sichuan, Mongke's army supported Ariq Boke. Kublai dispatched Lian Xixian to Shaanxi and Sichuan where they executed Ariq Boke's civil administrator Liu Taiping and won over several wavering generals. To secure his southern front, Kublai did try for a diplomatic solution by sending envoys to Hangzhou, but Jia broke his promise and arrested them. Kublai sent Abishqa as new khan to the Chagatai Khanate. Ariq Boke captured Abishqa, two other princes and 100 men and had his own man, Alghu, crowned khan of Chagatai's territory. Then came the first armed clash between Ariq Boke and Kublai. Ariq Boke was lost and his commander Alamdar was killed at the battle. In revenge, Ariq Boke had Abishqa executed. Kublai closed the food supply to Karakorum with the support of his cousin Khadan, son of Ogedei Khan. Karakorum fell quickly to Kublai's large army, but in 1261 Ariq Boke temporarily took it again after Kublai's departure. During the war with Ariq Böke, Yizhou governor Li Tan revolted against Mongol rule in February 1262. Hearing this, Kublai ordered his Chancellor Shi Tianze and Shi Shu to take the offense against Li Tan. These two armies crushed Li Tan's revolt in a few months and Li Tan was executed. Execution was also the fate of Wang Wentong, who was the father-in-law of Li Tan and had been appointed the Chief Administrator of the Zhongshusheng, "Department of Central Governing") early in Kublai's reign and became one of the most trusted Han Chinese officials of Kublai. This incident instilled in him a strong distrust of ethnic Hans. After he became emperor, Kublai began to ban the titles of and tithes to Han Chinese warlords.

The Chagatayid Khan Alghu declared his allegiance to Kublai Khan and defeated a punitive expedition sent by Ariq Boke against him in 1262. Ilkhan Hulegu also sided with Kublai and criticized Ariq Boke. Ariq Boke surrendered to Kublai at Xanadu on August 21, 1264. The rulers of western khanates acknowledged the reality of Kublai’s victory and rule in Mongolia. When Kublai summoned them to organize another kurultai, Alghu Khan demanded security for his illegal position from Kublai in return. Despite tensions between them, both Hulegu and Berke, khan of the Ulus of Jochi (Golden Horde), accepted Kublai’s invitation at first. However, they soon declined to attend the new kurultai. Although, Kublai pardoned his younger brother, he executed Ariq Boke's chief supporters.