Bing Toolbar is one of the most removed programs using Browser Cleaner. Browser Cleaner makes removing the Bing Toolbar easy. Start by finding all instances of Bing in Browser Cleaner (you can get Browser Cleaner here) and click to remove. Once you have removed Bing, Browser Cleaner will give you the option to run your control panel. Look in your control panel to see if there were any other unwanted programs installed on your computer.
Bing probably changed your homepage when you installed it. If you need to change your homepage settings you can read about that here.