The move by the world's largest software maker comes amid a steady decline in demand for PCs as people turn to tablets and other mobile gadgets.
CEO Steve Ballmer said in a memo to employees that the changes mean the company is "rallying behind a single strategy" and organising by function. While it has been widely anticipated, it's too early to tell how well the reorganisation will help Microsoft compete with more nimble rivals like Apple and Google.
"Although we will deliver multiple devices and services to execute and monetise the strategy, the single core strategy will drive us to set shared goals for everything we do. We will see our product line holistically, not as a set of islands," Mr Ballmer wrote.
The company's new divisions include engineering, marketing and business development. Microsoft named veteran executive Julie Larson-Green head of its devices and studios engineering group, overseeing hardware development, games, music and entertainment.
Mr Ballmer stressed the company's focus on "one Microsoft" in his memo. He said Microsoft will move forward operating as a cohesive company rather than a "collection of divisional strategies."
Janney Capital Markets analyst Yun Kim said the reorganisation helps align Microsoft's various divisions around its devices and services strategy, but added that he's taking a "wait and see" approach.
"We continue to look for signs on how the company can leverage its success in the Xbox business to re-energise its current efforts in the tablet and smartphone markets," the analyst wrote in a note to investors.
Microsoft shares rose 67 cents to 35.37dollars in early afternoon trading on Thursday. The Redmond, Washington-based company's stock is up 32% since the start of the year, compared with a 17% increase of the Standard & Poor's 500 index.