The move is an attempt to shift its focus to higher-margin businesses like cloud computing and artificial intelligence.
A new company focusing on legacy IT infrastructure will be named and spun off next year.
IBM shares closed nearly 6% higher after the announcement.
It marks the latest shift by the world’s first big computing firm to diversify away from its traditional businesses.
“We divested networking back in the 1990s, we divested PCs back in the 2000s, we divested semiconductors about five years ago because all of them didn’t necessarily play into the integrated value proposition,” Chief Executive Arvind Krishna said.
Mr Krishna was the key architect behind IBM’s $34bn (£26bn) acquisition of cloud company Red Hat last year.
Currently, Amazon Web Services and Microsoft dominate the market for cloud services.
“To drive growth, our strategy must be rooted in the reality of the world we live in and the future our clients strive to build. Today, hybrid cloud and AI are swiftly becoming the locus of commerce, transactions, and over time, of computing itself,” Mr Krishna wrote in a blog post.
IBM, which currently has more than 352,000 workers, said it expects the separation to cost $5bn.