In the first case of its kind ever in Europe, the five-year-old animal, named Vince, was found dead in Thoiry, west of the French capital.
See also: Injured baby rhino rescued after shooting
See also: Poachers kill one of Africa's oldest elephants
According to police the zoo raid happened late last night, and it was not until 9am today that zookeepers made the horrifying discovery.
A source speaking to Le Parisien newspaper said "the animal had been shot with three bullets to the head, and his horn was cut off with a chainsaw".
Vince the Rhino that was killed for his horn
The Rhinoceros Enclosure at Thoiry zoo is situated at the back of the hugely popular African Animal Park Reserve.
Police believe that one or more killers forced a door open at the back of the zoo, before entering the building where three rhinoceros lived under cover.
A source said: "It is possible that the thieves did not have time to attack other animals."
Park director Thierry Duguet said Vince was one of the most popular attractions at the zoo.
He said the murderous attack was "unbelievable".
He added: "Our fellow European park managers are astonished."
Poachers broke in and used a chainsaw to rip the horn off
Police and soldiers examining the scene were deeply moved, and angry at what had happened, added Mr Duguet.
Rhinoceros horns, which are said to be an aphrodisiac, can fetch up to £180,000 on the black market.
There is a well known smuggling route between France and China, but until now most of the horns have come from poaching gangs working in Africa.
It is not only the first time that a horn has been stolen in Europe, but also the first time that an animal has been slaughtered in a French zoo.
This rhinoceros had lived peacefully in Thoiry since November 2015.
There are around 21,000 white rhinos in the world, many still living in the wild, mainly in South Africa and Uganda, but the species is hugely under threat from poaching.
The white rhino is particularly vulnerable, because it is generally an unaggressive animal with very poor eye-sight.