Kiran yes there is an initial cost to the environment to begin implementing large scale solar, but energy-wise there comes a point when all power needed for further expansion can be entirely from the solar energy of earlier implementation. Concerning environment, the main materials for solar panels is silicon, which is just sand, all recyclable, nothing toxic. There are no rare elements needed.
On efficiency, panels do not need to be particularly efficient, if enough area is available to just add more panels to make up for losses.
Also no batteries are needed during build-up, flywheel technology requiring no exotic materials can be used for local night time energy storage.
There really is no technical argument against worldwide solar, it is the only way out of the pickle we are in.