Psalm 93 – Skeptic's Annotated Bible answered

According to the reference of the author of the SAB, this quote from Calvin is taken from his commentary on Genesis. However, it is not found there. It's just made up (see also The Foundations of Dialogue in Science and Religion (page 16) by Alister E. McGrath). With a source like that, there's little need to spend more time on Andrew Dickson White.
But it is clear that Calvin believed in a geocentric universe from his commentary on this particular verse:

How could the earth hang suspended in the air were it not upheld by God’s hand? By what means could it maintain itself unmoved, while the heavens above are in constant rapid motion, did not its Divine Maker fix and establish it?

But it's unfair to accuse Calvin of not accepting an idea the scientific consus had not adopted until long after as David Wootton writes in "The Invention of Science":

For the first hundred years after the publication of Copernicus's book in 1543 only a fairly limited number of specialists were familiar with the details of his arguments, which were only generally accepted in the second half of the seventeenth century.

But the issue here isn't what Calvin thought, but what the text actually says. It is not probable to read this exclamation as a scientific description. The intend of the psalmist is to compare land and sea with God's might. In verse one (verse 1) the psalmist mentions the earth: it doesn't move, i.e. the ground doesn't shake, it is firm. This is contrasted in verse two (verse 2) with God's throne: that is everlasting.
That in particular the ground, the shore, the dry land is meant in verse 1 (verse 1) can be seen by the second contrast the psalmist makes in verse three (verse 3). Unlike land, the sea does move. Waves can be high and in great turmoil, appear unstoppable. But God is mightier (verse 4).
The psalmist concludes with God's testimonies: the scriptures are sure. We can trust them. The whole psalm is very similar to chapter 19 where also nature is described in the first part, and God's kingdom in the second.
Note that John Gill takes this entire psalm as a description of the future kingdom of God, of the world to come.