Why is the typical name for a chauffeur James, as in "Home, James"? Where is this from?

"Home, James" has been an expression for a very long time, but it's difficult to find its precise origin. One source, Oxford's English Dictionary, places it in 1927:

"1927 E. WALLACE Mixer viii. 114 'All right, Paul,' returned Mr. Sparkes... 'Home, James.' 'James' grinned in the darkness, and the car moved forward. 1934 F. HILLEPAND (song title) Home James, and don't spare the horses. 1964 WODEHOUSE Frozen Assets xi. 213 Okay, Watson, drive on. Home, James, and don't spare the horses."

This passage can be found repeated here:
http://fanac.org/fanzines/IGOTS/igots24.htm

Others state that it has been in existence since the 1840s:

http://books.google.com/books?id=VOI29tQ2QF0C&pg=PA108&lpg=PA108&dq=%22home+james%22+etymology+and+%22don+t%22+spare+the+horses&source=web&ots=mPnV_uKN4D&sig=UsakQOwMeRsQGXXB_l8py1f0GS4#PPA108,M1

It seems to have found a large part of its popularity from the 1934 song by Fred Hillebrand. http://www.phrases.org.uk/bulletin_board/60/messages/355.html