Is there a historical trend towards shorter sentences?

From my own reading of older books (eg. 18th, 19th century) in various styles (novels, philosophical treatises, scientific publications), it seems that sentences were longer back then.

Is there good hard data on this? Have sentences in fiction shrunk faster than sentences in, say philosophy journals? (I expect so). What explanations can be given for this trend?

I believe the answer is definitely yes.

A quick google search turned up this book result, "The History of the English paragraph," by Edwin Herbert Lewis, where it says:

In view of the now well known fact(1) that the English sentence has decreased in average length at least one half in three hundred years the question arises whether the length of the paragraph has decreased increased or remained stationary.

The citation is:

(1) The fact was definitely demonstrated by Professor L. A. Sherman, in his Analytics of Literature, Boston, 1892.

Another google search turned up the book, and I found a certain Chapter XIX titled "The Literary Sentence-Length In English Prose."

On page 259 he supplies some hard data from various book sources, which I've converted to text here and filled in with full names, book titles, and dates. This shows the average number of words in between periods for the first few hundred periods:

Robert Fabyan, "Chronicle", written 1516-1559
First   hundred periods: 68.28
Second     "       "   : 66.68
Third      "       "   : 56.12
Fourth     "       "   : 65.77
Fifth      "       "   : 58.26
Average:                 63.02


Edmund Spenser, "A View of the Present State of Ireland", written 1590s
First   hundred periods: 49.78
Second     "       "   : 50.24
Third      "       "   : 53.67
Fourth     "       "   : 47.56
Fifth      "       "   : 47.88
Average:                 49.83


Richard Hooker, "Of the Lawes of Ecclesiastical Politie", written 1594-1597
First   hundred periods: 43.98
Second     "       "   : 40.90
Third      "       "   : 37.12
Fourth     "       "   : 41.63
Fifth      "       "   : 43.40
Average:                 41.41


Thomas Babington Macaulay, "Essay on History", written 1828
First   hundred periods: 23.23
Second     "       "   : 21.26
Third      "       "   : 25.95
Fourth     "       "   : 22.20
Fifth      "       "   : 19.65
Average:                 22.46


William Ellery Channing, "Self-Culture", written 1838
First   hundred periods: 25.15
Second     "       "   : 25.51
Third      "       "   : 25.38
Fourth     "       "   : 26.80
Fifth      "       "   : 25.84
Average:                 25.74


Ralph Waldo Emerson, "Address before the Senior Class in Divinity College", written 1838
First   hundred periods: 18.06
Second     "       "   : 20.15
Third      "       "   : 21.01
Fourth     "       "   : 24.18
Fifth      "       "   : 19.52
Average:                 20.58

The time periods are: Fabyan (? - 1513), Spenser (1552-1599), Hooker (1554-1600), Macaulay (source written in 1828), Channing (source written in 1838), and Emerson (1803-1882). To round it off, by my own reckoning, the preface to Sherman's book (1892) has an average of 24.77 words for its 168 sentences. There seems to be a gap of sources in the 1700s, so I wonder if those sentences were around 30-40 words long on average.

He goes on to show that the authors are pretty consistent within their own works, so these numbers are pretty indicative of an author's style. Furthermore, Sherman demonstrates that the number of predicates per sentence has also decreased with time. Unfortunately he does not claim to know the cause of this.

Writing for an internet audience means writing short, easily-consumed sentences that can be rapidly skimmed for content, and which don't force the reader to continue slogging through a wordy explanation when a short bullet point would have sufficed, because reading on a screen is unnatural and tiring, not to mention that contemporary literary tastes tend toward the clear and plebian in lieu of the florid purple prose of yesteryear, the equivalents of whose borderline poetic constructions in modern form tend to read as overly complex at best and run-ons at worst, again, in large part because they don't follow one of the simple rules of modern literature: information is paramount, and how information is presented has become much less important than what it actually is—temporarily disregarding standards of readability and good taste—so the obvious choice is minimalism and succinctness (and the epitome of this is Twitter, which a comment already mentioned) especially because it's significantly easier to write many brief sentences than one long one that retains even a modicum of comprehensibility.

(You have no idea how hard that was to do.)

(TL;DR) The short answer is yes, from an evaluation of US presidents' inaugural addresses and state of the union addresses, by University of Pennsylvania Professor of Linguistics Mark Liberman.

Beware that the following only extracts the summary of his evaluation, and omits the many helpful images in the original source. To improve ease of readability, I eschew blockquotes >.

Real trends in word and sentence length

October 31, 2011 @ 8:34 am · Filed by Mark Liberman under Computational linguistics, Linguistic history

A couple of days ago, The Telegraph quoted an actor and a television producer emitting typically brainless "Kids Today" plaints about how modern modes of communication, especially Twitter, are degrading the English language, so that "the sentence with more than one clause is a problem for us", and "words are getting shortened". I spent a few minutes fact-checking this foolishness, or at least the word-length bit of it — but some readers may have misinterpreted my post as arguing against the view that there are any on-going changes in English prose style.

So I wrote a script to harvest the inaugural addresses and state of the union addresses from the site of the American Presidency Project at UCSB, and some other scripts to (I hope) extract the texts of the speeches from their html wrappings, and to count word and sentence lengths. Why use these sources? Well, different kinds of writing have their own norms, and so it wouldn't be good evidence of an overall historical trend to show (for example) that 20th-century sports reporting is stylistically different from 19th-century sermons, or that 21st-century blogging is different from 18th-century pamphleteering. U.S. Presidential addresses are one accessible example of a body of texts, spanning more than 200 years, which ought to be fairly consistent in genre and register.

The results suggest that mean word lengths have decreased slightly in these addresses over the past century — by 5% or so — while mean sentence lengths have been falling since the founding of the republic, and have undergone a cumulative drop of perhaps 50%.

[...]

There are lots of obvious questions, if you care about things like this — for example, how much of the fall in mean sentence length is due to using less clausal embedding, and how much is due to splicing fewer sentences together paratactically, e.g. with semi-colons?

[...]