There is a very interesting article on the Canadian cricket website, about the under-development of cricket in the Americas.

It documents the first overseas tour in 1859, as All-England took on the Americans.

…the tour concluded with the notorious 'Frosty Match' played at Rochester, NY, on October 21, 24, and 25 between Eleven of England and Twenty-two of the United States and Canada. The home side was wrecked by the round-arm bowling of John Wisden (of the Cricket Almanack). But dismissal appeared to come as relief to the half frozen batsmen. "Shiver my timbers, I'm out!" was the relieved cry as they made a dash for the warmth of the pavilion. The home side was beaten, early on the third day, by an innings and 70 runs, but in fact, it was cricket that was defeated on the fields of Rochester.

Cricket simply failed to attract a broader audience that day. A PR disaster; cricket beaten by bad weather. It may have been the first such instance, but it wasn’t the last.

The article goes on to place partial responsibility for the death of cricket, on those that attempted to preserve it, specifically those in Philadelphia:

Indeed, I sometimes think that it was the snobbishness of Philadelphia which was largely responsible for branding cricket as a 'stuck-up' game in popular American perception

Compared to the everyman’s game of baseball, cricket didn’t have a chance. It was seen as exclusive. Which is, in itself, part of the contradictory nature of cricket: its charm lies in its antiquated and rather archaic rules that baffle outsiders. Cricket stops for tea, plays with a straight bat, and have you ever tried to explain a forward defensive to someone who knows nothing about the game? Cricket feeds on these snobberies, and appeals to those who are dissatisfied with American culture.

In another possible world cricket could have survived in America. But imagine the game if the yanks got hold of it. It would be a vulgar spectacle, with unnecessary American rules with silly titles like “power plays”. Games would last no more than twenty overs, and awful music would blare out when boundaries are scored. Ghastly.

Thankfully, this never happened, and cricket remains the better game. Conversely, baseball did catch on over here. Only we call in “rounders” and it is played by children.