LTE: Urban, suburban and rural discourse

After I labeled all the data by topic, I took a quick look at the topic tallies, both overall and by newspaper, keeping in mind the newspapers’ locations. The overall percentages are shown in the graph below:

Some of these stats were expected, others surprising. I expected most of the letters to be about politics and government; and that is indeed the case. Overall, we can identify 4 topics that account for the majority of the letters: politics, government, community and society. Additionally, healthcare, business and education are also important. Several topics are so rare that there are less than 5% of all letters under them (e.g., technology, sports and science).

An interesting finding is about the third most dominant topic. If you remember from the scraping web-sites post, the newspapers are from three different types of communities: urban, suburban and rural. I originally classified them as follows: urban (Chicago Tribune, Daily Herald), suburban (The Citizen, Times Union), and rural (Dubois County Press, Enquirer Democrat). However, after I looked at the topics data, I reconsidered my original classification.

Here are some data about the towns and counties where the newspapers are from.

It is clear from the population density data above why I had originally classified the Daily Herald as an urban newspaper and Times Union as a suburban one. The population density in Arlington Heights is much closer to the population density of Chicago, and the population density of Albany is much closer to the population density of Fayetville, GA, which is a suburb of Atlanta. However, Arlington Heights is 22 miles from the center of Chicago, and the Daily Herald calls itself “Suburban Chicago’s Information Source”. Similarly, while the population density of Albany is not very high, it is nonetheless an urban center.

Additionally, I double checked that Dubois County Press and Enquirer Democrat were indeed from rural areas. According to the US Department of Agriculture classification, Dubois county in Indiana is “Urban population of 20,000 or more, not adjacent to a metro area”. Since it is not inside a metro area, and the population is low, we can consider it rural, or much closer to a rural than a suburban or an urban area. Macoupin county in Illinois, where the Enquirer Democrat is from, is classified as “Counties in metro areas of 1 million population or more”, most likely because of its proximity to St. Louis, MO. However, this county is again much closer to a rural area than a suburban one, since its population density is low. Also, the neighboring Montgomery county is classified as “Nonmetro – Urban population of 2,500 to 19,999, adjacent to a metro area”. Clearly, the towns where the newspapers are located, Jasper, IN and Carlinville, IL, are just that, towns, but they are of a very different character than the other four towns/cities we are considering, and the surrounding areas they serve are more rural than the other four areas.

The graph below shows the topic percentages in each of the 6 newspapers. Pay close attention to the distribution of the top three topics and see if you can find a pattern (click to enlarge).

Excluding the two most dominant topics, politics and government, take a look at the community topic. In the urban newspapers, it takes a backseat to society and education, while in the rural newspapers it is either the third most common topic (Dubois County), or the first (Enquirer Democrat). The suburban newspapers follow the same pattern, where the community topic is the third most dominant one after government and politics.

Here is an example of a letter that was tagged with the topic “community”:

A bumpy thrill ride is here on Dunton
It’s Spring once again in Arlington Heights and vacation season is upon us. I encourage all readers of the Daily Herald to visit “Craters of the Moon State Park” on the 300 block of South Dunton Avenue, just south of the downtown business district. Hiking trails into the craters should be opening soon.

The data shows that the letters about local communities are more prominent in suburban and rural areas. While this is a preliminary analysis with just 6 data points, we can still speculate as to why that is. The most obvious reason is that people are more concerned with local issues in suburbs and rural areas. People in cities have less of a feeling of a community. Another possible reason is letter selection. It is possible that newspapers do not select letters pertinent to the local issues as much in the cities as they do in suburban and rural communities, or they do more selection in city newspapers than in the other ones. This would again reflect reader interest in community topics, depending on the area type. In any case, this is an interesting issue worth more investigation with more data points (and lots more tagging!). It would also be instructive to see if there is a continuum from urban to suburban to rural areas with community interest being lowest in cities and highest in rural areas.

The next post will be about preprocessing the collected data for machine learning. Stay tuned!