Mapping Online Publics

About a month ago I introduced my new Gawk script metrify.awk, which generates a wide range of Twitter metrics for a given Twapperkeeper/yourTwapperkeeper hashtag or keyword archive. Even as I was writing those posts, though – and certainly while playing with the language metrics I discussed in my last post –, I started to find a few areas where metrify could provide even more information on the dataset. So, the time has come for a first service release which upgrades metrify.awk to add some more functionality (and fix a few inconsistencies along the way). This is a revision rather than a full rewrite of the script, so let’s call it metrify 1.2; it’s now available for download here, where it replaces the older version.

As before, the new version of metrify.awk is called as follows:

gawk -F , -f metrify.awk time=”[year|month|day|hour|minute]” [divisions=x,y,z,…] [skipusers=1] input.csv >metrics.csv

(divisions defaults to ‘90,99’ – i.e. a 90%/9%/1% split of the userbase – if it is not specified).

Read the rest of this entry →

OK, this may be a somewhat esoteric subject for researchers who mainly work with Twitter data from specific countries and cultures, but over the past few weeks I’ve been working on a paper that analyses Twitter activities in the #egypt and #libya hashtags – and as part of that work, I’ve been interested in exploring the interactions between users tweeting in Arabic and users tweeting in other languages (mainly in English). Unfortunately, there’s no reliable means of identifying the language of specific tweets, or of the users who post them; while the Twitter API provides an ISO language code (e.g. ‘en’ for English, ‘no’ for Norwegian, etc.) for each tweet, this is drawn simply from the overall language setting of the user’s account, and not specific to each individual tweet itself. For users who alternate between languages in their tweeting, all tweets will be tagged with their chosen language code; for users who haven’t bothered to change their Twitter profile settings away from the default English, all their tweets will be tagged ‘en’, regardless of their actual language.

Read the rest of this entry →

It’s difficult to believe that one year ago, significant parts of Brisbane were inundated by floodwaters; thankfully, there has been no repeat of the flood crisis this year. One of the few good news stories to emerge from the disaster was the – overall, very successful – way in which social media such as Twitter and Facebook were used during the event, both by key emergency authorities and by everyday users, from directly affected local residents to onlookers further afield.

Particular kudos in this must go to the Queensland Police Service Media Unit, which – not quite from a standing start, but certainly without much time to prepare a comprehensive strategy for its social media crisis communication approaches – delivered timely, informative, and level-headed updates on the flood crisis as it unfolded. Its Facebook followers grew, literally overnight, by a factor of ten, and @QPSMedia also became the single most visible account participating in the #qldfloods Twitter hashtag.

Read the rest of this entry →

We’ve got a few busy years ahead of us, it seems. In addition to the ARC Linkage project on social media and crisis communication which was awarded to us (the QUT Mapping Online Publics team along with our CCI colleague Kate Crawford, the Queensland Department of Community Safety, and the Eidos Institute), which we’ll carry out during 2012-14, we’ve also had word in December that another project application has been successful.

Titled “The Impact of Social Media on Agenda-Setting in Election Campaigns:
Cross-Media and Cross-National Comparisons”, that project will study the use of social media in a series of election campaigns which are coming up over the next few years (2012-15) – including the Queensland state election and the US presidential election this year (and I’m tempted to throw in the French presidential election as well, just for fun), and elections in Sweden, Norway, and Australia which are coming up in 2013 and 2014.

Read the rest of this entry →

Those of you who have followed our adventures in Twitter research for some time now will know that we’ve relied to a significant extent on Joe John O’Brien III’s excellent Twapperkeeper as a tool for capturing tweets. Twapperkeeper (as a stand-alone, free Web-based service) no longer exists in its original form, however – though some of its functionality for creating Twitter archives appears to have been subsumed into the for-pay services available as premium offerings from Hootsuite – and so we’ve been getting the occasional inquiry about what to do now.

Read the rest of this entry →

Update: revision 1.2 of metrify.awk is now available (still at the link below), and introduces some further functionality, which is outlined here.

This is the final instalment of my four-part introduction to the metrify.awk script for generating detailed metrics for specific Twapperkeeper/yourTwapperkeeper hashtag archives. Over the last couple of posts, we’ve mainly dealt with overall stats for the hashtag, as well as for specific, definable percentiles of more or less active users. Finally, now, it’s time to look more closely at patterns within the overall userbase.

Read the rest of this entry →

Update: revision 1.2 of metrify.awk is now available (still at the link below), and introduces some further functionality, which is outlined here.

Over the past couple of posts, I’ve introduced our new metrify.awk Twitter metrics script, and looked at the first of the three metrics tables produced by the script. Let’s move on now to the second table, where I’ll use a snapshot of Australian political discussion on Twitter under the #auspol hashtag between February and August 2011, instead of #qldfloods – the overall metrics for the different user percentiles in the #qldfloods dataset turn out not to be particularly interesting… As before, we’re dividing the total userbase according to the 1/9/90 rule into the 1% of most active users, the next 9% of moderately active users, and the final 90% of least active users. (In the case of #auspol, that first percentile contains 142, the second percentile contains 1291, and the final percentile contains 12700 of a total of 14133 users.)

Read the rest of this entry →

Update: I’ve clarified/corrected some of the details relating to the percentile metrics contained in the first table which metrify.awk generates.

Update 2: revision 1.2 of metrify.awk adds further functionality in addition to what is described below. These changes are detailed here.

In the previous post, I’ve introduced metrify.awk, our new multi-purpose tool for generating Twitter metrics. Over the next instalments in this series of posts, I’ll take you through the results it produces. And seeing as we’re coming up to the anniversary of the January 2011 south-east Queensland floods, and as I needed to generate those metrics anyway, for a report on social media in the floods which we’re publishing soon, I’ll be using an archive of #qldfloods tweets between 10 and 17 January 2011 as an example here.

I’m running metrify.awk as follows for this:

gawk -F , -f metrify.awk divisions=90,99 time=day qldfloods.csv >qldfloods-metrics.csv

In other words, we’re using a 1/9/90 division of users, and we’re tracking activities per day; the skipusers switch is not set, so full stats for all users will be generated.

Read the rest of this entry →

So, 2011 is finally over – and what a year it’s been. While the confluence of natural disasters, political crises, and other major events has also provided us with the basis for a new research programme in crisis communication, let’s hope that 2012 is a little less intense, please…

To start the new year on a positive note, I’m finally getting around to sharing some more information about the new approach to generating Twitter metrics which we’ve developed over the past few months – this actually started during the research workshops we had with Stefan Stieglitz’s group at the University of Münster in August, so it’s taken some time to gestate into its present form. What it’s now turned into is quite a powerful tool for generating detailed information about a specific Twitter dataset – intended mainly for the study of hashtags, but with applications well beyond this as well. Amongst other things, it enables us to distinguish more effectively between different groups of participating users (from highly active lead users to much less active casual participants), and to track different types of participation, in total or by these specific groups, over time.

Read the rest of this entry →

As 2011 winds down (which may also give me the time to do some more Gawk coding again – watch out for more updates soon), we’re still in the process of harvesting the results of our work over the last twelve months. Over the past few weeks, a clutch of articles based on our Mapping Online Publics research have finally seen the light of day:

• I’ve converted my exploration of dynamic @reply network mapping methods (detailed in a series of blog posts starting here) into an article in Information, Communication & Society: “How Long Is a Tweet? Mapping Dynamic Conversation Networks on Twitter Using Gawk and Gephi.” In the article as well as in the original blog posts, I’m using the 2010 Rudd/Gillard Australian Labor Party leadership spill – #spill on Twitter – as a case study.
   

Read the rest of this entry →