Log in / Register
Home arrow Computer Science arrow Social Computing, Behavioral-Cultural Modeling and Prediction
< Prev   CONTENTS   Next >

3 Team Communications Data and Observer-Based Leaders

The verbal transcripts were generated in a prior research study [19] in which 12 threeperson ad hoc teams produced a solution to a military logistics task. The teams were randomly assigned to a face-to-face or a virtual (audio Skype) condition. The participants were undergraduate students specializing in homeland security and intelligence analysis and thus accustomed to analytical tasks. Each team had 90 minutes to create a written plan to transport military troops and cargo to a specified location, choosing the best combination of routes and vehicles, while minimizing cost and maximizing security. A hidden profile task was employed in which each participant had unique information which defined their roles as Fuel (e.g., available routes and fuel), Capability (e.g., carrying capacity of vehicles), or Security (e.g., access to intelligence reports). The sessions were audioand video-taped and transcripts were generated by a research assistant from the audiotapes, reviewing the videotapes as necessary.

Two observers [EP and the research assistant] independently identified team leaders for all twelve teams by reviewing the transcript data using the manual indicators cited above (first to speak, dominance, praise, etc.). In cases in which the leader changed over time, the individual identified as the leader at the end of the session was used. For every session, one leader was uniquely identified, even when there was suggestive evidence of power-sharing between two team members, in order to facilitate a direct comparison with the automated approach. The two investigators initially agreed on the leader of 11 of 12 teams. Resolution on the leader of the remaining team was easily achieved via discussion.

4 Automated Coding of Team Leaders

This study builds upon prior research [2] investigating dialogue acts related to teamwork effectiveness, e.g., whether team members were making a correction to previous information, anticipating one another's information needs, confirming if information was correct, or indicating their uncertainty about some information. In that study, a multiple regression on these DAs accurately predicted observer-based measures of team performance. The categories and features were then updated to be applied to email from a large Army Command and Control exercise [26]. The current 29 categories represent more standard dialogue acts (question, acknowledgment, positive reply, negative reply), as well as categories related to command and control, such as intentions (command, alert, proposal), internal state or attitude (certainty, uncertainty), and process (reasoning, factual statement, clarification). The goal of these categories is to usefully classify all the linguistic features of communications that do not refer to the actual tasks being conducted. To capture that content—what is being talked about—a statistical topic analysis [3] is applied, which typically removes textual features using a stop word list. The dialogue acts defined here can then be used to assess process—how topics are being discussed—making use of many of the stop words excluded from topic analysis. To do this in practice with new data sets, such as the one here, all the unique utterances from the experiment are tagged with the current gazetteer, any untagged words or phrases that are not directly related to the task or content are then manually binned into one of the categories or left as a stop word. This process has been conducted by AD on over 80,000 messages from six data sets [2], [19]–[23].

Each message was analyzed with the subsequent gazetteers for 29 DAs. Each message was given a score for each category which was the proportion of all the message's DAs that were in that category. A participant's score for each category was determined by dividing the sum of their message scores in that category by the total sum of message scores in that category from the entire team. In addition, the proportion of the team's statements spoken by the participant was also calculated as an indicator of conversation dominance (i.e., speaking the most). From the DAs, it was predicted that Certainty, Command, and Proposal would serve as indicators of social dominance (i.e., a higher position in a social hierarchy). Uncertainty was hypothesized to have an inverse relationship (i.e., a higher position in a command hierarchy is associated with less expressed uncertainty). Finally, Reasoning was expected to be an indicator of cognitive dominance (i.e., having the most influence over how decisions are made and on what basis) and expected to be most related to thought leadership. Table 1 provides examples of the most frequent features in these categories present in the transcripts.

Table 1. Dialogue Acts Examined

We hypothesized that these six independent variables from all 36 participants would be related to the binary value of being the leader in a logistic regression. The logistic regression analysis found that the overall model was significant (χ2(6, N=36)

= 17.944, p<0.01), with only Reasoning being individually significant (B = 33.475, p < 0.04). In a stepwise regression starting with Reasoning, only Uncertainty added significantly to the model (χ2 (1, N=36) = 4.6002, p<0.04). With these two factors, the final model was significant (χ2 (2, N=36) = 12.688, p<0.002) with estimates for Reasoning = 30.906, and Uncertainty = -13.519. Note that the proportion of messages was not as significant as the proportion of Reasoning and Uncertainty features spoken. Using these estimates, each individual could be assigned a probability of being the ETL, p(ETL), of their team. The highest scoring team member was the same as the one assigned by observers in 8 of the 12 teams, and was in the top two in 11 of the 12 teams. If p(ETL) can discriminate between leaders and non-leaders, then large differences in the value should clearly determine the leader. To assess this idea, for each team we compared the max p(ETL) score to the second highest. We see in Fig. 1 that large differences (above 0.38) always correctly identify the leader.

Fig. 1. Cumulative probability of determining the correct ETL based on the difference between highest p(ETL) on a team and the second highest

In three of the four discrepant cases (Teams 1, 3, 11, 12) marked with ×, the teams were single gender: two all-female and one all-male. Upon re-examination of the transcripts by EP, two of the three all-female teams (11 and 12) appeared to have more of a power-sharing arrangement between two members as compared to the other teams.

Found a mistake? Please highlight the word and press Shift + Enter  
< Prev   CONTENTS   Next >
Business & Finance
Computer Science
Language & Literature
Political science