In addition to English and Italian versions of the Rubrica Valutativa della Competenzia Interculturale (see HubICL Tool #467), this toolkit contains a reflection logbook for students, guidelines for a student's capstone presentation about the intercultural learning experience, and a teacher checklist (rubric) for evaluating the presentation. All items available in either English or Italian.

Created by the Transcultural Pedagogical Research Group, a team of Purdue doctoral students who have all gone on to tenure-track glory, this coding system, which includes examples, stands as a shining way to make sense of qualitative data as revealed in students' efforts to become better writers while working together across cultural differences.

This comprehensive qualitative toolkit includes a focus group protocol for measuring student satisfaction with a virtual exchange program (also often known as Collaborative Online Intercultural Learning, or COIL) as well as probing their learning outcomes and behavioral change. It also includes a question protocol for understanding one's collaborative partner's perspective(s) and a checklist for observation by an outside observer of a learning activity. Most of these tools would be very easily adapted for use with non-virtual or blended intercultural exchange programs.

Some researchers feel that a rubric is not, strictly speaking a qualitative research method. Rather, using a rubric is framework analysis, which is a quantified way of rapidly making sense of lots of textual or qualitative data. Others call it "an inherently comparative form of thematic analysis which employs an organized structure of inductively- and deductively-derived themes (i.e., a framework) to conduct cross-sectional analysis" --in other words, a qualitative methodology. Knowing about framework analysis may become important when you are submitting your research article to a journal that tends to favor quantitative analysis methods. (Look it up in Google Scholar or Web of Science for the latest on this debate.)

In the meantime, the above link will lead you to a collection of the rubrics curated into the HubICL.

When one is unfamiliar with qualitative research or when one is trained primarily in quantitative methods, qualitative methodologies may seem fuzzy, obscure and/or biased. Because of a societal focus on numerical or mega-data, it can also often be hard to make the case to upper administration that gathering and analyzing qualitative data is important to institutional research.  Below are links to a few articles that may assist the qualitative neophyte or the institutional researcher wanting to highlight (for example) student voices.

How is validity and reliability realized in qualitative research? (U. of Miami School of Education)

Myths and misconceptions about using qualitative methods in assessment (Harper & Kuh, 2007)

The use of scoring rubrics: validity, reliability and educational consequences (Jonsson & Svingby, 2007)

Is qualitative research more inclusive?  (Santorini, APA Trends Report, 2023)

The Inclusion Competencies Inventory is a proprietary, online, research-validated survey instrument which measures Openness to Change, Adaptability, Connecting with Others, Reading Others, Valuing Different Perspectives and Power Sensitivity.

The Inclusion Competencies Inventory is a proprietary, online, research-validated survey instrument which measures Openness to Change, Adaptability, Connecting with Others, Reading Others, Valuing Different Perspectives and Power Sensitivity.

This handy down-loadable reference sheet lists all of the HubICL assessment tools alphabetically by type (survey, qualitative method, rubric, etc), as well as indicating whether the instrument in question is: (a) cost-free, proprietary, or behind a paywall, (b) suitable for learners under 18, and/or (c) available in languages other than English.

This activity, created by Dr. Dan Jones, CILMAR, is based on the chapter by Toni Schmader, William Hall, and Alyssa Croft, “Stereotype threat in intergroup relations”. This activity will help participants recognize the mechanisms that cause negative impacts of stereotyping. This activity explores the ways to combat negative performance by identifying and removing stereotype threats. This activity and handout are especially beneficial to instructors and program leaders in addressing issues of academic performance among marginalized and minority students.

Scenery, Machinery, People asks learners to analyze who in their lives they categorize as scenery to be observed or ignored, who is machinery to be used, and who they actually allow to be the people. Learners also analyze to whom they themselves might be scenery, machinery, or people. After this analysis, we discuss the energy that must be expended to let someone move from scenery to machinery and from machinery to people. It’s really much easier to leave people in the category that you originally put them in. For example, the person who takes your money at a fast food place is just a machine until you ask them how their day is going. Only then do they begin to move from being machinery toward being a person. But as a participant once told me, “If I wouldn’t give you a kidney, then I don’t have the energy to let you be in the People category.”

Likewise, if you are used to seeing students who are different than you in some way as only the scenery, it is easy to leave them there, to other them, and not to ever really get to know them. Only when we exert the energy to change our sorting mechanism to default to “different than me is more interesting than same as me” will we begin to see progress in our students—and in ourselves.

“Six differences” asks participants to find someone in the room who is different than themselves in six ways which aren’t appearance-based. “Six differences” is a great way to partner people up for “25 Questions.”

 “Different Similarities” offers polarized students the opportunity to see how they are similar to someone that they thought was much different, and it also gives students who minimize difference the opportunity to find out just how different they might be than others. 

“25 Questions” gives practice for domestic and international students to ask one another interesting questions that they might not think of on their own.

In the activity entitled “Living in a Bubble,” learners analyze the places that they regularly go for sameness and difference and discuss the pros and cons of experiencing heterogeneous and homogeneous communities.

A similar but different tool which increases self-awareness about our own personal networks is entitled Auditing Your Personal Networks. In this activity, participants sort their contacts based on feelings of intimacy—how close they feel to people in different zones. Groups include personal relationships in the middle, social relationships in the blue circle, and the public in the outermost circle. Participants put actual names in each circle and then talk about how they could draw more people into their circles.

Both the Bubble and Auditing tools begin with a reading of a 2019 article from The Atlantic by Green entitled “These are the Americans who live in a bubble." The Auditing tool also includes a reading of an article by Kos entitled “Relationship circles—the most important diagram of your life.” The sources for these are included in the HubICL Toolbox.

Another activity that gets at this idea of how we sort people is called Critical Mass. This particular activity builds off of a reading in Claude Steele’s Whistling Vivaldi, entitled “The Strength of Stereotype Threat: The Role of Cues.” Steele begins the chapter by talking about how it felt for Sandra Day O’Connor to be the only woman on the Supreme Court. There was a little less stereotype threat when Ruth Bader Ginsberg (RBG) was added, but there were still a lot of comments by reporters that mentioned “one of two women on the court.”  It wasn’t until there were three women on the court  that it began to feel like women had reached critical mass and that RBG, Sonia Sotomayor, and Elena Kagan could speak on behalf of their own opinion and not on behalf of all women.

After reading the chapter on critical mass and stereotype threat, we ask participants to look at their own college’s webpages and analyze who is represented, who is missing, who might feel excluded, whether the pages challenge or reinforce stereotype, etc.

This tool comes with a complete lesson plan for talking through each of the videos and applying the Sapir-Whorf Hypothesis. In the particular context of the class in which we talked about sorting, this could have made a nice follow up to talk about how each of their languages reinforces the way that they categorize objects, ideas, and even people. I especially like this particular activity because it asks viewers to either watch the movie Arrival or to watch one of the many YouTube videos about Arrival, along with a popular TED talk by Lera Boroditsky. 

Kris Acheson-Clair has created another variation of Language Envelopes, which she calls Language Coding, that asks learners to sort sentences in a similar way to sorting objects. We offer both a hard copy of this for you to use with a group face-to-face and a ready-to-use jamboard for your use with virtual groups.

If you were to visit my office, I have 30 envelopes each containing the items that you see on the screen. Depending on how many students are in the room, I number them off in such a way that there are two 1’s, two 2’s, two 3’s, two 4’s, etc. in the room, and no one originally sitting side by side has the same number. The two people who have number 1 sit facing one another with the envelope of objects between them. The two people who have number 2 sit facing one another with the envelope of objects between them. The two people who have number 3 sit facing one another with the envelope of objects between them, etc. until the entire room is paired off with someone that they were not sitting by when they came in the room.

Silently, one member of the pair empties out the envelope and sorts the objects while the other person in the pair observes. When all is sorted, the observer guesses the sorting logic that was used, and the sorter acknowledges whether the observer is correct in their guess. Then the roles reverse—the sorter becomes the observer, and the observer becomes the sorter.  Taking turns, each player should get to sort 3 times and observe each time, each time using a different sorting logic.

Let me give you an example of a couple of ways that I’ve seen this sorted. If I were to put the pencil, the nail, the screw, the coffee stirrer, the stick, the toothpick, and the Q-tip into a pile together, you might guess that I had sorted those objects by what qualities? If I put the feather, the leaf, the shell, the rock, and the stick together, you might guess that I sorted those objects by what characteristic?

As I said, I facilitated this activity with the group, and then we discussed the natural ability and tendency of humans to sort and our ability and tendency to see the similarities of things that have been sorted. I am going to leave this story for a time and come back to it in the end, so I can tell you what else is in the HubICL that you might be interested in for answering this question.

As a result of the pandemic this past year and so many things moving to on-line learning, this particular tool in the HubICL also includes a jamboard for you to use and copy, so you can partner off participants into breakout rooms, and they can manipulate pictures of the items, just as they would the real items.

In this article, Thiagi details important questions for facilitators to ask themselves to ensure effectiveness when deciding which facilitation activities to use. 

In this article, Thiagi highlights his philosophy on game facilitation and adaptation, encouraging facilitators and trainers to consider new ways of implementing old games to build their skills and encourage new learning objectives.

This article delineates important characteristics of games, training games, and simulations. 

This article details the history and vision of the "simulation and games movement."

This article presents an analysis of decades of simulation research to identify patterns, issues, and areas of opportunity.

This article includes tips for simulation success, defined as an "aha" moment for learners. 

This article discusses what needs to be present in design and implementation for effective simulation results.