webchick/OSSUMMIT - Gender Bias in Tools

## OSSUMMIT - Gender Bias in Tools
Gender biases embedded in open source software by Anita Sarma @ Oregon State University

Open source is hugely impactful. We’re impacting a lot of people. If we’re not diverse & inclusive, leaving a whole lot of people behind.

OSS communities rely on newcomers. Diversity of thought, new ideas, bringing in life.

How difficult is it to get involved?

Steinmacher et al analyzed OSS communities.

Problems:
 Absence of response, lack of politeness, lack of usefulness of documentation.

Lots of people left behind.

Every 6 months checked in with projects.

82% dropoff. :(

Women are especially left behind.

Women already underrepresented in CS.
57% of jobs in general held by women.
26% of computing jobs held by women.
Within open source, < 10%.

David/Shapiro, Robes
Ghosh: 1.5%
< 5% projects on GitHub own top 5000 projects.

Not about competence. (Terrell et al)

Newcomers whose gender was known had 12% less chance of getting code acceptance if they were women. Elsewhere, they were the same.

Why do we care?
Bad: Bias in software
Good: diversity of thought.


Solutions:
- Fix the people: force us all to think alike
No; fix the software to support diverse ways of problem solving.

Why else to care?
- Ignorance leads to unwitting barriers.
- Studying population segment can help everyone; for example, Curb cuts to help all wheelchair users, also people with walkers, delivery people, cyclists, people with roller suitcases….
Making tools inclusive help lots more people.

Lots of researchers looking into D&I in OSS communities…
But what about the tools?

How are tools contributing to:
- Everyone being left behind by OSS
- Newcomers
- Women in particular

5 teams, 2 companies.
Software professionals used GenderMag
Evaluated software from perspective of “abby” persona (woman newcomer - 4th yr university student)
Use case: “Abby wants to” e.g. submit a pull request


Issues found?
- Their own OSS projects created barriers
- Tools they use — command line, GitHub website
- Infrastructure — docs, wikis

Problems:
- How to set up dev environment.. not where to find things to work on?
- Abby is new… doesn’t even know what “CLA” is. She has no employer.
- I know my stuff works, but I don’t know what a pull request looks like. (Has to learn GitHub UI, Git workflows, etc. to understand.)
- The hard part about PR is to find the right button.

Issues exist in different context….

Finding help w/ pull request
Finding an issue
Getting familiar and finding tag
Set up environment
Review submitted pull request.

Found problems at nearly every step in each area.

Not just fear bugs or UI issues, bt whole spectrum.

What does it mean for newcomers?
58 types of barriers to newcomers to open source in 6 categories.
e.g. Newcomer Orientation

- Finding a task to start on
Finding a mentor
Finding correct …

Poor directions on how to contribute.
“Before you start, reviewing contribution guidelines”
Should she have read it before?
Go back and read now and stop what she’s doing?

Newcomers don’t know contribution flow.
Poor “how to contribute” info.

She’s confused about how to contribute

So newcomer tools are at least in part due to tools.

Are there gender biases inherent in tools?

- Does software support a variety of smart users?
  - For example, cameras to check you in at Canadian Customs, not built for people 5’0” :)
  - We all have unconscious bias; see the world how we see it.

So if all software being built by white males, other smart users being left behind. (Unconscious)


How to identify gender biases in tools?

GenderMag: http://gendermag.org/

Gender Inclusiveness Magnifier
Method/process to evaluate your tool to see if it has inclusivity “bugs” or not.

Set of GenderMag personas, range of users from 5 problem solving facets:

- Motivations
- Information processing style
- Coputer self-efficacy
- Risk averseness
- Tech learning style

e.g. Abby Jones: http://gendermag.org/downloadables/Editable-Personas+Forms/CustomizablePersonas/AbbyPersona-electronicallyCustomizable.pdf

Attitude towards risk: Rarely has spare time, so is risk averse to tech that needs to spend extra time.


Risk facet:

Risk tolerant <=> Risk averse
Men way more risk tolerant (42% men vs. 25% women); women way more risk averse (29% men vs. 38% women), in general.

We tend to build software for the most risk-tolerant audience. This leaves behind 3/4 personas.


How GenderMag works:

1. Pick a persona, e.g. Abby http://gendermag.org/downloadables/
2. Pick a use case/scnario “in an augmented bookstore, find science fiction books”
3. Walk through scenario via “intended” sub-goals/actions

“I need to see a map” => translate to physical actions in interface.
“Would Abby have created this sub-goal?” Yes/No/Maybe
Check if a problem-solving style is a reason Abby would/would not have done this task.

Separate UI issue (e.g. empty error message) vs. how information is provided. (Jargon)


This might take awhile to go through project page, because she has comprehensive information processing (reads a LOT before she starts)
-or-
Resources provided are counter-intuitive to the way that Abby likes to learn.

Results:
41/56 (73%) gender bias barriers in newcomer orientation
23/36 (64%) documentation barriers…
…
Totals: 160/220 (73%) barriers had some sort of gender bias.

How accurate were the findings? Were newcomers actually facing these? How to validate?
- Can do a survey but hard to find people who tried to make a contribution and could not; they usually leave.
- Some people don’t even make the first attempt because they don’t feel included enough.

But, we have access to students.
Empirical study of 18 newcomers: 9 women / 9 men.

Students were asked to fill out diaries for 6 months as they learned how to contribute to an open source project written in a language they already knew. As/when they found problems, jot it down.

For example, todos send you to documentation that isn’t actually there.

Diaries allow you to observe when students hit problems.
Also allow students to get help.

Significant difference in # of gendered barriers:

Women found significantly more barriers than men, and higher percentage of those had gender biases:

Women: 153/251 (61%)
Men: 32/83 (39%)

Tools/infrastructure are implicated in gender biases. Checked this through multiple facets, backed by empirical research.

Conclusion: the “glass floor”

Women in technology do not generally need extra help. But the current environment they work in does need help.

- Support diverse ways of thinking/problme solving
- Fix one facet at a time.

Be a partner:
Use GenderMag in tools/infra
Contribute to GenderMag Recorder’s Assistant
- Help us identify good practices (and anti-patterns) in creating inclusive design
  - Process to follow when creating tools / design practices
  - Product — for example, some info first, click to get more info. Accounts for both “read everything first” learning style, and “dive right in” learning style too.

How to help:
- Collaborate
- Support grad students researching this. ($$$ :))

@GenderMag / #GenderMag
gendermag.method

gendermag.org

anita.sarma@oregonstate.edu

====

QUESTIONS

====

What are people creating in open source? Is software the only product? Do the tools in the docs support additional value, or do they assume software is the only product?

Use cases were tasks in issue tracker, and for software projects it was a software task.
But, we are involved in CHAOSS project D&I about making sure non-code contributions are seen as equally important.

Drupal got a shout-out for being one of the projects interested in writing good documentation and promoting non-technical contributions w/ mentorship. :D :D

—

Learning styles / gender correlation?

We’re looking at 5 learning styles, cross-referenced with risk averseness.
More women than men have this learning style.

These are generalizations. Not all 5 facets belong to all men or all women.
Also, there are situational context… I’m generally risk tolerant, but on release date.. ;)

——

Friction created amongst teams not necessarily around gender lines. For example, more friction in US team, largely homogenous vs. India team which is gender mixed.
- Culture plays a role, and might be a different facet.
- Older vs. younger
- Dis/abled

We focused on “big 5”… these are the 5 different learning styles, validated throughout history, and statistically proven to be more relevant to men vs. women.

But good to dig in and see what’s happening “in the field”… for example, Asian cultures can be less prone to speaking up, respect authority more.

—

More info about cognitive styles?
http://gendermag.org/publications.html

—
Does this have applicability to other places than software?

GenderMag uses “cognitive walkthrough”… go through this page, and ask these questions. Does this have applicability to hardware? Maybe? The elevator at the Hyatt needs help. ;)

Also have researchers looking into applying this in e.g. Africa to ensure it crosses cultural lines
	Gender biases embedded in open source software by Anita Sarma @ Oregon State University

	Open source is hugely impactful. We’re impacting a lot of people. If we’re not diverse & inclusive, leaving a whole lot of people behind.

	OSS communities rely on newcomers. Diversity of thought, new ideas, bringing in life.

	How difficult is it to get involved?

	Steinmacher et al analyzed OSS communities.

	Problems:
	Absence of response, lack of politeness, lack of usefulness of documentation.

	Lots of people left behind.

	Every 6 months checked in with projects.

	82% dropoff. :(

	Women are especially left behind.

	Women already underrepresented in CS.
	57% of jobs in general held by women.
	26% of computing jobs held by women.
	Within open source, < 10%.

	David/Shapiro, Robes
	Ghosh: 1.5%
	< 5% projects on GitHub own top 5000 projects.

	Not about competence. (Terrell et al)

	Newcomers whose gender was known had 12% less chance of getting code acceptance if they were women. Elsewhere, they were the same.

	Why do we care?
	Bad: Bias in software
	Good: diversity of thought.


	Solutions:
	- Fix the people: force us all to think alike
	No; fix the software to support diverse ways of problem solving.

	Why else to care?
	- Ignorance leads to unwitting barriers.
	- Studying population segment can help everyone; for example, Curb cuts to help all wheelchair users, also people with walkers, delivery people, cyclists, people with roller suitcases….
	Making tools inclusive help lots more people.

	Lots of researchers looking into D&I in OSS communities…
	But what about the tools?

	How are tools contributing to:
	- Everyone being left behind by OSS
	- Newcomers
	- Women in particular

	5 teams, 2 companies.
	Software professionals used GenderMag
	Evaluated software from perspective of “abby” persona (woman newcomer - 4th yr university student)
	Use case: “Abby wants to” e.g. submit a pull request


	Issues found?
	- Their own OSS projects created barriers
	- Tools they use — command line, GitHub website
	- Infrastructure — docs, wikis

	Problems:
	- How to set up dev environment.. not where to find things to work on?
	- Abby is new… doesn’t even know what “CLA” is. She has no employer.
	- I know my stuff works, but I don’t know what a pull request looks like. (Has to learn GitHub UI, Git workflows, etc. to understand.)
	- The hard part about PR is to find the right button.

	Issues exist in different context….

	Finding help w/ pull request
	Finding an issue
	Getting familiar and finding tag
	Set up environment
	Review submitted pull request.

	Found problems at nearly every step in each area.

	Not just fear bugs or UI issues, bt whole spectrum.

	What does it mean for newcomers?
	58 types of barriers to newcomers to open source in 6 categories.
	e.g. Newcomer Orientation

	- Finding a task to start on
	Finding a mentor
	Finding correct …

	Poor directions on how to contribute.
	“Before you start, reviewing contribution guidelines”
	Should she have read it before?
	Go back and read now and stop what she’s doing?

	Newcomers don’t know contribution flow.
	Poor “how to contribute” info.

	She’s confused about how to contribute

	So newcomer tools are at least in part due to tools.

	Are there gender biases inherent in tools?

	- Does software support a variety of smart users?
	- For example, cameras to check you in at Canadian Customs, not built for people 5’0” :)
	- We all have unconscious bias; see the world how we see it.

	So if all software being built by white males, other smart users being left behind. (Unconscious)


	How to identify gender biases in tools?

	GenderMag: http://gendermag.org/

	Gender Inclusiveness Magnifier
	Method/process to evaluate your tool to see if it has inclusivity “bugs” or not.

	Set of GenderMag personas, range of users from 5 problem solving facets:

	- Motivations
	- Information processing style
	- Coputer self-efficacy
	- Risk averseness
	- Tech learning style

	e.g. Abby Jones: http://gendermag.org/downloadables/Editable-Personas+Forms/CustomizablePersonas/AbbyPersona-electronicallyCustomizable.pdf

	Attitude towards risk: Rarely has spare time, so is risk averse to tech that needs to spend extra time.


	Risk facet:

	Risk tolerant <=> Risk averse
	Men way more risk tolerant (42% men vs. 25% women); women way more risk averse (29% men vs. 38% women), in general.

	We tend to build software for the most risk-tolerant audience. This leaves behind 3/4 personas.


	How GenderMag works:

	1. Pick a persona, e.g. Abby http://gendermag.org/downloadables/
	2. Pick a use case/scnario “in an augmented bookstore, find science fiction books”
	3. Walk through scenario via “intended” sub-goals/actions

	“I need to see a map” => translate to physical actions in interface.
	“Would Abby have created this sub-goal?” Yes/No/Maybe
	Check if a problem-solving style is a reason Abby would/would not have done this task.

	Separate UI issue (e.g. empty error message) vs. how information is provided. (Jargon)


	This might take awhile to go through project page, because she has comprehensive information processing (reads a LOT before she starts)
	-or-
	Resources provided are counter-intuitive to the way that Abby likes to learn.

	Results:
	41/56 (73%) gender bias barriers in newcomer orientation
	23/36 (64%) documentation barriers…
	…
	Totals: 160/220 (73%) barriers had some sort of gender bias.

	How accurate were the findings? Were newcomers actually facing these? How to validate?
	- Can do a survey but hard to find people who tried to make a contribution and could not; they usually leave.
	- Some people don’t even make the first attempt because they don’t feel included enough.

	But, we have access to students.
	Empirical study of 18 newcomers: 9 women / 9 men.

	Students were asked to fill out diaries for 6 months as they learned how to contribute to an open source project written in a language they already knew. As/when they found problems, jot it down.

	For example, todos send you to documentation that isn’t actually there.

	Diaries allow you to observe when students hit problems.
	Also allow students to get help.

	Significant difference in # of gendered barriers:

	Women found significantly more barriers than men, and higher percentage of those had gender biases:

	Women: 153/251 (61%)
	Men: 32/83 (39%)

	Tools/infrastructure are implicated in gender biases. Checked this through multiple facets, backed by empirical research.

	Conclusion: the “glass floor”

	Women in technology do not generally need extra help. But the current environment they work in does need help.

	- Support diverse ways of thinking/problme solving
	- Fix one facet at a time.

	Be a partner:
	Use GenderMag in tools/infra
	Contribute to GenderMag Recorder’s Assistant
	- Help us identify good practices (and anti-patterns) in creating inclusive design
	- Process to follow when creating tools / design practices
	- Product — for example, some info first, click to get more info. Accounts for both “read everything first” learning style, and “dive right in” learning style too.

	How to help:
	- Collaborate
	- Support grad students researching this. ($$$ :))

	@GenderMag / #GenderMag
	gendermag.method

	gendermag.org

	anita.sarma@oregonstate.edu

	====

	QUESTIONS

	====

	What are people creating in open source? Is software the only product? Do the tools in the docs support additional value, or do they assume software is the only product?

	Use cases were tasks in issue tracker, and for software projects it was a software task.
	But, we are involved in CHAOSS project D&I about making sure non-code contributions are seen as equally important.

	Drupal got a shout-out for being one of the projects interested in writing good documentation and promoting non-technical contributions w/ mentorship. :D :D

	—

	Learning styles / gender correlation?

	We’re looking at 5 learning styles, cross-referenced with risk averseness.
	More women than men have this learning style.

	These are generalizations. Not all 5 facets belong to all men or all women.
	Also, there are situational context… I’m generally risk tolerant, but on release date.. ;)

	——

	Friction created amongst teams not necessarily around gender lines. For example, more friction in US team, largely homogenous vs. India team which is gender mixed.
	- Culture plays a role, and might be a different facet.
	- Older vs. younger
	- Dis/abled

	We focused on “big 5”… these are the 5 different learning styles, validated throughout history, and statistically proven to be more relevant to men vs. women.

	But good to dig in and see what’s happening “in the field”… for example, Asian cultures can be less prone to speaking up, respect authority more.

	—

	More info about cognitive styles?
	http://gendermag.org/publications.html

	—
	Does this have applicability to other places than software?

	GenderMag uses “cognitive walkthrough”… go through this page, and ask these questions. Does this have applicability to hardware? Maybe? The elevator at the Hyatt needs help. ;)

	Also have researchers looking into applying this in e.g. Africa to ensure it crosses cultural lines