helloladies: Horseshoe icon with the words Lady Business underneath. (Default)
[personal profile] helloladies posting in [community profile] ladybusiness
The Backstory
In mid-2011 after a discussion about diversity in reading with my partner, we agreed to focus our reading in 2012 on women by undertaking a project called She Wrote What?. The project is not focused on SF/F, but those are the genres he likes the best. To find recommendations for us I began following popular SF/F blogs for their reviews and back catalogues.

A trend emerged as I collected women writers. Women's names started to repeat while more and more new male writers emerged. This reminded me of The Count 2010 by VIDA and The SF Count by Strange Horizons (which I hope they repeat). I did two test cases on blogs I was already following, using their current reviews for a three month period. Toward the end of my test case, I mentioned it on Twitter and had a conversation, which I include here for context:

@renay: I am basically looking at SF/F blogs and counting how many books by ladies they've reviewed in 2011. SPOILER: it's depressing.
@kingrat: I've thought about doing that project. I could predict the results.
@renay: Is that why you never did it? End result too predictable?
@kingrat: When @niallharrison did the print mag review numbers this year, i thought a similar blog project might be worthwhile.
@niallharrison: @kingrat @renay I shied away from blogs at the time because so many of them are single-author it felt a bit personal.
@niallharrison: But it would be instructive to see, I think.
@renay: @niallharrison Many position themselves as professional, especially those receiving review copy.
@niallharrison: Even so, I think there's a difference in the implicit promises magazines and blogs make: comprehensiveness vs idiosyncracy.
@renay: Whereas if I sample a group of them over a one year period, meaning will be had! What kind, though, remains to be seen.
@niallharrison: Yeah, that's the level at which it gets interesting. Though I wouldn't want to be the one selecting the sample. :-)
@renay: @niallharrison I started at the Major Players and used blogrolls. I'm sure this is the most effective and least wanky solution! #doom
@niallharrison: Defined methodology is good! I look forward to seeing the results, anyway.

I'll come back to defined methodology.

The Inevitable Disclaimer
This has been such an undertaking; five months of my life were consumed by this project. An unexpected development is that after spending time with the blogs that became my sample, I feel protective of them even though all of the owners, sans The Book Smugglers, are strangers to me. Toward the end, even though KJ told me early on I couldn't hoard the data and mask the blog identities, that it would be a Jerk Move, the more tempted I became to do so. But here we are, and nothing is being hidden, but I still feel like standing on the rooftop to wave my arms and shout about how awesome these blogs are for literature, how diverse of opinion and thought, how lucky we are to be in a community and fandom with this much to choose from, and to please not go pick on them or call them names or accuse them of things based on a year's worth of numbers gleaned from narrow rules I applied to collect the information. Honestly: don't do it. That's a Jerk Move.

I did not begin this project as an expose of individual blogs, because individually there's nothing to glean from the numbers. One year of a person's reviewing habits tells you exactly zero about that person, and furthermore, not all reviews are included — only SF/F and speculative fiction, defined by me, by using Amazon and Goodreads and tags, and reviews of the work elsewhere. On top of that, a list of reviews and an actual reading list are two different things.

I'm not making any claims as to how each blog owner or contributor feels about women writers.

Although Niall had qualms about examining blogs in this way because of the personal aspect, I obviously disagreed, as here we are, on the cusp of Graphs and Lots of Data. Yes, it can be personal, but when we have review policies (most of the blogs I examined did) we're positioning ourselves as a voice for literature to other people, even if they happen to be our friends. When we tell a publisher we're willing to read and review books and then post those reviews, we're telling the world we have something to say and inviting the world in to listen. Furthermore, for me this is personal, because recommendations from mostly recreational, non-professional reviewers is how I tend to find the SF/F media I consume.

I did my best to apply my decisions equally across all reviews. I stopped at 21 blogs because of severe issue fatigue. There are hundreds of SF/F blogs and 21 barely skims the surface.

I chose my sample by using the blogs I follow, blogs from their blogrolls, recommendations from my co-contributors to Lady Business and Ana of The Book Smugglers who is active in the SF/F community (this is apparently called snowball sampling). I based my decision on whether it seemed like the blog was SF/F related and subscriber count. I did not research archives of any blogs but my two test cases beforehand and once I started processing a blog, I did not remove it from my sample.

I am not sure what I would have found if I had kept adding blogs. Maybe 50 would have been the magic number where the scale tipped the other way from what my results turned out to be. However, anyone can feel free to look over their favorite SF/F blog and its reviews and report back what they find and how it compares to my results. Google Docs is free! \o/

This survey focuses on reviews of science fiction and fantasy texts (more on sequential art below). Many blogs were good enough to note the genre. Some didn't and I had to do a little legwork. Several people came behind me and gave me advice and opinions on reviews I was unsure about and did checks on author gender for me.

In the reviews total, each book was counted M, F, or U. This project is, at its core, a binary one. I make no claims as to the gender identities of the authors or bloggers classified as Unknown.

Authors can appear multiple times in the reviews total. The only exception to this was when there were multiple books by the same author in the same post. That's counted as one because those reviews are happening at the same time in the same entry. That's different than putting that author and their work in front of eyeballs three different times.

1. I took a very generous line on what I counted as SF/F and speculative. There's going to be something like Spellcast by Barbara Ashford in the same study as The Clockwork Rocket by Greg Egan. That's how I'm rolling; it's okay if you don't agree.

2. I only counted reviews hosted on the actual blog. Woe, should they be posted elsewhere with a link to them!

3. Young Adult and adult SF/F and speculative fiction were included, but not Middle Grade. If anything was unclear, I used Amazon/Goodreads based on where it was listed to make that decision.

4. Anthologies with multiple authors are treated by authors (sorry editors).

5. Writers under a shared pseud that's publicly known were counted individually. Unfortunately concerning authors filed under Unknown, it's impossible to know if those are shared and thus they are counted as one author.

6. Co-authors were counted individually.

7. Guest Post reviews were not included (unless they weren't labeled as Guest Posts, and if so, well-played blog owners. Well-played.).

6. Originally I included sequential art (comics, manga, and graphic novels) in these numbers, but unfortunately, they skewed the results too much. While all blogs included traditional texts, only a handful covered sequential art: strike one. The sheer number of men in the industry also made some blogs have wild ratios that, bottom line, annoyed me and made their coverage look terrible through no fault of their own due to my inclusion standards: strike two. It was extremely difficult to find complete lists of all artists. Including all artists — colorists and pencilers, etc. — was a requirement for me based on opinions of artists I asked. Some projects were almost impossible to find information for, which meant some listings would be complete but others wouldn't be. Because this project relies on complete contributor lists, I deemed it too time intensive: strike three. It was also like a bonus round of "wow, women sure are invisible in the comics industry, huh?" It was supremely depressing and in the end that was a discussion I simply didn't want to dive into at this time.

7. Horror that was psychological or based on gore or torture porn didn't count. Horror that was fantastical or paranormal did. This is still my most personally conflicting category; there are probably a lot of errors and missing information or additional information that shouldn't count. It still makes me unhappy. I asked for second opinions here because I don't read horror and did the best I could with research.

8. On gender of authors and bloggers: when possible, I checked when gender was not clear. I noted when it was unknown or if I couldn't find enough data to make a clear decision. I did my best to examine each. It feels pertinent to note that one of the hardest things I faced was women writing under male pseudonyms. There are so many men represented that it's truly difficult to say I researched each one as thoroughly as I could have. I did my best and erred when in doubt. If I was wrong, please tell me, and I will make the correction with apologies.

I was also dealing with hundreds of blog posts and Google Reader, which underwent a horrible, eye-gougingly awful redesign in the middle of this project (thanks Google). I am certain there are reviews I missed because of this awesome development, authors I've placed incorrectly because of lackluster research, and other fun, humiliating errors. I'm willing to update my spreadsheets and include notes in this post if the information is brought to me and presented convincingly, although the data presented graphically in this post will not be updated for a few months, as I'll let any corrections float in and request a final update of the graphs to be included alongside the current information.

Version one of my spreadsheets can be found at Coverage of Women on SF/F Blogs (I). All the data and the blogs I acquired it from that I am discussing below is contained in these sheets.

The Results
In the beginning, I was fairly sure of what I was going to find: men discussing mostly men, and women discussing both either equally or more. Does the data follow?

This is a fairly standard result that's not ideal, but is better than what it could be. Men still dominate the literary conversation, but women are in there, too. I was initially surprised by this result, because my gut back in 2011 had said it was not this even.

Even when expressed in a different way, the "overall" score seems to indicate that my gut was jumping the gun:

However, if you start rearranging the data a bit, things change. There are women being reviewed by men, yes, but there are also women being reviewed by women. My initial instinct was correct. My results highlight an interesting development here given the context of Girls as Omni-Readers, as Jodie wrote last year during an unrelated debate in the YA community, where she said,

'Omni-reader' is a term Maureen Johnson used on her Twitter feed two weeks ago. She used this word as a way of describing beliefs that girls are readers who will read about anything regardless of whether the subject matter looks traditionally masculine, or traditionally feminine. Articles like this one by Sarah Pekkanen of The Washington Post agree that girls tend to accept a broad range of books and will read a book featuring a boy on the cover [...]

If you look at the same graph, reorganized to take into account the blogger's gender (as expressed in profiles, about pages, etc.):

Here we have the data reorganized by gender of the reviewer, female, to mixed gender, to male, to the one unknown. And thus, the feeling I had turned out not to be so wrong after all: the 40/60 is an average, and that average is the way it is because the women reviewing women drive it up.

The breakdown comes out to:

  • Group blogs: 25% women

  • Female bloggers: 58% women

  • Male bloggers: 19% women

(The blogger with the unknown gender is not included in this graph.)

So no, I wasn't wrong last year to go, "hey, wait a minute..." and feel like things were unequal. If you're following popular SF/F blogs run by men I believe this is a problem you will continuously run into, except by those focused on their review diversity. Reviews of books by women don't feature as often on blogs run by men or shared by men, but on blogs run by women, it's more equal and sometimes even women as majority because culturally women are trained to read "traditionally" male things, while the reverse is not true.

Recently I linked to a post by [personal profile] coffeeandink where she examined a panel of authors talking about SF/F titled The erasure of women writers in sf & fantasy. I quoted that post then and said I would come back to quote it again because it's so insightful and says everything I wasn't educated enough to know or speak about:

I am not suggesting that the participants are consciously sexist or intend to suppress or erase the existence of women writers. I am saying that this conversation follows a typical social pattern in which (a) men talk more than women in mixed company; (b) men promote male writers significantly more than they promote women writers; (c) the criteria which determine value or worth inherently favor men's contributions over women's, which are deemed trivial or inapplicable; (d) women's contributions to the critical or cultural canon are systematically devalued, forgotten, or erased.

Are there answers in these numbers? I'm only finding more questions. There are no good, easy answers. Of course, some like to pretend there are easy answers and that's where I've watched this debate fall apart in the past whenever it's cropped up. It goes to Disasterland immediately and becomes a shame spiral. Someone inevitably shows up and starts talking about why quotas are bad and no, no, how dare you call them sexist and a game of Defensive Assholes is launched. The entire situation devolves into Don't Read the Comments™ and it's Gender Catastrophe Theater and general badness, which accomplishes less than nothing, because if we go back to the beginning, no one called anyone any names at all. I want to avoid that as much as possible. Reading diversity is a complicated subject and book selection often a process that we're not conscious of. We're impacted on all sides by a myriad of things influencing our decisions. But reviewing and talking about titles on public blogs and journals is an active decision that we're making every time we put a book down and go, "I'm going to write 1000 words about that and share it with the Internet!"

As friends finalized this data for me, made the graphs you see above and I started thinking about what to say about them, VIDA released The 2011 Count. It's not specific to SF/F like my project and and it's professionally focused, but I think it's fascinating to look and see the same trends in an unrelated sphere repeated in this one.

What are we saying to those who trust our reading choices? What are we saying to the publishers who send us materials to review about the books that deserve that kind of virtual hand-selling? Does it impact what they think is relevant and sellable? What does it mean when we review that book by a man, and that one, and that other one and pass over the women writing the same kind of story? There's worth in examining the reviewing choices we're making. There's worth in thinking about what messages we're sending when our promotional energies favor the dominant gender without letting ourselves get mired in arguments grounded in gender essentialism.

Diverse voices in literature we celebrate is incredibly important. I encourage everyone to look at their SF/F review statistics. It's not an easy task if the results are uneven, but it's a worthwhile one.

— Renay

Credit: Ira and Susan for advice, Lex for beta help (congrats on finding a typo in the 200th edit), Ana and Jodie for their support and suggestions, handholding, long chats about all the feelings this project caused and beta help, Ana from The Book Smugglers for blog suggestions, feedback and beta help, KJ for graphing the data and being a sounding-board, and Philip for derail critique, enabling and supporting from backstage with the promotion hat. Thank you all for your assistance, I will never ask you to do this again...probably. ;)

Further reading:
Sidetracks - Science Fiction Edition!
Summary of comments for SF Signal's Russ Pledge post
Everyone Can Promote Equality In Genre Writing
How to Suppress Women's Writing

eta - 3/9/12 3:15P.M.: Going forward, to leave anonymous comments on this post you must sign your comment with the name you use online or a name created specifically for commenting across this post. Any non-signed comments will be screened upon discovery. We will not engage with unsigned anonymous comments.

Base rate

Date: 2012-03-08 05:50 pm (UTC)
From: [personal profile] omgkittens
It's awesome that you've done all this. I think you need to consider the base rate, though; it really matters what percentage of books published are by women.

For instance, I went to amazon.com, selected SF&F paperbacks, sorted by date (which should be a fairly random selection) and counted the number of books in the top 100 with at least one female author. I counted 35. Now a sample of 100 isn't that many and the 95% confidence interval is about +/-9% and I'm not really happy with how I handled the multiple authors, but for the sake of argument let's work it through assuming 35% women and 65% men. If I denote the probability of someone reading a book by a same-gendered author by 'ps' and by an opposite-gendered author by 'pd' then

for women:

(pd/ps)*(65/35) = 42/58
pd/ps = 35*42/(58*65) = 0.39

for men:

(pd/ps)*(35/65) = 18/82
pd/ps = 65*18/(82*35) = 0.41

So these numbers suggest, bearing in mind all the caveats above, that if I pick a book entirely at random from the SF&F books being published, the women in your sample are 39% as likely to review it if it's by a man as if it's by woman, and the men are 41% as likely to review it if it's by a woman as if it's by a man. The difference doesn't make men seem so awful when you express it like that!

Now here's the thing: I bet the publishers have similar figures to yours. And I bet they're reading them the same way many people are, as 'women read male authors and men don't read female authors', since publishers generally aren't statisticians. And they may well be drawing the conclusion that it's better to publish books by men because they appeal to a wider audience. And they might be *completely wrong*.

Re: Base rate

Date: 2012-03-08 06:50 pm (UTC)
julieandrews: (Default)
From: [personal profile] julieandrews
Amazon is hit or miss whether YA or 'Teen' books are included in SF/F. I searched within SF/F for 'stephenie meyer' and got The Host and Breaking Dawn, but none of her others.

So I'd say mostly your search didn't include any YA. Which should skew female.

Re: Base rate

Date: 2012-03-08 07:56 pm (UTC)
julieandrews: (Default)
From: [personal profile] julieandrews
change that to: most likely your search didn't include much or any YA.

Writing in haste meant that sentence didn't make much sense. :)

Re: Base rate

Date: 2012-03-10 04:06 am (UTC)
From: (Anonymous)
I didn't know that! Thanks, that's a pretty big bias in the data.

If some genres skew more towards male or female authors (which is not surprising of course) then I think it would definitely be useful to stratify the data according to genre.

Re: Base rate

Date: 2012-03-08 07:01 pm (UTC)
coffeeandink: (Default)
From: [personal profile] coffeeandink
So these numbers suggest, bearing in mind all the caveats above, that if I pick a book entirely at random from the SF&F books being published, the women in your sample are 39% as likely to review it if it's by a man as if it's by woman, and the men are 41% as likely to review it if it's by a woman as if it's by a man.

That's quite an assumption

Re: Base rate

Date: 2012-03-08 07:16 pm (UTC)
renay: artist rendition of the center of a nebula (Default)
From: [personal profile] renay
Unfortunately, I don't understand any of those numbers. That's just me being unskilled in mathematics, sorry, I will try to get my math friends to explain it to me offline. I am also leery of using only Amazon to judge anything, especially (seconding the commentator below) they have such an odd system that often includes/blocks YA SF/F based on rules I don't understand. Sometimes I see YA SF/F by searching for science fiction and sometimes I do not. I don't trust any system they treats YA SF/F the way they often do.

Also, I think we should be careful how we frame this: I'm not calling men awful. I'm saying men are more likely than women fall into cultural potholes regarding reviewing diversity. This project isn't about reading. It's about reviewing. Although, if it makes people look at their numbers and consider them, that's a bonus.

Anyway, my theory that publishers think it's better to publish books by men because the feedback they get is that books by men generate the most buzz may be right or wrong. My hope is less that publishers fix themselves and more that reviewers start to look at their promotional history and change publishing by actively asking after books written by women and supporting women writers, translating the need for diversity into something publishers understand ($$$).

This isn't only about facts and numbers. It's also about feelings and habits of a community that's also, at this very moment, having a debate about including bloggers in the Hugo nominations and ignoring women writers there, too.

Anyway, I don't think we disagree! This is a super small and ultimately biased sample that I am theorizing about and posing questions about that I don't even know if there are correct answers to (or even any answers at all). Maybe publishers should hire a stat person or two. It's done pretty well for the creepy retailer circuit. :)

Re: Base rate

Date: 2012-03-10 04:20 am (UTC)
From: [personal profile] omgkittens
Right, I don't think we disagree either, and of course it's a shame if men are rarely reading fiction by women for _whatever_ reason. I just know that our monkey brains want to interpret numbers far beyond their literal significance, which is fine, but it needs to be done with statistical subtlety if the conclusions are to be valid. But it looks like you're right about the bias against YA authors; taking them into account the picture might look even more gloomy than in a simple analysis.

(And your sample really isn't that small btw, I've seen plenty of published studies with fewer data points.)

Re: Base rate

Date: 2012-03-09 05:38 am (UTC)
From: (Anonymous)
I was gonna say something similar, thanks for stealing these thoughts in advance. :)


Re: Base rate

Date: 2012-03-09 08:18 am (UTC)
From: (Anonymous)
For someone so interested in number-crunching accurately and not presenting misleading data, you show an astonishing faith in the Amazon search engine's ability to give you accurate data. (For example, you're assuming all novels are correctly categorised in the metadata Amazon uses; tip: they aren't.)

Surely a better measure would be totalling up what is published in a given year (as made available by venues like Locus), rather than performing a haphazard search on whatever happens to be in the top sellers at any given moment.

Nic (@bibliolicious)

Re: Base rate

Date: 2012-03-10 04:08 am (UTC)
From: [personal profile] omgkittens
Of course I could have done a much better job all round, but then I couldn't have done it over breakfast. My point was to be illustrative, not authoritative.


Lady Business welcome badge

Pitch Us!
Review Policy
Comment Policy
Writers We Like!
Contact Us

tumblr icon twitter icon syndication icon

image asking viewer to support Lady Business on Patreon

Who We Are

Ira is an illustrator and gamer who decided that disagreeing with everyone would be a good way to spend their time on the internet. more? » twitter icon tumblr icon AO3 icon

By day Jodie is currently living the dream as a bookseller for a major British chain of book shops. She has no desire to go back to working in the real world. more? » tumblr icon last.fm icon

KJ KJ is an underemployed librarian, lifelong reader, and more recently an avid gamer. more? » twitter icon tumblr icon AO3 icon

Renay writes for Lady Business and co-hosts Fangirl Happy Hour, a pop culture media show that includes a lot yelling about the love lives of fictional characters. Enjoys puns. more? » twitter icon pinboard icon tumblr icon

Susan is a library assistant who uses her insider access to keep her shelves and to-read list permanently over-flowing. more? » twitter icon pinboard icon AO3 icon


Book Review Index
Film Review Index
Television Review Index
Game Review Index
Non-Review Index
We Want It!
Fanwork Recs
all content by tags

Our Projects

hugo award recs

Criticism & Debate

Indeed, we do have a comment policy.

What's with your subtitle?

It's a riff off an extremely obscure meme only Tom Hardy and Myspace fans will appreciate.

hugo award winner
Powered by Dreamwidth Studios