helloladies: Horseshoe icon with the words Lady Business underneath. (Default)
[personal profile] helloladies posting in [community profile] ladybusiness
The Backstory
In mid-2011 after a discussion about diversity in reading with my partner, we agreed to focus our reading in 2012 on women by undertaking a project called She Wrote What?. The project is not focused on SF/F, but those are the genres he likes the best. To find recommendations for us I began following popular SF/F blogs for their reviews and back catalogues.

A trend emerged as I collected women writers. Women's names started to repeat while more and more new male writers emerged. This reminded me of The Count 2010 by VIDA and The SF Count by Strange Horizons (which I hope they repeat). I did two test cases on blogs I was already following, using their current reviews for a three month period. Toward the end of my test case, I mentioned it on Twitter and had a conversation, which I include here for context:

@renay: I am basically looking at SF/F blogs and counting how many books by ladies they've reviewed in 2011. SPOILER: it's depressing.
@kingrat: I've thought about doing that project. I could predict the results.
@renay: Is that why you never did it? End result too predictable?
@kingrat: When @niallharrison did the print mag review numbers this year, i thought a similar blog project might be worthwhile.
@niallharrison: @kingrat @renay I shied away from blogs at the time because so many of them are single-author it felt a bit personal.
@niallharrison: But it would be instructive to see, I think.
@renay: @niallharrison Many position themselves as professional, especially those receiving review copy.
@niallharrison: Even so, I think there's a difference in the implicit promises magazines and blogs make: comprehensiveness vs idiosyncracy.
@renay: Whereas if I sample a group of them over a one year period, meaning will be had! What kind, though, remains to be seen.
@niallharrison: Yeah, that's the level at which it gets interesting. Though I wouldn't want to be the one selecting the sample. :-)
@renay: @niallharrison I started at the Major Players and used blogrolls. I'm sure this is the most effective and least wanky solution! #doom
@niallharrison: Defined methodology is good! I look forward to seeing the results, anyway.

I'll come back to defined methodology.

The Inevitable Disclaimer
This has been such an undertaking; five months of my life were consumed by this project. An unexpected development is that after spending time with the blogs that became my sample, I feel protective of them even though all of the owners, sans The Book Smugglers, are strangers to me. Toward the end, even though KJ told me early on I couldn't hoard the data and mask the blog identities, that it would be a Jerk Move, the more tempted I became to do so. But here we are, and nothing is being hidden, but I still feel like standing on the rooftop to wave my arms and shout about how awesome these blogs are for literature, how diverse of opinion and thought, how lucky we are to be in a community and fandom with this much to choose from, and to please not go pick on them or call them names or accuse them of things based on a year's worth of numbers gleaned from narrow rules I applied to collect the information. Honestly: don't do it. That's a Jerk Move.

I did not begin this project as an expose of individual blogs, because individually there's nothing to glean from the numbers. One year of a person's reviewing habits tells you exactly zero about that person, and furthermore, not all reviews are included — only SF/F and speculative fiction, defined by me, by using Amazon and Goodreads and tags, and reviews of the work elsewhere. On top of that, a list of reviews and an actual reading list are two different things.

I'm not making any claims as to how each blog owner or contributor feels about women writers.

Although Niall had qualms about examining blogs in this way because of the personal aspect, I obviously disagreed, as here we are, on the cusp of Graphs and Lots of Data. Yes, it can be personal, but when we have review policies (most of the blogs I examined did) we're positioning ourselves as a voice for literature to other people, even if they happen to be our friends. When we tell a publisher we're willing to read and review books and then post those reviews, we're telling the world we have something to say and inviting the world in to listen. Furthermore, for me this is personal, because recommendations from mostly recreational, non-professional reviewers is how I tend to find the SF/F media I consume.

I did my best to apply my decisions equally across all reviews. I stopped at 21 blogs because of severe issue fatigue. There are hundreds of SF/F blogs and 21 barely skims the surface.

I chose my sample by using the blogs I follow, blogs from their blogrolls, recommendations from my co-contributors to Lady Business and Ana of The Book Smugglers who is active in the SF/F community (this is apparently called snowball sampling). I based my decision on whether it seemed like the blog was SF/F related and subscriber count. I did not research archives of any blogs but my two test cases beforehand and once I started processing a blog, I did not remove it from my sample.

I am not sure what I would have found if I had kept adding blogs. Maybe 50 would have been the magic number where the scale tipped the other way from what my results turned out to be. However, anyone can feel free to look over their favorite SF/F blog and its reviews and report back what they find and how it compares to my results. Google Docs is free! \o/

This survey focuses on reviews of science fiction and fantasy texts (more on sequential art below). Many blogs were good enough to note the genre. Some didn't and I had to do a little legwork. Several people came behind me and gave me advice and opinions on reviews I was unsure about and did checks on author gender for me.

In the reviews total, each book was counted M, F, or U. This project is, at its core, a binary one. I make no claims as to the gender identities of the authors or bloggers classified as Unknown.

Authors can appear multiple times in the reviews total. The only exception to this was when there were multiple books by the same author in the same post. That's counted as one because those reviews are happening at the same time in the same entry. That's different than putting that author and their work in front of eyeballs three different times.

1. I took a very generous line on what I counted as SF/F and speculative. There's going to be something like Spellcast by Barbara Ashford in the same study as The Clockwork Rocket by Greg Egan. That's how I'm rolling; it's okay if you don't agree.

2. I only counted reviews hosted on the actual blog. Woe, should they be posted elsewhere with a link to them!

3. Young Adult and adult SF/F and speculative fiction were included, but not Middle Grade. If anything was unclear, I used Amazon/Goodreads based on where it was listed to make that decision.

4. Anthologies with multiple authors are treated by authors (sorry editors).

5. Writers under a shared pseud that's publicly known were counted individually. Unfortunately concerning authors filed under Unknown, it's impossible to know if those are shared and thus they are counted as one author.

6. Co-authors were counted individually.

7. Guest Post reviews were not included (unless they weren't labeled as Guest Posts, and if so, well-played blog owners. Well-played.).

6. Originally I included sequential art (comics, manga, and graphic novels) in these numbers, but unfortunately, they skewed the results too much. While all blogs included traditional texts, only a handful covered sequential art: strike one. The sheer number of men in the industry also made some blogs have wild ratios that, bottom line, annoyed me and made their coverage look terrible through no fault of their own due to my inclusion standards: strike two. It was extremely difficult to find complete lists of all artists. Including all artists — colorists and pencilers, etc. — was a requirement for me based on opinions of artists I asked. Some projects were almost impossible to find information for, which meant some listings would be complete but others wouldn't be. Because this project relies on complete contributor lists, I deemed it too time intensive: strike three. It was also like a bonus round of "wow, women sure are invisible in the comics industry, huh?" It was supremely depressing and in the end that was a discussion I simply didn't want to dive into at this time.

7. Horror that was psychological or based on gore or torture porn didn't count. Horror that was fantastical or paranormal did. This is still my most personally conflicting category; there are probably a lot of errors and missing information or additional information that shouldn't count. It still makes me unhappy. I asked for second opinions here because I don't read horror and did the best I could with research.

8. On gender of authors and bloggers: when possible, I checked when gender was not clear. I noted when it was unknown or if I couldn't find enough data to make a clear decision. I did my best to examine each. It feels pertinent to note that one of the hardest things I faced was women writing under male pseudonyms. There are so many men represented that it's truly difficult to say I researched each one as thoroughly as I could have. I did my best and erred when in doubt. If I was wrong, please tell me, and I will make the correction with apologies.

I was also dealing with hundreds of blog posts and Google Reader, which underwent a horrible, eye-gougingly awful redesign in the middle of this project (thanks Google). I am certain there are reviews I missed because of this awesome development, authors I've placed incorrectly because of lackluster research, and other fun, humiliating errors. I'm willing to update my spreadsheets and include notes in this post if the information is brought to me and presented convincingly, although the data presented graphically in this post will not be updated for a few months, as I'll let any corrections float in and request a final update of the graphs to be included alongside the current information.

Version one of my spreadsheets can be found at Coverage of Women on SF/F Blogs (I). All the data and the blogs I acquired it from that I am discussing below is contained in these sheets.

The Results
In the beginning, I was fairly sure of what I was going to find: men discussing mostly men, and women discussing both either equally or more. Does the data follow?

This is a fairly standard result that's not ideal, but is better than what it could be. Men still dominate the literary conversation, but women are in there, too. I was initially surprised by this result, because my gut back in 2011 had said it was not this even.

Even when expressed in a different way, the "overall" score seems to indicate that my gut was jumping the gun:

However, if you start rearranging the data a bit, things change. There are women being reviewed by men, yes, but there are also women being reviewed by women. My initial instinct was correct. My results highlight an interesting development here given the context of Girls as Omni-Readers, as Jodie wrote last year during an unrelated debate in the YA community, where she said,

'Omni-reader' is a term Maureen Johnson used on her Twitter feed two weeks ago. She used this word as a way of describing beliefs that girls are readers who will read about anything regardless of whether the subject matter looks traditionally masculine, or traditionally feminine. Articles like this one by Sarah Pekkanen of The Washington Post agree that girls tend to accept a broad range of books and will read a book featuring a boy on the cover [...]

If you look at the same graph, reorganized to take into account the blogger's gender (as expressed in profiles, about pages, etc.):

Here we have the data reorganized by gender of the reviewer, female, to mixed gender, to male, to the one unknown. And thus, the feeling I had turned out not to be so wrong after all: the 40/60 is an average, and that average is the way it is because the women reviewing women drive it up.

The breakdown comes out to:

  • Group blogs: 25% women

  • Female bloggers: 58% women

  • Male bloggers: 19% women

(The blogger with the unknown gender is not included in this graph.)

So no, I wasn't wrong last year to go, "hey, wait a minute..." and feel like things were unequal. If you're following popular SF/F blogs run by men I believe this is a problem you will continuously run into, except by those focused on their review diversity. Reviews of books by women don't feature as often on blogs run by men or shared by men, but on blogs run by women, it's more equal and sometimes even women as majority because culturally women are trained to read "traditionally" male things, while the reverse is not true.

Recently I linked to a post by [personal profile] coffeeandink where she examined a panel of authors talking about SF/F titled The erasure of women writers in sf & fantasy. I quoted that post then and said I would come back to quote it again because it's so insightful and says everything I wasn't educated enough to know or speak about:

I am not suggesting that the participants are consciously sexist or intend to suppress or erase the existence of women writers. I am saying that this conversation follows a typical social pattern in which (a) men talk more than women in mixed company; (b) men promote male writers significantly more than they promote women writers; (c) the criteria which determine value or worth inherently favor men's contributions over women's, which are deemed trivial or inapplicable; (d) women's contributions to the critical or cultural canon are systematically devalued, forgotten, or erased.

Are there answers in these numbers? I'm only finding more questions. There are no good, easy answers. Of course, some like to pretend there are easy answers and that's where I've watched this debate fall apart in the past whenever it's cropped up. It goes to Disasterland immediately and becomes a shame spiral. Someone inevitably shows up and starts talking about why quotas are bad and no, no, how dare you call them sexist and a game of Defensive Assholes is launched. The entire situation devolves into Don't Read the Comments™ and it's Gender Catastrophe Theater and general badness, which accomplishes less than nothing, because if we go back to the beginning, no one called anyone any names at all. I want to avoid that as much as possible. Reading diversity is a complicated subject and book selection often a process that we're not conscious of. We're impacted on all sides by a myriad of things influencing our decisions. But reviewing and talking about titles on public blogs and journals is an active decision that we're making every time we put a book down and go, "I'm going to write 1000 words about that and share it with the Internet!"

As friends finalized this data for me, made the graphs you see above and I started thinking about what to say about them, VIDA released The 2011 Count. It's not specific to SF/F like my project and and it's professionally focused, but I think it's fascinating to look and see the same trends in an unrelated sphere repeated in this one.

What are we saying to those who trust our reading choices? What are we saying to the publishers who send us materials to review about the books that deserve that kind of virtual hand-selling? Does it impact what they think is relevant and sellable? What does it mean when we review that book by a man, and that one, and that other one and pass over the women writing the same kind of story? There's worth in examining the reviewing choices we're making. There's worth in thinking about what messages we're sending when our promotional energies favor the dominant gender without letting ourselves get mired in arguments grounded in gender essentialism.

Diverse voices in literature we celebrate is incredibly important. I encourage everyone to look at their SF/F review statistics. It's not an easy task if the results are uneven, but it's a worthwhile one.

— Renay

Credit: Ira and Susan for advice, Lex for beta help (congrats on finding a typo in the 200th edit), Ana and Jodie for their support and suggestions, handholding, long chats about all the feelings this project caused and beta help, Ana from The Book Smugglers for blog suggestions, feedback and beta help, KJ for graphing the data and being a sounding-board, and Philip for derail critique, enabling and supporting from backstage with the promotion hat. Thank you all for your assistance, I will never ask you to do this again...probably. ;)

Further reading:
Sidetracks - Science Fiction Edition!
Summary of comments for SF Signal's Russ Pledge post
Everyone Can Promote Equality In Genre Writing
How to Suppress Women's Writing

eta - 3/9/12 3:15P.M.: Going forward, to leave anonymous comments on this post you must sign your comment with the name you use online or a name created specifically for commenting across this post. Any non-signed comments will be screened upon discovery. We will not engage with unsigned anonymous comments.

Re: very interesting

Date: 2012-03-08 10:17 pm (UTC)
From: (Anonymous)
I agree with the sentiment of what you are saying. And I also agree that the overall trend that encompassing so many male bloggers is the bigger deal.

However, I stand by my curiosity and emphasis that a more detailed and expert statistical analysis would be more revealing. Just on the example that I gave and you mention - assuming a blogger reviews only books sent from a publisher, if a publishers send books where 90% of them are by male authors, then that blogger is going to be heavily skewed toward male authoers. Even if it's only 60%, it's (probably) enough to be statisitcally significant for an analysis.

I'm not trying to sound defensive (actually, I'm trying for the opposite). So, I'm not trying to enable an argument that 'it's all the publisher's fault' or anything similar. I'm just advocating for not misusing the statistics.

Neth Space

Re: very interesting

Date: 2012-03-08 10:56 pm (UTC)
bookgazing: (revolution)
From: [personal profile] bookgazing
'I'd love to see what a trained statisical expert would think of these numbers.'

I'm pretty sure that Renay would be very keen for people with more rescources than she had available to look at the data and expand on it. I am also very interested to see people do that, because more data and more analysis can only increase our understanding. Trained statisticians please apply. Renay has begun the conversation very well, lets see it continue.

'How would one control for various important factors? [for example - how skewed are my reading habits by the books that are sent to me and the gender of the writers publishers choose to send me? further, does the gender of the publicist sending me the books play an additional factor?]'

It really doesn't matter whether books come from publishers, or whether a blogger buys them. The results are the same, more male authors are reviewed than female authors. This project is about checking the gender equality of reviews, not apportioning blame or even assigning sexism (unconcious or concious) to any bloggers who have reviewed more books by male authors than female authors. Analysis and results stand seperately, each valid, each suggestive. However, while the analysis is important, the factual results show the concrete reality of which gender is being given important visibility and which gender is not, as well as clearly showing which gender is reviewing more books by women. If you want to search for motives analysis is the way to go (and like I said I would really love to see all kinds of discussion on that, as long as I can weigh in with my own thoughts too), if you want to look at the stark substance of gender equality in blog review numbers and the involvement of a sample of male bloggers in creating these gender review numbers then the answers are in the graphs.

Re: very interesting

Date: 2012-03-08 11:17 pm (UTC)
From: (Anonymous)
@helloladies and @bookgazing

I agree that the simple stats here show an very clear correlation and I don't disagree that that in and of itself is very usefull for the discussion at hand. And I certainly don't disagree that the numbers from my own blog (which is T in the graph) clearly show me as being pretty much dead-on with that correlation. It's just that I'm a scientist at heart and I immediately start asking questions. What are some of the factors controlling this? How influenced are bloggers by what publishers send them? etc. etc. etc. There are so many great things that could potentially come out of a full on statistical analysis and it would be very interesting to me personally.

In the mean time there is certainly a pretty clear lesson to be learned - dude, we need to read more women writers and blog about them.

Neth Space

Re: very interesting

Date: 2012-03-09 07:17 am (UTC)
From: (Anonymous)
"What are some of the factors controlling this? How influenced are bloggers by what publishers send them?"

This really interests me and connects with the point Renee made about how the books we review are a choice. In that sense why should what publishers send us be a factor at all if ultimately, the choices we make about the books we pick to read and review for our blogs are (or really should be) OURS? If publishers are only sending us books by male writers, perhaps it is worth asking them for more variety? From our experience with The Book Smugglers, publicists usually send us lists and catalogues for us to pick from and they have a variety of authors - male and female - in them. They are usually also happy to send things we request. In all fairness though, I know sometimes it is not that easy and at times, we have been frustrated by what we have received from publishers. This was easy to deal with though, by contacting them and saying "x types of books don't interest us". We have also requested a few publishers to remove us from their lists.

I feel like this should be obvious but there are other options to what publishers send - buy the books you want to read and if you can't afford that, there is also the library.

Ana (The Book Smugglers)

Re: very interesting

Date: 2012-03-09 02:26 pm (UTC)
From: (Anonymous)
I've noticed that, as a female blogger, I tend to get put on a lot UF fantasy lists and that what is sent to me is definitely more female-oriented. The queries to my male-reviewers are totally different. So it would seem that the publishers are paying attention to the gender of the blogger and sending material they think is more likely to be reviewed by that person.

Re: very interesting

Date: 2012-03-09 02:27 pm (UTC)
From: [personal profile] sqt
Oops. Didn't mean to post as anonymous. That last one was me.

Re: very interesting - directed to Ken

Date: 2012-03-09 09:25 am (UTC)
bookgazing: (Default)
From: [personal profile] bookgazing
Ken, after the first VIDA count, Bookslut ran a set of posts looking into what real life contributing factors might have influenced the stats:


VIDA have provided their own critique of some of these ideas:


and there are many articles that continue the conversation of stats and analysis:


Although this does not relate directly to Renay's project, I think the theories and rhetoric advanced after VIDA 2010 is likely to be relevant to any further investigative analysis of the stats Renay has produced, so I suspect this is the kind of information you'd be interested in. And Ana has some great thoughts above.
Edited Date: 2012-03-09 09:25 am (UTC)


Lady Business welcome badge

Pitch Us!
Review Policy
Comment Policy
Writers We Like!
Contact Us

tumblr icon twitter icon syndication icon

image asking viewer to support Lady Business on Patreon

Who We Are

Ira is an illustrator and gamer who decided that disagreeing with everyone would be a good way to spend their time on the internet. more? » twitter icon tumblr icon AO3 icon

By day Jodie is currently living the dream as a bookseller for a major British chain of book shops. She has no desire to go back to working in the real world. more? » tumblr icon last.fm icon

KJ KJ is an underemployed librarian, lifelong reader, and more recently an avid gamer. more? » twitter icon tumblr icon AO3 icon

Renay writes for Lady Business and co-hosts Fangirl Happy Hour, a pop culture media show that includes a lot yelling about the love lives of fictional characters. Enjoys puns. more? » twitter icon pinboard icon tumblr icon

Susan is a library assistant who uses her insider access to keep her shelves and to-read list permanently over-flowing. more? » twitter icon pinboard icon AO3 icon


Book Review Index
Film Review Index
Television Review Index
Game Review Index
Non-Review Index
We Want It!
Fanwork Recs
all content by tags

Our Projects

hugo award recs

Criticism & Debate

Indeed, we do have a comment policy.

What's with your subtitle?

It's a riff off an extremely obscure meme only Tom Hardy and Myspace fans will appreciate.

hugo award winner
Powered by Dreamwidth Studios