AO3 News

Post Header

Published:
2011-10-11 08:35:12 UTC
Tags:

Have you ever wondered about what it is tag wranglers do? Are you thinking about volunteering as a wrangler? Do you have a question about tags on the Archive of Our Own? Is your fandom in need of some temporary assistance? The Tag Wrangling Committee is hosting an open house! This is a drop-in session where you can ask us what's on your mind, or just have a chat about tags. We'll also have some AO3 invites to give away.

All are welcome! The chat will be held on Saturday, 15 October at 22:00 UTC (what time is that where I live?) in OTW's public chatroom on Campfire. The chatroom can be accessed at: https://fanarchive.campfirenow.com/e79cc

The Tag Wrangling Committee maintains and administers the curated folksonomy system within the Archive of Our Own, assuring accessibility, diversity of fannish expression, descriptive practices, and a high level of user ease.

Comment

Post Header

Published:
2011-10-10 20:59:29 UTC
Tags:

The AO3's parent org, the Organization for Transformative Works, is running its biannual membership drive!

Once again we've reached that point in the year where we ask for your help! We appreciate the ways in which you help us day after day, year after year, since the Organization for Transformative Works would be nothing without its volunteers, staff, and members. Our projects exist because you've dreamed them, asked for them, and built them. And now we're hoping you can help us build the OTW's membership.

Membership ensure voting privileges in the November Board election, so now is a great time to renew your membership or become a member for the first time. We think OTW Membership is pretty nifty but if you have questions, we have answers -- and feel free to contact us about anything not answered in the FAQ.

Between now and October 16th we ask you renew your membership and tell a friend or two about the OTW. Use the image and link below and spread the word!

OTW logo: red circle with an arrow. Text reads: Organization for Transformative Works Membership Drive October 9-16, 2011
<a href="http://transformativeworks.org/how-you-can-help/support"><img src="http://transformativeworks.org/sites/default/files/2011-fund-drive.PNG" alt="OTW logo: red circle with an arrow. Text reads: Organization for Transformative Works Membership Drive October 9-16, 2011" /></a>

Comment

Post Header

Published:
2011-10-07 15:17:37 UTC
Tags:

Happy Ada Lovelace Day from everyone at the Organization for Transformative Works!

Celebrating women in technology is a subject close to our hearts: when the OTW came into existence in 2007, one of our major motivations was the desire to give fans control of the tools and infrastructure which support fannish creativity. The predominately female fannish communities from which the OTW emerged have a long history of mastering new skills and sharing expertise for fannish pursuits — the vidders of the 1970s were pioneering mashup techniques decades before they became trendy! — and we want to extend that skill-sharing to the creation of a fan-owned home that welcomes all fans.

The vast majority of OTW volunteers identify as female, and the amazing things our teams have achieved demonstrate that they all deserve to be considered tech heroines! Below, we highlight the work of our tech-focused teams and the individual voices of some of our staff and volunteers.

Archive of Our Own

The AO3 is the major tech project for the OTW, and is supported by several committees and volunteer groups: Accessibility, Design, & Technology; Systems; Support; Tag Wranglers; Coders; and Testers. We're one of the largest female-majority open source projects in existence, and we're proud that in less than four years we've developed from nothing more than a cool idea to become a thriving site with more than 23,000 users.

Last Ada Lovelace Day we polled AO3 volunteers to find out a bit more about them, and we thought we'd repeat the experiment this year. The charts below give a summary of their answers:

Bar chart showing the gender identifications of AO3 volunteers: Female - 83%, Male - 12%,  Other -25%.

Bar chart showing the capacities in which people have contributed to the project: A coder - 29%, A designer - 15%, A tester - 44%, A tag wrangler - 49%, A support team member - 20%, A docs member - 7%, A systems member - 15%, Other - 37%

We're still very definitely a female-dominated project; however, we're interested to note that since last year the number of volunteers who identify as male has increased by 10%. We think this reflects the fact that we are focused on making a welcoming and supportive environment for people to gain new skills. As Skud pointed out in hir 2009 Oscon keynote, making a project welcoming for newbies is particularly beneficial to women — who are often excluded from traditional tech contexts — but that doesn't mean it becomes less welcoming to people who aren't women!

Not all the contributors to the project are coders or sysadmins; the AO3 also relies on the work of testers, tag wranglers, support staff, designers, and docs writers. We value their contributions just as much: a tech project is about more than lines of code, and without them the AO3 wouldn't exist.

A key part of our goal is giving fans (whatever their gender identity) the skills to build the tools they want to use. We were super-proud to see some of the fruits of this mission during the recent Delicious debacle, when fannish talk quickly turned to "We should build our own bookmarking service — if the AO3 could do it, so can we!" Our volunteers have achieved so much — they're all tech heroines (and heroes)!

The AO3 team would like to give special thanks to one particular tech heroine — Sidra, Systems co-chair and primary guardian of the servers for the AO3. The Accessibility, Design, & Technology Committee have posted a separate post celebrating Sidra's awesome work.

Fanlore

Another major technical undertaking for the OTW is Fanlore, our fannish history wiki. Since Fanlore is built on existing MediaWiki software rather than a custom-built application like the AO3, the tech aspects of this project are not as immediately obvious, but they are just as important. Our Wiki staff have learnt to maintain and use the MediaWiki software, creating custom templates, investigating new software modules, and getting to grips with wiki maintenance. They are awesomely assisted by our Systems team, who installed the software on our servers and keep everything running smoothly (we love you, Systems ♥).

Fanlore is celebrating Ada Lovelace Day with a new challenge on Women Characters, Science Edition! Why not create a Fanlore article about your favorite female character who is a scientist, engineer, or mathematician? Tell us about your fannish experiences with these characters — the women themselves, the relationships they’re in (het, lesbian, canonical, fannish, etc.), the fanworks they star in — whatever you can think of! You can stub out a new page, or add a sprinkle of information on an existing page.

Systems

If you've read this far, you've probably realized that Systems is involved in every OTW project. They tend the AO3 servers; install software for Fanlore, Transformative Works and Cultures, Open Doors, and the main OTW website, plus the software that helps us process donations and manage volunteers; and set up the mailing lists that help all the committees and volunteer groups do their daily work. The heroines and heroes of the Systems committee work largely behind the scenes to keep our technical infrastructure running smoothly, and the entire OTW benefits enormously from their dedication and expertise.

Webmasters

The Webmasters are another committee whose work is spread among a wide variety of projects. They maintain the OTW's main website, the Open Doors site, and the Elections site, manage our donation processing software, serve as layout coders for Transformative Works and Cultures, design styles for the OTW's social media accounts, and manage media hosting for various internal projects. To date, the Webmasters have all been women, and have been largely self- or peer-taught in the technical skills they use.

Some thoughts from our volunteers

In a post that celebrates women doing it for themselves, it seems appropriate to close with some thoughts from our volunteers, as they reflect both on their own work and on that of other women they admire. We'll be adding links to individuals' blog posts at the end of this post throughout the day.

It's exciting to work in teams that are overwhelmingly female. I really like the testing parties, as it's a little confusing and intimidating to try to work from written descriptions. I joined to support an organization I trust and approve of, and to get some practical tech experience. I just started volunteering a few weeks ago, so not much to say yet!


Sometimes I have conversations about servers, code, etc and I realise that former!me wouldn't have understand ANY of it. I've only learnt enough to contribute a tiny amount of code, but I am able to be a fully functioning member of AD&T because I have absorbed enough to be able to take part in these conversations as a useful laywoman.


I like finding interesting bugs and feel good whenever I find one before it hits Beta.


I like that the archive tries to accommodate a variety of people and systems instead of saying: get browser x with y settings or we don't care about your problems.


I love wrangling big fandoms with lots of problems and characters-shared-between-fandoms, it's a big undertaking but it's nice to see everything all neat once you're done!


Since I come from a background of relatively no coding, it has been really exciting to submit my bug fixes and see my changes on the archive! The whole experience has been really rewarding!


Since beginning my work with the Archive, I have improved my computing skills dramatically. I have learned a great deal about linux and switched to a more complex, text-based distro. I have gained an exceptional amount of skill and confidence with unix commands and bash. I now have an understanding of how the Archive is put together via Ruby on Rails, and that understanding deepens and develops with every issue I work on. This has been an amazing experience and I am excited to keep learning and growing as a coder!


I've never been part of a mainly women-identified group before, and it's really been rewarding for me in so many different ways. I'm so proud to be part of the OTW!


It combines two of my dearest hobbies: Coding and fandom. Both Open Source people and fandom people build great, communicative communities with lots of collaboration, and if you put those two together you get fun squared. :D It's really great to share more than the passion for coding with my fellow coders, so when I'm in a phase where I code less in favour of writing or squeeing over a new shiny fandom, it's never really off-topic, thus making it easier to keep in touch with coding stuff.


[Something I'm proud of accomplishing.] Dragging a committee up from its bootstraps at the project's launch, in such a way that it perfectly well survived (and prospered after) my own burnout-related crash and burn.


I really love it. I quit grad school in a blaze of disillusionment and have been unemployed and completely at sea in my life since, and it's been really heartening to have something I can contribute to in small ways, especially something that's part of fandom, which has been such a wonderful aspect of my life for so many years.


It is one of the more nurturing and family-building projects/organizations I've seen.


It's a delight to work on a project where people not only don't jump to assumptions about you, but where people are supportive even if you make the smallest contributions.


ruby metaprogramming! redis! There is just nothing quite so fantastically satisfying as working with a smart and dedicated and passionate team on a project that we all actually use ourselves and value deeply as a result.


I've really enjoyed being AD&T training lead, running sessions for new people to learn how to code from scratch, and mentoring them as they advance. It's so rewarding to see people gaining new skills, and particularly when you know they've previously been excluded from opportunities because of their gender or disability, e.g. by lack of part-time courses that can fit around childcare or flare-ups.


I'm *so excited* to be part of the team that's creating the Archive that I love so much. I think fandom is amazing to have worked so hard together to create the Archive.

Mirrored from an original post on the OTW blog, where we'll be collecting links to Ada Lovelace Day blog posts from OTW members throughout the day.

Comment

Post Header

Published:
2011-10-07 14:24:37 UTC
Tags:

Happy Ada Lovelace Day ! This year, Accessibility, Design and Technology staffers and everyone who works on the Archive of Our Own have more reason than usual to give thanks to one particular tech heroine - Sidra, keeper of the AO3 servers. Sidra is the co-chair of the Systems Committee (along with Arrow, another tech heroine ♥) and a long-serving staffer on AD&T. She's been with us since the beginning and an impressive 17% of code commits to the Archive are hers; we don't have stats on exactly how many lines of code she's committed, but it's safe to say that the answer is a lot! Even more importantly, Sidra is the member of Systems dedicated to looking after the AO3 servers - when we're deciding what servers to buy, deploying new code, or dealing with site performance Sidra is right there at the heart of the action! Sometimes this involves being woken up at ungodly hours - if the Archive goes down, it's Sidra who we call in the middle of the night! We are in awe of her ability to climb out of bed and work her magic on the servers!

This past weekend was a compelling reminder of just how awesome Sidra is, as she battled mysterious loads on the servers, tweaked settings, and managed to restore the AO3 after some alarming and unexpected overloads. We know how incredibly hard she worked (and continues to work) to ensure that we can keep serving up delicious fanworks to our users - thank you, Sidra!

Our Systems team is a fantastic group of people, both men and women, but we're particularly proud to say that the Archive has a female sys-admin, since there are relatively few women working in this area. In fact, Sidra also deserves credit for helping to pave the way for women in tech: she entered the tech industry in the 1980s and is now a senior sys-admin in her professional life. Sidra is definitely proof that women can not only work in technology, they can kick ass at it! (Arthur C. Clarke tells us that any sufficiently advanced technology is indistinguishable from magic, but we're not at all convinced that Sidra's ability to vanquish recalcitrant technology isn't actually magic.)

Thanks Sidra - a fannish tech heroine! The Archive definitely wouldn't be the same without you! ♥ ♥ ♥

Comment

Post Header

Published:
2011-10-05 21:23:25 UTC
Tags:

The Accessibility, Design and Technology committee oversees technology-related projects within the OTW. Currently we are responsible for designing and building the Archive of Our Own. Our regular meeting updates keep you informed about developments on the AO3!

This was a nice busy meeting - 11 of our 14 staffers were able to attend! Managing different timezones and meeting times, as well as other commitments, means that it can be challenging for everyone to attend every meeting, so it's nice when lots of us make it (although some of us were off celebrating birthdays *looks about shiftily*).

Meeting highlights!

Improving our code commits and deploy process

This is an important subject to us, although one that isn't immediately apparent to our users! When a coder writes some new code, they 'push' it to our code repository and then submit a 'pull request' to ask for it to be included in the main project. Another coder then looks at the code and (if they don't spot any problems) merges it into the main project; when we have a few changes to that, we put all the code on our Test archive for testers to work on. Ideally, this whole process should be pretty quick - it's nice for coders to have their code accepted quickly (or sent back for changes) and it's nice for users if we get new code tested quickly and thus out onto the live site!

Right now, this process isn't as fast as we'd like it to be. Unfortunately, our automated tests currently take a significant amount of time to run, and that causes a bit of a chain reaction: coders run the tests as they work, run all the tests before they push their code out, run all the tests before they approve somebody else's code, and have a job set up on our development server to run all the tests every day and email the whole team with any failures. While we work on making the tests faster, we're hoping to do some reshuffling, where we reduce some of the duplication and the time required to merge in code, while doing more manual review. We'll be aiming to get code merged in and out to our testers on a quicker and more regular basis, so that ultimately users will have faster access to fixes and new features.

Mentoring

Ana and Jenny have been working on some ideas for improving mentoring for coders. We have lots of informal mentoring - one of the great things about coding with the OTW is that people on the project are so generous with their time, and are happy to answer questions and support new people. However, sometimes people feel awkward about asking for help at busy times, or they want some more intensive one-to-one mentoring to help work out a big project or get over a specific learning 'hump'. It can be challenging for us to provide this kind of mentoring, because we have such a small number of developers, all of whom are volunteers and many of whom are also learning this stuff as they go! The new plans are designed to make it a bit easier, and to make people more comfortable about asking for a specific mentor if that's what suits their style.

Front-end changes

We're continuing to work on changes to the HTML and CSS of the site. lim has been radically reworking the CSS to make it more flexible and much much easier to skin. Meanwhile, we have been working through front-end issues logged in our Google code issues project to identify old issues which still need attention, and close the ones which are now obsolete. Thanks to everyone who has been working on this!

Latest deploy

We deployed Release 0.8.8 this weekend, which included performance fixes, lots of JavaScript changes, and a fix for the annoying '500 error on download' issue. You can see the full list of changes in our release notes.

As most of you will have noticed, this deploy did not go as smoothly as we'd have liked! One of the problems with performance enhancements in particular is that they are very hard to test, since there's no way of reproducing the exact conditions of the live site. So, we were alarmed when after our big deploy (full of performance enhancements) the load on the site went UP instead of down. A whole lot of investigation and work later, our wonderful Systems team were able to get everything under control. The site went up and down quite a few times while all this work was going on, but the good news is that the server load is now lower than it has been for the past six months! The site is running correspondingly faster. We have a list of additional changes to make - one of the useful side-effects of the problems was that they highlighted some things we could improve - but we're really happy to see the 502s banished at last! A million thanks to Sidra and james_ who did a LOT of work to fix everything!

News from our sub-committees

  • Coders did lots of last-minute work to get everything ready for the deploy, then the senior coders and Systems staff worked like CHAMPIONS to resolve the issues that showed up during the deploy.
  • Testers also worked like champions getting this deploy ready for primetime. Jenn, Kylie, Seph, Tai, Weimar, xparrotx and Zebra, you all deserve a special shout-out after the hard work you put into sorting out this monster deploy!

News from our sister committees

  • Support continue to keep up with tickets after their amazing work overcoming the backlog. They are happy to answer all kinds of questions about the Archive, whether you think you've found a bug or you just want clarification on something - please don't feel shy about asking if there's something you'd like to know!
  • Tag wranglers will be holding an open house on Saturday 15th of October at 22:00 UTC. This will be a drop in session where you can find out about volunteering as a wrangler, ask about tags, help out with fandoms we don’t have wranglers for yet, or just have a chat about what we do. It will be held in the OTW Public Discussion Room - if you've ever been curious, drop by and find out more!

If there are things you'd like to do or say, please share them in comments, via the AO3 support and feedback form, by volunteering, or in whatever medium you feel comfortable with. Everyone is welcome to this party!

This meeting round-up by Lucy.

Comment

Post Header

Published:
2011-10-03 01:02:51 UTC
Tags:

Welcome to Release 0.8.8! Many thanks to everyone who helped with this release: Bingeling, Elz, Enigel, Erda, Jenny, Karen, Laura, Lucy, Mumble, Naomi, Rebecca, Sidra, Tel, and Velocitygrass contributed code, which was tested by our awesome testing team: Jenn, Kylie, Seph, Tai, Weimar, xparrotx and Zebra. This release required extra attentive testing because we were making site-wide changes to all of our Javascript code - we did our best to review everything across a variety of browsers and operating systems, but let us know if you spot any problems.

Highlights!

Performance

A few days ago, we gave a run-down of some of the challenges we've been facing with performance on the site. The coding team and the systems committee have been brainstorming ways that we can make things better, and we've started by moving some our most read- and write-heavy operations off our main MySQL database and to a system called redis, which allows for very fast reads and writes as it stores information in memory. Additionally, you may have noticed that we were up and down a bit this weekend - we were moving things around and also doing some tuning of our main database. Right now, the database is the primary bottleneck for our system and the reason you may run into 502 errors, so we're hoping that all of this work pays off in greater stability and fewer errors for users. Apologies to those of you who got stranded in the middle of multi-chapter works! As fellow fans, we understand exactly how inconvenient that is. And special thanks and ALL the cake to Sidra and james_ from Systems, who did great work getting everything back in order!

Javascript changes

When we started coding the archive, a Javascript library called Prototype was the standard for Ruby on Rails, and Rails had a lot of built-in helpers that utilized it. Over time, Rails has become more flexible about allowing you to switch libraries (and has even switched defaults itself), and we've been interested in making the jump to jQuery ourselves for a while now. Enigel from our coding team finally finished the epic work of migrating everything over! This was a big undertaking, but it's also a big help to us going forward, in terms of volunteer skills and available resources. And if you're a jQuery wizard who'd like to help out, there's never been a better time to volunteer! (hint, hint)

Filtering improvements

One thing that's been requested a lot is the ability to only view completed works when you're searching and browsing, and we've finally enabled that. (It may take a little time for our search index to catch up with it, since that takes a good bit of time to finish running.) It's also now possible to filter by language more easily, and to sort works by the date they were first added to the site. We'll be continuing to improve our browsing interface in the next few months, but we hope the new features will help you find what you're looking for!

New icon color for Teen-rated works

In response to user feedback, we changed the color of the Teen rating icon so that it's no longer the same as Mature; if you're browsing on a phone or a small screen, the letters can be a little tricky to make out. If you're not a fan of the new color, we have a site skin that you can use to restore the original orange icon.

No more 500 errors on downloads

One of our most pernicious bugs has been the error that you get if you happen to be the first person to download a work in a particular format. We believe we've resolved the problem there! It's a tricky one, so please do let us know if you run into any similar errors.

Known Issues

See our Known Issues page.

Release Details

Features

  • Switched to jQuery from Prototype for all Javascript-related functionality
  • Searching and filtering by WIP/Complete
  • Seaching and filtering by language
  • Sort by Date Updated/Date Added
  • New icon color for Teen-rated works
  • Optimized the History feature that keeps track of all the works you've viewed and allows you to mark works to read later

Bug fixes

  • Admin:
    • the interface to update the Known Issues was broken and has now been fixed
    • improvements to the Admin banner feature, for users with and without Javascript
  • Accounts:
    • there was a loophole where you could join the invite request queue even when it was temporarily turned off, this has been fixed
    • changed the error message wording for problems when activating a new account
    • added a link back to "Edit My Profile" from the "Change Password" and "Change Email" pages
  • Works:
    • trying to download a work for the first time would throw up a 500 error while the file was being generated in the background, this has now been fixed
    • trying to approve a related work from the link in the notification email would sometimes give an error 500, this has been fixed
    • you can now assign a language to an external work, especially helpful when you're posting a translation of said work
  • Collections & Challenges:
    • tweaked the listing of Open Challenges: a gift exchange will appear if the "Signup open?" box is ticked and it falls between the "Signup opens:" and "Signup closes:" times given by the owner; prompt memes will show up in either case
    • added options for owners and moderators of a challenge to delete prompts and edit challenge signups if necessary
    • the option for Challenge owners to download signup data has been adjusted to produce .csv files (although these are tab-separated rather than comma-separated) instead of potentially broken .xls files. (Please note that you might have to tell your spreadsheet program that this is a tab-separated file before viewing.)
    • changed wording on settings button from "Submit" to "Update"
    • there was a bug that would cause bookmarks of unrevealed works to disclose information, this has now been fixed: bookmarks will default to the "Mystery Work" blurb if the work hasn't been revealed yet
    • the "Add to Collection" button was broken and has now been fixed
    • when creating a challenge and selecting Gift Exchange from the dropdown, this selection would not be retained if you ran into an error (e.g. because of missing information) and had to submit the form again, this has been fixed
    • the page title on static work pages now reflects the title, fandom, and author of the work
  • Prompt Memes:
    • fixed the issue with autocomplete not working beyond the first prompt form in a signup
    • fixed capitalisation in prompt meme signup wording
    • when editing a prompt meme signup, you will see a list of jump links at the top of the page to quickly get to the prompt you want to edit (especially helpful if your signup consists of twenty prompts); you will now find a brief summary of your tags beside each prompt and a helpful button if you just want to add another prompt
    • the link to the list of prompts is now visible in the navigation for logged-out users as well; the list of claims is still only accessible for logged-in users
    • added pagination links for long prompt lists
    • tags in the prompt overview are now a comma-separated list instead of one-per-line
    • added a link to the prompt overview to edit a single prompt without having to open the whole signup
    • when signing up for a prompt meme, you can pick one of your pseuds to sign up with under "General Information"; there was a bug where this section would show up even if you only had one pseud, this has been fixed
    • when creating a challenge, there's a section referring to assignment notifications; this has been reworded to only apply to gift exchanges and not prompt memes
    • improved caching of information in prompts and signup overviews
  • Comments:
    • the character counter now works when posting and editing a comment
    • the link to leave a comment from within a downloaded work or a static page in a collection was broken and has now been fixed
  • Tag Wrangling:
    • there was a rather thorny issue with tag relationships and filters (for example, when a tag's "synonym of" relationship was changed, it wouldn't always show up in filtered work lists for the new tag), this has been fixed
    • fixed a bug that prevented the autocomplete help from popping up in Meta Tag fields
  • Misc.
    • some unused code has been tidied up to speed up URL routing
    • fixed the issue where only the first frame of an animated icon would display
    • removed obsolete "Limited CSS properties" text from the skins form and fixed a dangling sentence fragment in the help box
    • in the post new/edit work form, the text box to create a new series was misaligned in IE7, this has been fixed

Thanks to mumble for compiling the list of fixes and new features and to Lucy who normally writes up our release notes but is currently off celebrating her birthday! --Elz

Comment

Post Header

Published:
2011-09-29 07:34:38 UTC
Tags:

As most people are sadly only too aware, the Archive of our Own has been experiencing some performance issues recently. The sad 502 page is showing up increasingly frequently, to the frustration of everyone concerned! We have also had a couple of instances of downtime. We are working on the problem, but we know that our users are wondering what is going on, so we want to give a bit more information on what is going on behind the scenes.

Why is this happening?!

The main reason for 502 errors is the sheer load on the site. We've seen a massive increase in user numbers over the past few months - on Sunday 25th September we had 105,000 visits, of which 46,000 were unique visitors. These visitors racked up an impressive 575,000 page views! This is now a pretty average day for us - we're thrilled that so many people are enjoying and using the site, but we've expanded a bit more rapidly than we expected, so it's a little bit challenging for our servers.

Our recent server outages were caused by a problem with the servers themselves; our Systems team are tracking this to its source and making some changes to fix it.

Why do I sometimes get a 502 as soon as I click on a page?

Since 502 errors are associated with site slowness, it can be a bit unexpected to get a 502 as soon as you click. The reason this happens is that our servers are set up to keep an eye on how many requests are in the queue for them to handle. If there are more than a certain number, then the chances that the page will time out before it can be delivered are high. So, the server gives the 502 page right away instead so that you don't wait a long time only to be disappointed.

Didn't you just buy new servers recently?

Yes, pretty recently! Thanks to the generosity of fans who donated to our parent Organization for Transformative Works, we were able to purchase 5 new servers at the beginning of the year. These are actually doing a great job (if we were still on our original two servers, the site would have keeled over completely by now). However, the speed of our expansion means that we will need to purchase more servers sooner rather than later. When we do this, we'll also have to make some big changes to the underlying infrastructure - one option is shard the database, which means we'd split it up into separate chunks so each server only has to deal with a bit of it (if you're interested in the problem of scaleability and how sites deal with lots of users, this post on how LiveJournal handled it is a good read). We're actively researching now to figure out the best way of doing this and the kind of hardware we'll need.

What are you doing to fix the problem?

One reason the high loads on the site are having such a drastic effect is that some of our code is really optimised for fewer users (since we weren't expecting to have so many this soon!). So we're currently making a number of changes in the code to make it more efficient and reduce the number of database reads/writes (these are the things that tend to cause slowdowns) and updating the application software to one which should be more efficient. Our Systems team are also investigating updating the server operating system which might be necessary to take care of the issue which caused the server outages (it's hard to reproduce the conditions on the real site in our test site). The current set of new code has required intensive testing, which means we haven't been able to roll out the changes as quickly as we would like, but we hope to make these updates in early October. Longer term, we'll be doing some sustained work on scalability so that we can restructure things and buy more servers.

What can I do to help?

We really appreciate how understanding users have been about the problems - it helps our team a lot! If you want to do more and you have some financial resources, then a donation to the OTW will help ensure that we are able to continue expanding our servers (money donated to the OTW supports all OTW projects, but the vast majority is spent on servers and other running costs for the AO3). Finally, if you happen to be an experienced sys-admin or database expert - especially one with experience in performance tuning and scalability - and you would be willing to donate some time, we would welcome the additional expertise and support. (If you're interested, get in touch with our Volunteers and Recruitment Committee.)

And finally...

We'd like to say thanks to everyone who has supported the site in various different ways - while the 502 errors are HUGELY annoying, they do show that people enjoy using the AO3. Thanks also to all our users for your patience while we get to grips with our new success and deal with the performance problems. Finally, an HUGE thank you to our Systems team, who bear the brunt of the work on these issues - the wonderful Sidra has been woken in the night more than once to deal with server issues, and we appreciate her hard work and dedication more than we can say. ♥

We are working hard to resolve the performance issues, and we'll keep updating users as we have more news. The latest site status updates can be found on our Twitter AO3_Status.

Comment

Post Header

Published:
2011-09-27 22:05:30 UTC
Tags:

As the majority of fans have already noticed, AVOS, the new owners of delicious, launched the new version of the site today. Fans have also been quick to point out that the new site has lost a substantial amount of functionality, including some features particularly heavily used by fans, such as tags with a / in them, tag subscriptions, and networks. The new site is also incompatible with the scripts used by many fannish newsletter compilers. Since today was the site launch, we're hopeful that many of these issues will soon be fixed (we're certainly familiar with the idea of making lots of rapid changes after new code is released!). However, in the wake of all this, many fans are looking for a new fannish bookmarking service. Right now, the Archive of Our Own doesn't have enough features to provide a delicious replacement, but we wanted to let you know what we do have and what we have planned for the future.

Bookmarking on the AO3

Registered users of the AO3 can use the site to create bookmarks, and as part of our forthcoming revamp of site CSS, coders Naomi and lim are making some improvements to the bookmarking form which should make it easier to use. At present, bookmarks offer the following functionality:

  • You can bookmark works hosted on the Archive directly from the Archive interface, and your bookmark will include the tags the author used (so you don't have to re-enter fandom and pairing tags, for example).
  • You can bookmark works which aren't hosted on the Archive ('external works'). We have a bookmarklet which makes this easier - you can grab it from the external bookmarks page.
  • You can tag your bookmarks. Bookmark tags can be multiple words and can contain a range of punctuation, including slashes (they can't contain commas, however, since this is what we use to separate bookmarks).
  • You can make a bookmark private or mark it as a rec.
  • You can filter all the bookmarks on the Archive by tag, and you can choose to search for just recs.
  • You can see how many other people bookmarked a link or work, and view their bookmarks.

We know that there are lots of improvements which are needed to make AO3 bookmarks really useful. Most importantly, there's currently no way to filter your own bookmarks. We know this severely limits the usefulness of bookmarking on the AO3 - we've wanted bookmark filters since we launched, but it has proved tricky to achieve. The good news is that AD&T co-chair Elz has now done lots of work to lay the foundation for bookmark filtering - we hope we'll be able to build on this work and launch the feature within the next few months.

Planned features for the AO3

We have a few features planned for the AO3 in the nearish future which will improve bookmarks and otherwise replace some of the functionality fans were using delicious for:

  • Bookmark filtering - as mentioned above, some of the work for this has already been done, so it's finally on the horizon.
  • Improvements to anonymous posting and prompt memes - we know that a lot of kink memes rely on delicious to record and categorise prompts and fills. If you host a prompt meme on the AO3, then you can already filter and sort prompts and fills by tag or status. We've started work on true anonymous posting (where there's no chance of anyone ever seeing the link between an anon work and your ID unless you reveal it), which we hope will make the Archive a more attractive place for kink memes. We continue to work on the prompt meme code to make it even spiffier.

Longer term, we also have plans for other bookmarking features:

  • Ability to import bookmarks from other sites and to export to a file.
  • Recs post / newsletter feature so you can easily compile a list, add your comments and generate a nicely coded post.
  • More subscriptions features, so you can subscribe to a particular user's tags, a particular rec list, etc.

Right now, the number one priority for our coders is improving site performance and stability (we'll be posting separately about this in the next day or so.) Our coding team is small, so we can't implement improvements right away, but bookmarks are definitely not forgotten. We'd love to know if you have ideas about how our bookmarks could be improved, or if there are particular areas you'd like us to prioritise - please leave your thoughts on this post!

If you feel passionate about bookmarks on the AO3 (or anything else about the site) we always welcome volunteers! If you already know how to code and feel moved to jump on board to revamp bookmarks, we'd love to have you! (We won't make you wait around and work on other things!) If you want to learn to code with a view to contributing a tiny part, that's also awesome. Finally, we are desperately in need of testers - no experience or special knowledge is required, just a willingness to bang on the site and check everything is working. If you're interested in joining us, please contact our volunteers and recruitment committee and let them know what you'd like to do.

The delicious fail sucks, but we have faith in fandom's power to find creative solutions - and we hope we can be part of it! So while we're aware of and working on some of the fundamental limitations of our bookmarking system, we're interested in hearing your ideas about the future: what new features would be most useful to you? What things would you like to be able to do that other bookmarking sites like Diigo and Pinboard don't currently offer? All ideas welcome - let us know what you're thinking!

Comment


Pages Navigation