Ao3 data dump. · This tool is op The OTW has suggested protective measures like restricting works to AO3 users-only and implemented code to deter large-scale scraping. py at master · cutesytootsie/ao3-data-dump A look at freeform tags over time, using AO3's Selective data dump for fan statisticians. We A user going by "nyuuzyou" on the HuggingFace platform uploaded a dataset a few days ago - containing scraped content from AO3. By any chance do you happen to have a version of the metadata for the AO3 dump that includes kudos count for parts 1-16 of the dump? Only part 17's sqlite has that information at the Data Analysis on a 2021 dataset released by Ao3! Investigating fanworks and fandom behavior over the years - jiljames/ao3_data Data analysis and visualization of 2021 official data release from archiveofourown. We are Hi, I downloaded a couple of the AO3 story dumps, but I couldn't seem to find a certain fic. Experiments with AO3 and Python Tagged with ao3, python Posted 7 January 2017 Recently, I’ve been writing some scripts that need to get data from AO3 [^1]. Capture a web page as it appears now for use as a trusted citation in the future. This process is often referred to as dumping and includes different processes depending on This guide explains how to obtain your PlayStation 3 (PS3) games and then use your backups with RPCS3. With the proliferation of AI tools in recent months, many fans have voiced concerns regarding data scraping and AI-generated works, and how these developments can affect AO3. net and AO3, but should only be attempted after checking The Wayback Machine A fan-created, fan-run, nonprofit, noncommercial archive for transformative fanworks, like fanfiction, fanart, fan videos, and podfic more than 77,100 fandoms | 10,250,000 users | 17,020,000 works The ao3_hits_to_kudos: List all your works in ascending order of their hits/kudos ratio ao3_purge: Deletes saved stats such that there is a minimum time between the remaining ones The This dump contains the first ~500k stories on archiveofourown. This process is often referred to as dumping and includes different processes depending on Fandom Stats is an ongoing project to create open-source tools for "fandom analysis" - data-driven exploration of behavior. Exports your entire Archive of Our Own reading history as JSON data with all details Per their post: 'Selective data dump for fan statisticians' on the 21st March 2021 "We hope to one day be able to provide regular, automatic dumps of this data, but for now, our focus is on other projects. Now with HASTAC 2017 presentation slides! Features: Given a fandom URL You can create a release to package software, along with release notes and links to binary files, for other people to use. Yet another dump of ao3 stories https://archive. This JSON configuration should now allow you to scrape data from your AO3 bookmarks. You can find a tool Feels Family Feels Data dump SHIELD agents - Freeform Competent Tony Stark BAMF Jarvis (Iron Man movies) Protective Loki (Marvel) Sharon Carter & Tony Stark are cousins Tony Stark Has a SHIELD info dump has been made a synonym of SHIELD Data Dump (Marvel). ladyofthelog / ao3-data-dump Public Notifications Fork 0 Star 1 Security Insights Automate your workflow from idea to production This article details a python script that scrapes the fiction text of any subsection of the fanfiction and fan works site: Archive of Our Own. I know that someone dumped a huge backup file of ff. txt. Code to read in, clean up, and manipulate the AO3 data dump from March 2021 - AO3_Data_Dump/Initial_Data_Processing. org and collected every non-user-restricted work posted before 2020-07-17 as well as most of the work's meta data (such as tags). 💬 2 🔁 800 ️ 1387 · Cool Stuff FAQ | Archive of Our Own · We have updated our list of third-party tools, userscripts, and bookmarklets! It's not a complete list, just a selection of things we fou ao3-archivist --username [USERNAME] --cookies /path/to/cookies. - amecreate/AO3-Data-Dump-By-Year Python code for saving the official AO3 data dump into smaller files, filtered by year. A simple Python Archive of Our Own scraper. I'm a statistician and programmer, and I love answering people's questions with data. You can create a release to package software, along with release notes and links to binary files, for other people to use. The first includes information about works: The second provides the key to the tag IDs: We hope to one day be able to provide regular, automatic dumps of this data, but for now, our focus is on other projects. AO3 provided the dataset for analysis to the general public, which was then downloaded and uploaded to BigQuery AO3-Data-Dump-By-Year Python code for saving the official AO3 data dump into smaller files, filtered by year. Please also check our official status Twitter, @AO3_Status ao3scraper is a python webscraper that scrapes AO3 for fanfiction data, stores it in a database, and highlights entries when they are updated. has been made into a downloadable, still avaible Torrent file by the person who scrapped AO3 And all the others are with your actual AO3 username and the total number of pages in your bookmarks. - Issues · amecreate/AO3-Data-Dump-By-Year A web scraper that scrapes, cleans, and exports fanfiction metadata of one’s choice from Archive of Our Own. org/details/AO3_story_dump_continuing number 12 now up. This scraper serves a different purpose, which is to scrape as much information as possible The data also showed that weekends - and specifically weekend nights and Sundays, in particular - in the respective local time zones of America and Python code for saving the official AO3 data dump into smaller files, filtered by year. (2008-2021) Code to read in, clean up, and manipulate the AO3 data dump from March 2021 - mousemode/AO3_Data_Dump Code to read in, clean up, and manipulate the AO3 data dump from March 2021 - mousemode/AO3_Data_Dump Easier access to AO3 data dump I have split up the AO3 tag data into smaller files, in case people want to access smaller subsets of the tags and/or view the data as a spreadsheet. Works and bookmarks tagged with SHIELD info dump will show up in SHIELD Data Dump (Marvel)'s filter. To access In collaboration with @ssterman. I was wondering if you could help? It's a Harry Potter Fanfic by Nocturnememory called Bittersweet We would like to show you a description here but the site won’t allow us. We are proactive and innovative in protecting and defending our A fan-created, fan-run, nonprofit, noncommercial archive for transformative fanworks, like fanfiction, fanart, fan videos, and podfic more than 77,100 fandoms | 10,250,000 users | 17,030,000 works The puts the data provided by ao3 into a (semi-)neat database - ao3-data-dump/import_utility. Learn more about releases in our docs Python code for saving the official AO3 data dump into smaller files, filtered by year. Originally posted on tumblr. We are proactive and innovative in protecting and defending our This guide explains how to obtain your PlayStation 3 (PS3) games and then use your backups with RPCS3. You can either copy the data into a new plain text An unofficial sub devoted to AO3. Unfortunately, AO3 How to recover fic deleted from AO3 that’s NOT on the Wayback machine swissmissficrecs: “futureevilscientist: “Sharing this because I just found out This is a program intended to help you download fanfiction from the Archive of Our Own in bulk. Works and bookmarks tagged with SHIELD Information Dump will show up in SHIELD Data Dump (Marvel)'s filter. (Gaiden and Warriors are excluded because each has With the proliferation of AI tools in recent months, many fans have voiced concerns regarding data scraping and AI-generated works, and how these developments can affect AO3. The username and password are self-explanatory, and the output is just the destination file, such as ao3. BLUF: This method will work for both fanfic. The Archive of Our Own (AO3) offers a noncommercial and nonprofit central hosting place for fanworks. In the meantime, there are a number of tools available From time to time, we get contacted by students, scholars, and people interested in fandom stats who would like to access information about the fanworks in the AO3 database, such as I recently did a web-scraping project on ArchiveOfOurOwn. You know how you can export your MyAnimeList or goodreads . Oddball things like this "statistical data dump" are out there. An unofficial sub devoted to AO3. EDIT: ao3continuing and updateable are compilations of datadumps I've had sitting around a while, ao3's are identical to the previous ones, with the addition of the newer dumps in one place. R at main · mousemode/AO3_Data_Dump The dataset used in this project was obtained from the Archive of Our Own (AO3) website. org, but I couldn't find anything like that for AO3. Amidst these discussions, a commenter on the We would like to show you a description here but the site won’t allow us. Have I Been Pwned allows you to check whether your email address has been exposed in a data breach. We share your AO3 Data Dive This project consists of a dashboard that allows the user to input any tag of their choosing on AO3, then returns graphics summarizing statistics of works containing that tag. Works and bookmarks tagged with Data dump will show up in SHIELD Data Dump (Marvel)'s filter. Table with an updated entry highlighted. It is not an official API. The data comes in two CSV files. - Branches · amecreate/AO3-Data-Dump-By-Year Is there a way to back up the URL of the bookmarked fics and my notes (I do NOT want to download all the fics, just back up my bookmarks). net (FFN), and Wattpad to gather fandom data. ly | Organization for Transformative Works From time to time, we get contacted by students, scholars, and people interested in fandom stats We would like to show you a description here but the site won’t allow us. Easier access to AO3 data dump. Another argument that isn't With the proliferation of AI tools in recent months, many fans have voiced concerns regarding data scraping and AI-generated works, and how these developments can affect AO3. A web scraper that extracts bookmark metadata from Archive of Our Own and saves it to a CSV file. ao3-metadata_full. Note that you must provide the cookies for the AO3 account matching the username. More than 150 million people use GitHub to discover, fork, and contribute to over 420 million projects. 211180 more stories. I have split up the AO3 tag data into smaller files, in case people want to access smaller subsets of the tags and/or view the data as a spreadsheet. All data Author's Note (s): This fic contains references to the likely fall out of not curating a massive file dump of an international intelligence organization with active ops being ran around the world. It also contains An unofficial sub devoted to AO3. txt depending on how you want to use the exported data. This will show up to 2,000 scraped works for most usernames. ao3continuing and updateable are compilations of datadumps I've had sitting around a while, ao3's are identical to the previous ones, with the addition of the newer dumps in one place. This program is primarily intended to work with links to the Archive AO3 Works List (Download) AO3 Works List (Manual) (For Internet Explorer and other browsers that the other version doesn't work for. I'm An unofficial sub devoted to AO3. A look at freeform tags over time, using AO3's Selective data dump for fan statisticians. org/details/AO3_story_dump_continuing ao3-10 is the new one. Extract and analyze your AO3 reading history. ini configurations. org. Work Text: // AO3P - parsing AO3 data dump stats // Written in 2021 by lizard-socks // To the extent possible under law, the author (s) have dedicated all copyright and related and Read here by lizard_socks Statistics on works posted to AO3 for the various Fire Emblem games. I'm looking for a fanfic that was deleted a while ago. However, we don't have a policy against responsible data collection — such as those done by academic researchers, fans backing up works to Wayback Machine or Google's search indexing. In Analysis of AO3's Selective data dump for fan statisticians (March 2021) - Issues · ladyofthelog/ao3-data-dump 💬 133 🔁 2536 ️ 2619 · Most people should use this link to check if they were included in the March 2025 AO3 scrape. AO3 did a data dump a few weeks ago, with loads of information about works, fandoms, and tags. Archive of Our Own 2021 Data Dump Explortory Data Analysis The Archive of Our Own (AO3) is a popular fanfiction archive with over 7 million fanworks, encompassing various fandoms, pairings, and Mergers Data dump has been made a synonym of SHIELD Data Dump (Marvel). md at main · jiljames/ao3_data Selective data dump for fan statisticians bit. We share your To use this app, you need to first export your AO3 history using one of these options: Install the AO3 History Exporter browser extension for Firefox Go to your AO3 reading history page Click the "Export The AO3 scraper by radiolarian scrapes IDs from the search results and then scrapes the individual works. Learn more about releases in our docs Looker Studio turns your data into informative dashboards and reports that are easy to read, easy to share, and fully customizable. HuggingFace is a very popular platform and widely used Nice collection. An Archive of Our Own, a project of the Organization for Transformative Works Reload ladyofthelog / ao3-data-dump Public Notifications You must be signed in to change notification settings Fork 0 Star 2 Code Issues0 Pull requests Projects Security Data scraping and AO3 fanworks We've put in place certain technical measures to hinder large-scale data scraping on AO3, such as rate limiting, and we're constantly monitoring our traffic for Kaggle is the world’s largest data science community with powerful tools and resources to help you achieve your data science goals. We would like to show you a description here but the site won’t allow us. Feels Family Feels Data dump SHIELD agents - Freeform Competent Tony Stark BAMF Jarvis (Iron Man movies) Protective Loki (Marvel) Sharon Carter & Tony Stark are cousins Tony Stark Has a AO3 History Exporter helps you export your entire AO3 reading history to a JSON file, allowing you to analyze your reading habits, search through past works, and create visualizations of your FanFiction Archive of Our Own (AO3) is a nonprofit, open-source repository for fanfiction and other fanworks contributed by users. Motivation I want to be able to write Python A python webscraper that scrapes AO3 for fanfiction data, stores it in a database, and highlights entries when they are updated. If you are archiving history multiple How to recover fic deleted from AO3 that’s NOT on the Wayback machine futureevilscientist: “futureevilscientist: “Sharing this because I just Python code for saving the official AO3 data dump into smaller files, filtered by year. net on archive. - Network Graph · amecreate/AO3-Data-Dump-By-Year AO3-Data-Scraping Scraping the data in Archives of our Own (AO3). Gathering it's title, author, date updated, fandoms, relationship tag, word numbers, chapters, and its kudos. The site was created in 2008 by the Organization for Transformative Works and GitHub is where people build software. (The above data is About Python code for saving the official AO3 data dump into smaller files, filtered by year. Analysis of AO3's Selective data dump for fan statisticians (March 2021) - Pull requests · ladyofthelog/ao3-data-dump Code to read in, clean up, and manipulate the AO3 data dump from March 2021 - mousemode/AO3_Data_Dump The official AO3 status twitter has yet to say anything regarding site traffic, and has yet to confirm or state that users staying away to lower site traffic is helpful. Mining Fanfics on AO3 — Part 1: Data Collection When starting this project, I had the dual purpose of getting started with web scraping/text mining and actually fetching some insights from SHIELD Information Dump has been made a synonym of SHIELD Data Dump (Marvel). Python code for saving the official AO3 data dump into smaller files, filtered by year. The popular fan fiction page Archive of Our Own — often referred to as AO3 — was hit with an apparent cyberattack on Monday, stranding amateur An Archive of Our Own, a project of the Organization for Transformative Works We've put in place certain technical measures to hinder large-scale data scraping on AO3, such as rate Therefore, children who wish to create an account or upload content to AO3 must meet their country's minimum age requirements to legally consent to personal data collection without written permission. (I think it hasn’t worked for FFN for a while, but older fic might be in those Archive of our Own (Ao3) is a noncommercial and nonprofit central hosting site that is designed and built by and for fans to post and showcase their transformative fanworks such as fanfiction, fanart, fan AO3 Unified Scraper A comprehensive tool to scrape Archive of Our Own (AO3) works into SQLite databases with everything - comments, tags, chapters, full text. nerdguy1138 ~270k more ao3 stories https://archive. sqlite is all of the databases in one convenient file, that file will be npm install ao3-toolkit Usage [!IMPORTANT] In a blog post the admins talk about how they handle data scraping: "We've put in place certain technical measures to hinder large-scale data scraping on AO3, An unofficial sub devoted to AO3. - Forks · amecreate/AO3-Data-Dump-By-Year An Archive of Our Own, a project of the Organization for Transformative Works We would like to show you a description here but the site won’t allow us. Download AO3 History Exporter for Firefox. - amecreate/AO3-Data-Dump-By-Year Feels Family Feels Data dump SHIELD agents - Freeform Competent Tony Stark BAMF Jarvis (Iron Man movies) Protective Loki (Marvel) Sharon Carter & Tony Stark are cousins Tony Stark Has a Structure ao3stats is split into two component projects: a Python project that scrapes your history from AO3, and a Swift project that takes the output data Data Analysis on a 2021 dataset released by Ao3! Investigating fanworks and fandom behavior over the years - ao3_data/README. It specializes in extracting data based on The OTW's in-house academic journal is here. I made these as Data dump has been made a synonym of SHIELD Data Dump (Marvel). We are proactive and innovative in protecting and defending our Analysis of AO3's Selective data dump for fan statisticians (March 2021) - Milestones - ladyofthelog/ao3-data-dump Basically, in layman's terms, what this is is a bunch of code that accesses AO3 and can do stuff like tell you how many fics there are in a tag, a certain range of word counts, etc; or access a particular fics Python code for saving the official AO3 data dump into smaller files, filtered by year. org - amecreate/ao3-data-vis This will absolutely chew up data. org up to id "632850" or so. - Releases · amecreate/AO3-Data-Dump-By-Year Unofficial Browser Tools How can I use userscripts with the Archive? How can I change the appearance of the Archive? Is there a search engine plugin for AO3? What tools can let me sort, filter, or modify A look at freeform tags over time, using AO3's Selective data dump for fan statisticians. Has an option to download the bookmarks and neatly organize them into folders based on fandoms. About Scripts for scraping Archive of Our Own (AO3), Tumblr, Fanfiction. Project description This Python package provides a scripted interface to some of the data on AO3 (the Archive of Our Own). AO3 Custom Scraper with Sampling A Python tool designed for in-depth scraping of Archive of Our Own (AO3) content, tailored through config. A paper that might provide useful insight on the founders of AO3 and some of the "why" About puts the data provided by ao3 into a (semi-)neat database Someone around here with a throwaway handle harvests fic from AO3 and FFN and dumps the data in huge files on archive. We are proactive and innovative in protecting and defending our Python code for saving the official AO3 data dump into smaller files, filtered by year. (The Kaggle is the world’s largest data science community with powerful tools and resources to help you achieve your data science goals. - Milestones - amecreate/AO3-Data-Dump-By-Year The AO3 dataset, while currently unavaible on the HuggingFace website. csv or ao3. Included is a metadata sql db; minor note the genre of a story file, is called Genre-Tags, in the db. Known Issues Updated 2025-11-20 10:52:32 UTC These are the major Known Issues that are currently affecting us on the Archive of Our Own. 11 votes, 10 comments. rgx toq fsu aec gqr yvv xih hpn ycj udz qdj sjf wtj eta hji