Furry data 

Fiiinally managed to make my way through processing the e621 metadata dump they sent me. Pulled it from a corrupted and gross JSON file into a sqlite db.

sqlite> select count(*) from submissions;
700479
sqlite> select count(*) from tags;
1903
sqlite> select count(*) from artists;
186
sqlite> select count(*) from sources;
388

That's a lot of submissions @.@

Follow

re: Furry data 

:c

:C

I cleaned the data of duplicates and...

sqlite> select count(*) from submissions;
399

· · Web · 3 · 0 · 0

re: Furry data 

@makyo I'm almost certain that can't possibly be right

re: Furry data 

@makyo Some of those weren't duplicates, perhaps?

re: Furry data 

@orrery Will do a bit more digging, but I filtered on both ID and image checksum.

re: Furry data 

@makyo ... that's a lot of duplicates.

Sign in to participate in the conversation
Awoo Space

Awoo.space is a Mastodon instance where members can rely on a team of moderators to help resolve conflict, and limits federation with other instances using a specific access list to minimize abuse.

While mature content is allowed here, we strongly believe in being able to choose to engage with content on your own terms, so please make sure to put mature and potentially sensitive content behind the CW feature with enough description that people know what it's about.

Before signing up, please read our community guidelines. While it's a very broad swath of topics it covers, please do your best! We believe that as long as you're putting forth genuine effort to limit harm you might cause – even if you haven't read the document – you'll be okay!