Reddit Bans Deepfake ‘SFW’ Community

Over the weekend, apparently around Sunday, Reddit banned the nominally “SFW” (Safe For Work) deepfake community titled r/deepfakesfw. The subreddit was one of the first responses to the social media giant’s prompt erasure from the original, AI-inflated r/deepfakes sub in 2018.

The (relatively) all-purpose legend that salute now (archive snapshot taken Monday, June 13) anyone visiting the submarine explains that it ‘was banned due to a Reddit violation rules against involuntary pornography”.

The new r/deepfakesfw homepage.

The r/deepfakesfw subtitle was not frequently archived by popular curatorial platforms, but most recent snapshot of WayBack Machinetaken about ten days ago (June 3, 2022), indicates that the sub had 3,095 readers at that time.

This is actually a higher number of subscribers than r/DeepFakesSFW (note the extra ‘s’), which currently has 2,827 readers (archive snapshot taken Monday, June 13, 2022). The most popular “legit” SFW deepfake community currently appears to be r/SFWdeepfakeswho at the time of writing has 16,636 readers (photo taken on Monday, June 13, 2022).

Don’t even ask

What seems to have doomed r/deepfakesfw was not the post but the ask deepfake pornography. Reddit’s Rules on Sexually Explicit Media state:

‘[Images] or another person’s video posted for the specific purpose of simulating explicit content or soliciting “look-alike” pornography (e.g., “deepfakes” or “bubble porn”) is also against the rule. »

Based on the latest available RSS feeds from the now-banned SFW sub, it was demand rather than posting of deepfake NSFW content that ultimately triggered the ban:

Some of the final, clearly NSFW posts from the now dead r/deepfakesfw subreddit.  Source: Ino Reader.

Some of the final, clearly NSFW posts from the now dead r/deepfakesfw subreddit. Source: Ino Reader.

Although the posts these RSS posts point to are no longer available, clicking on the requester profile links (blurred in the image above) reveals that many posters are requesting deepfake content that is banned on more than one forum, and request or offer some of the most extreme content of its kind, including incest porn and various shades of unintended porn themes.

This suggests that the remaining Reddit “SFW” subs survive on the speed, diligence, and quick response times of their moderators, as it seems Reddit doesn’t want to become some kind of “classified ads” forum. allowing people to log in and deal with illegitimate deepfakes via DM (in cases where this cannot be explicitly requested in a message).

If so, then the not-yet-banned r/DeepFakesSFW may be living on borrowed time; his current two main posts when entering the root subreddit url with the “old” Reddit theme enabled were specifically tagged “NSFW” (a tag that probably shouldn’t even be available in that particular community).

Depending on the circumstances, some rule-breaking posts currently appear in the r/DeepFakesSFW subtitle.  Source: https://www.reddit.com/r/DeepFakesSFW/

Depending on the circumstances, some rule-breaking posts currently appear in the r/DeepFakesSFW subtitle. Source: https://www.reddit.com/r/DeepFakesSFW/

You can see these two NSFW posts at the top of today’s r/DeepFakesSFW forum archived here (photo taken on Monday, June 13, 2022).

There’s actually no NSFW “flair” available when creating a new post on r/DeepFakesSFW, which has been up and running for four years, according to information in its sidebar:

Tags available when posting to r/DeepFakesSFW.

‘flairs’ available when posting to r/DeepFakesSFW.

Therefore, the NSFW r/DeepFakesSFW’s tags appear to have been applied either by forum moderators or higher-level administrators – or even algorithms, possibly without any available intervention or opposition from local moderators. I have contacted the only moderator listed at r/DeepFakesSFW about this and will post any updates available.

(EDIT, Tuesday, June 14, 2022 – A moderator from r/DeepFakesSFW responded by stating “This seems to happen when the user’s account is NSFW”. Presumably, therefore, the only way to slow down the likelihood of such requests in SFW deepfake subreddits would be to ban users who have “earned” this rating.)

Most of the content on this subreddit, and the larger r/SFWdeepfakes, is about using deepfake technology to “recast” popular or classic movie actors, or have more harmless AI-based fun with the popular media and deepfake technology, and are indeed referred to as “SFW”.

Even though many such posts are perilously close to copyright-based bans, the continued survival of non-pornographic “IP-appropriate” deepfake video posts by major YouTube deepfakers such as Ctrl-Shift-Face suggests broad studio and rights-holder tolerance for such activity, at least for now.

The “innocent” appearance of the Deepfake process and ecostructure

Two weeks ago it appeared (and many discussion) that Google had suddenly banned deepfake creation software from its popular Colab cloud processing environments. Even to this day, Colab is apparently only “triggered” by execution attempts DeepFaceLab (DFL), the most popular deepfake software in the world (and the most closely associated with AI pornography), rather than less common implementations such as face swapwhich is more reputable, even though it performs the exact same tasks as DFL.

To some extent, this approach may have been the only way for Colab to implement any restriction on the production of deepfake pornography, since the deepfake training process only uses open source software and very cropped face images (usually only 512x512px, often less), with almost no scope for recognition algorithms to image to intercept any actual pornographic content.

By the time the deepfake model is trained, the deepfake no longer needs Colab’s massive processing power or an expensive local GPU. The most current conversions (i.e. the process of imposing a face into a video, pornographic or otherwise), can be accomplished locally, on low-power laptop GPUs – or even on a CPU (with a slightly longer processing time). By the time something pornographic happens in the deepfake process, Colab has long since been knocked out of the picture.

Therefore, the constituent parts that pass between the creators of deepfake porn rarely amount to more than potential minor copyright infringement, in the form of the transmission of “face sets”, which deepfake creators have extracted from videos. YouTube, Blu-ray, TV interviews, social media and various other sources.

Is a “scorched earth” policy coming for Deepfake communities?

The question haunting people in deepfake Discord communities such as DeepFaceLab (including DeepFaceLive), face swap, machine video editor (MVE, a GUI environment for DFLwith many additional features), and new audio-based deepfake Discord servers such as Deepfake audio is Will current portal and platform owners soon simply ban deepfake communities on principle, rather than bear the cost of constant moderation against a wave of NSFW adherents and creators?

Bryan Lyon, one of the FaceSwap developers and owner of the Discord server associated with the project, commented:

“We’ve managed to stay on Discord, despite multiple purges, staying absolutely spotless. We have no hidden channels or secret second servers with illegal content. A few years ago there was a purge of the popular DeepFaceLab server. Everyone on it has been banned. They had invite-only roles with access to channels containing non-consensual pornography.

Given that Google, itself one of the largest tech hosting resources in the world, has generally banned deepfake training on its servers and GPUs, it’s even possible that Microsoft-owned GitHub may decide that facilitate these technologies* ultimately won’t be worth the moral ambiguity or the investment of oversight.

At present, the process of legitimizing deepfake technologies so that their use transitions from homemade pornography to film and television production, and non-infringing/criminal uses, is too nascent to produce clear policy guidelines for community hosting platforms and services.

* Archive from https://github.com/search?q=deepfake from Monday, June 13, 2022.
† In private DMs on Discord.

First published June 13, 2022. Edited 3:08 PM UTC for image change.
Edited Tuesday, June 14, 2022, 06:24 UTC, to add the Reddit moderator’s response regarding the NSFW status of some posters. Also change the image on the RSS feed illustration.

Sam D. Gomez