My guess—and I’m not privy to any inside information here—is that operations at places that are building neural-net-based models look something like this:

1. “Harvest” people are responsible for obtaining data and as much associated information as possible, stuff like where data was obtained from, timestamps, any identity information, etc (handle “X” on platform “Y”). If a comment or webpage or whatever gets revised, maybe they keep multiple copies. They’ll avoid things like following infinite loops on websites, deal with corrupt archives, all that. That probably goes into some giant database. Maybe some of these guys buy access, like Google did with Reddit. A lot are just going to write spiders, scraping bots, whatever. A search engine fundamentally has to have a spider slurp down data to create its index to work at all, so I’m sure that Google is going to have a bunch of expertise in this area, built up over many years. They’ll probably retain this data in pretty raw, unprocessed form.

2. “Unification” people are going to work on the above. They write code to convert the above into a standardized-across-all-platforms format. Maybe they convert stuff to unify timezones or time formats on comments. Maybe they strip off formatting stuff added by particular platforms. Markdown or HTML titles, or titles in Microsoft Word documents, for example, probably wind up represented in one way. Where possible, maybe usernames get attached, if data is from a forum.

3. “Scoring” people work on the unified dataset. They are going to be developing some system—and there might be a number of parallel projects—for generating a score of various sorts that might be a useful value for either a model or training information. With this as well, search engines have to do this; Google’s PageRank, for example, tries to assess the “value” of a page, determine what’s most-useful and reliable information, so that it can order results (and maybe blacklist some). A lot of what search engines do isn’t just getting you hits, but putting the most-valuable ones at the top. If you can develop heuristics to rank information based on reliability or political position or any number of other things, those are going to be useful numbers.

4. “Pruning” people use information, probably including the “scoring” information, to winnow down the size of the data, eliminate data where the cost of inclusion in the training set—it’s costly to train a neural net, and reducing the size of the training corpus reduces that cost—outweighs the benefits. They probably try to cut out garbage, irrelevant binary data, poisoned data, duplicate data, misinformation, whatever.

5. “Training” people are going to do the training and probably develop systems for rating the value of the above “scores” in producing desirable output. Maybe some “scoring” heuristic is really super-useful in producing accurate and useful output as defined by some benchmark that these guys set up. Like maybe, oh, one “score” is based on whether information was obtained from a .gov TLD, or another whether a text classifier says that the language is English and the Lix school year readability is above a certain level. Could be many of these. Some of what these guys find may go back to the “scoring” guys in trying to guess what heuristics should be further developed (“known time of domain age to time content was initially spidered is particularly valuable in identifying SEO-generated spam website content; see if you can produce more scores that identify mass-created spam websites”), some to the “pruning” guys to increase the low-value data that they prune out, and some of what they find may go into weighting importance of text for the training process. Some may even go all the way back to the “harvest” guys to determine how frequently to spider data from a given source—Google doesn’t respider information at a fixed rate, but hits “valuable” sources, like major news sites, very frequently. For non-pure-LLM neural nets—I don’t know if Google is doing this yet with Gemini, but I’d imagine that eventually, probably more-advanced AIs will need to move past just being an LLM—this might also be an input used in making a decision at runtime.

There are probably going to be a lot of points in this process where not-valuable data gets identified and removed or deprioritized. Even if no engineer at somewhere like Google has tried to write a “scoring” heuristic to identify text specifically to identify output from a tool that mass-edits comments on Reddit, my guess is that text consisting of nonsense words, which is what some of the Reddit mass-comment-editing-tools I’ve seen insert, probably intrinsically get identified as part of that process by multiple classification tools that generate heuristic scores. If all of a user’s comments look like the output of some simple text generation tool, or if all of them change in a given short period, that’s probably kind of a big red flag. That does mean that the information won’t be in the output model, but I think that people who are aiming to poison LLMs during training probably are expecting more impact from putting nonsense in their comment versus just blanking the comments than they’ll likely have. Google—and more broadly, people who are running search engines and even more, building neural nets trained on data on the Internet—are just going to have to already have a lot of fairly-sophisticated work in place to assess the value of and filter their inputs.

I went a different route. Let them choke on old information. It’s going obsolete anyway, let’s live on somewhere else.

Make sure to keep the account and check in after a few days, then weeks, then months and delete again if needed. When i left reddit i had found them restore about a third of my content about a month later, sneaky fucks. But since my last check, all remain wiped.

at some point they will gatekeep all that with subscription or something

Good luck to 'em. But also fuck 'em.

Correct. I got banned for repping guillotines as a solution to Americas problems, and then doubling down. That “this account has been banned” banner has been a point of pride for me. I joined reddit in 2009. For so many years I answered questions and commented all day long. In certain subs I was a major contributor. I switched phones and now can’t even log in without an app. I will not be downloading that app. Goodbye Felicia good luck with your bots

Smarter than I. I scrambled them first, then deleted. Used the blackmail guy too. I did it all wrong. Now all my accounts are banned and I can’t do shit about it.

Same brother. Adding my comment under yours to mark the moment!

Check this out, it’s actually quite interesting: https://www.youtube.com/watch?v=dU7gCt-7kKk

Edit: and no, it’s not rickroll, even if it would have been very appropriate here.

Nah it changed, if you were there early on you know the vibe started changing by 07 and by 10 it was kinda what it is now. I’m pretty sure I fully left in 2011.

While yea alot was user base ageing out it was also leadership, rules, posting numbers, OC reduction, government actions.

It probably does. To my knowledge they’ve never deleted old accounts, though you might have an issue if you’ve forgotten your password and the e-mail address you used to sign up is no longer accessible to you.

The latest ban was for calling someone gay. The sub I used was known for being a place where you could do that. Call it edgelord or hateful or whatever, I’m immature and still find it funny. Sue me. I had done it dozens of times on that account.

You’d probably be more comfortable over on Twitter or Truth social then. Casual homophobia doesn’t go over well around here.

Yikes.

Immature? No, the word you’re looking for is “homophobic.”

Until google stops letting you log in because you didn’t link your alt account to your main…