View source for User:TemplatesFTW

{{/Header}}
{{bot user|Kenny2scratch}}
{{bots
|image=File:TemplatesFTW Icon.png{{!}}128px
|name=TemplatesFTW
|task=<ul><li>Adding dates to templates</li><li>Adding {{t|inaccurate}}</li><li>De-1st/2nd-personifying pages</li><li>Updating references to follow [[Help:References#Format|format]]</li><li>Adding {{t|bad style}}</li><li>Adding {{t|External Programs}}</li><li>Compressing images</li></ul>
|process=Fully-automatic
|status={{/Status|foo}}
|added=18/08/2017
|language=Python
|bgColor=black
|other=Activates periodically
}}
<!-- much based on WikiMonitor's userpage -->
__TOC__
{{----}}

==Documentation==
This bot activates when the host runs it (normally via the command line).

Source code for this bot can be found [https://github.com/Kenny2github/deathly_hallows on GitHub].

===Edit Process===
==== De-1st/2nd-personification ====
The bot asks the owner how many pages to depersonify, and if the response is not empty:
# Requests that many random pages
# For each page:
## Checks that the page has any words as specified in [[/Config/FirstAndSecondPersonWords|its configuration]],
## If it does, pops up a dialog box with the page's content, with words matching its configuration highlighted on either side with Unicode FULL BLOCK characters (&#x2588;)
## The owner makes their edits, and presses OK
## The bot submits the edit (minus full block characters)
## OR, if "Cancel" was pressed instead of OK (the words might have been perfectly legitimate), skips making the edit.

==== Reference updating ====
The bot asks the owner how many pages to update references on, and if the response is not blank:
# Requests that many random pages
# For each page:
## Checks that the page has a <code><nowiki><references/></nowiki></code> tag,
## If it does, pops up a dialog box with the page's content, with references not matching [[/Config/ReferenceFormat|its configuration]] highlighted on either side with Unicode FULL BLOCK characters (&#x2588;)
## The owner makes their edits, and presses OK
## The bot submits the edit (minus full block characters)
## OR, if "Cancel" was pressed instead of OK (the references might have not needed to follow that format), skips making the edit.

==== Updating dates on templates with dates ====
The bot runs through the entire [[:Category:Pages with Dateless Templates]] for pages, and for each page:
# Checks that the page has a template without a date parameter in the first place,
# If it does, finds the date the template was added by searching through the page history for when the template was added (with the help of a package the owner wrote, [https://pypi.org/xmlx xmlx]),
# After getting the date, adds it to each occurrence of a dateless template,
# Checks how many {{t|citation needed}} or {{t|cn}} templates the page contains,
# If the number is over the count specified in [[/Config/InaccurateCnCount|its configuration]], adds {{t|inaccurate}} to the beginning of the page,
# Finally, submits the edit.

==== Adding or removing templates for external links ====
The bot asks the owner how many pages to check for external links, and if the response is not blank:
# Requests that many random pages
# For each page:
## Checks whether the page has {{t|External Programs}} (or any of its template redirects) on it, and stores the result in a variable,
## Checks that the page has any links whose domain names are not whitelisted in [[/Config/LocalExtDomains|its configuration]], and stores the result in a variable,
## If the page does have the template, and has external links, continues to the next page;
## If the page does not have the template, and has external links, adds {{t|External Programs}} to the top;
## If the page does have the template, but has no external links, removes {{t|External Programs}} from the page;
## Otherwise, ignores the page.

==== Checking for bad style ====
The bot asks the owner how many pages to check for bad style, with a default of 10, then:
# Requests that many random pages
# For each page:
## For each style guideline configured in its source, checks if the page breaks that guideline,
## If it does, adds the code for that guideline to a list,
## Pieces together a {{t|bad style|first guideline broken|second guideline broken|third|etc}} including all the broken guidelines,
## Finally, inserts the pieced-together template just before the first category link, and submits the edit.

==== Compressing files ====
The bot asks the owner how many file upload logs to check for compression, with a default of 10, then:
# Requests that many upload log entries
# For each entry:
## Fetches information about the file, including upload URL, comment, and filesize
## Compresses the file by URL with the TinyPNG API
## If the difference between the returned file's size and the original file's size is less than a kilobyte, skips the entry - that small of a difference is not worth uploading for
## Uploads the compressed file as a new version

==== Adding templates for citations ====
The bot runs through [[Special:WhatLinksHere/Template:Citation needed|what links to]] {{t|citation needed}} and [[Special:WhatLinksHere/Template:Cn|what links to]] {{t|cn}}, and for each page:
# Checks how many {{t|citation needed}} or {{t|cn}} templates the page contains,
# If the number is over the count specified in [[/Config/InaccurateCnCount|its configuration]], adds {{t|inaccurate}} to the beginning of the page,
# Finally, submits the edit.

===Turn off===
To turn off the bot, [[Special:Block/TemplatesFTW|block]] it. '''This should only be done as a last resort'''. Since this bot is run under supervision from its owner, the owner will likely already know about the bad edits. In any case, after disabling it, contact the owner.

If it is only malfunctioning on one particular page, add {{t|NoBots}} to the page - it will unconditionally skip any page with {{t|NoBots}} on it.

==Task==
This bot searches through [[:Category:Pages with Dateless Templates]] for pages that have templates with no <code>date</code> parameter, adding the parameter if applicable.

This bot also semi-automatically de-1st/2nd-personifies articles and updates references to follow the [[Help:References#Format|format]].

Additionally, it searches through [[Special:WhatLinksHere/Template:Citation needed|what links to]] {{t|citation needed}} and [[Special:WhatLinksHere/Template:Cn|what links to]] {{t|cn}} and adds {{t|inaccurate}} if there are too many {{t|citation needed}} templates on the page (see [[/Config/InaccurateCnCount/]]'s history for the current number defined as "too many").

Besides that, it also adds {{t|bad style}} to pages if they break guidelines on [[Scratch Wiki:Style Guide]].

Also, it adds {{t|External Programs}} to pages that contain external links.

Finally, it automatically compresses images if they have not been compressed.

==Credits==
65536 thanks to the following people for making this bot possible:
* {{user|KrIsMa|name}}, for being the main contributor to its purposes and for accepting the bot in the first place.
* {{user|jvvg|name}}, for showing the futility of some original concepts.
* {{user|ErnieParke|name}}, for adding to the concepts.
* The entire [[mediawikiwiki:API:Main Page|MediaWiki API documentation]], for being the sole help with technical issues.
* The entire Wiki community, for putting up with all of the failed tests and for bases for ideas.
* {{user|scmb1|name}}, for making the necessary technical changes (and for being the best bureaucrat in general).

==Configuration==
Most configuration is stored in the form of ''edit summaries'' on pages. To see the configuration, look at the page history of the configuration pages, stored [[/Config|here]].

==Creation==
:''(First person from creator's perspective)''
I had always wanted to make a bot, after hearing about [https://github.com/tjvr/scratch-wiki-client blob8108's MediaWiki API], but I had never been able to think of a task. Then, I thought of following the same convention that Wikipedia does - showing the dates for templates. However, the technical boundaries for making this easy were too great to cross. Then, I thought that a bot could add the dates, and Deathly_Hallows was born. (The name is just the name of my alternate account on the main site, [https://scratch.mit.edu/users/Deathly_Hallows Deathly_Hallows].) (After acceptance, however, scmb1 thought the name "didn't reflect what it does", not taking into account bot names like [[B:TIAC|ThisIsAnAccount]], so it got a name change to its current name, TemplatesFTW.)

After I had made the bot request, I thought that it could also add some full-fledged templates automatically as well. However, jvvg showed me the futility of making it add {{t|stub}} and {{t|expand}} (high chance of false positives and missed cases), but adding {{t|inaccurate}} remained a workable idea.

Finally, the bot’s status turned to "test on main account". It was finally time to test it. Thankfully, I tested it privately first (by printing the intended page contents to the console, instead of submitting the edit), as many tries failed for one reason or another - even a bug with MediaWiki itself! However, it finally succeeded - and after a successful trial run on real pages, the status turned to "accepted". So I submitted an account request, KrIsMa accepted it, and here it is!

Funny thing: after thinking so much about how easy blob8108's MediaWiki API was, I never actually used it - I did all the dirty work myself.

The first run didn't go smoothly - I ran into problems with the same page multiple times. However, after more debugging and some help from my own Python module, [https://github.com/Kenny2github/xmlx xmlx], I finally got around that problem - then hit another immediately. Fixed that, then the "adding dates to templates" stage went fine!

Then, while in the middle of its second stage, "adding {{t|inaccurate}}", it crashed - turned out my connection was throttled by my router. I fixed that by waiting five seconds after each page, and now it works.

Update after the [[Scratch Wiki#Server Transfer|transfer]]: I rewrote it to use much less regex and much more [https://github.com/earwig/mwparserfromhell mwparserfromhell], a dedicated parser for wikitext. Since the transfer also updated the MediaWiki version, part of the rewrite changed it to use [https://github.com/Kenny2github/mw-api-client my MediaWiki API client]. I also removed the xmlx dependency, since it was badly written and I am no longer developing it. Finally, I also made it pass [https://python.org/dev/peps/pep-0008/ Pylint].

I run my bot after I finish patrolling edits. If you look at the recent changes [{{fullurl:Special:RecentChanges|hidebots=0}} without hiding bots], you might see it around :D.

==Status==
{{/Status}}