← All Projects

v0rt

A crawler that collects the human things hidden in the web's page source, and the gallery that shows them off.

Python BeautifulSoup PHP SQLite
v0rt screenshot

Most of the web is written by people, and people leave traces of themselves in the parts no visitor is meant to see: a joke in an HTML comment, an apology above an ugly hack, an ASCII-art banner, a "we're hiring" tucked into a console message, a memorial baked into a response header. v0rt wanders the open web collecting those small human artifacts and puts them on one wall.

It exists because I like building crawlers and watching them move from page to page. I've built dozens with different intentions, and this was the one that felt worth sharing.

The crawler is a single-threaded Python script that respects robots.txt everywhere it goes, pauses between requests, and mostly looks at a site's homepage, since that is where the finds cluster. It keeps a per-domain page cap so it keeps hopping to new sites instead of sinking into one. Its output is an append-only log that it reads back on startup, so a crawl can be stopped and resumed at any point.

It looks for several kinds of artifacts: hand-written HTML comments, ASCII art and kaomoji, deliberate console messages, playful response headers such as X-Clacks-Overhead, and humans.txt credits. Each candidate is scored, and only the ones that read like a person wrote them on purpose are kept.

Publishing is a separate, deliberate step. An export pass writes a redacted, capped file of the best finds, and that file is the only thing the public site ever reads. The raw log never leaves my machine. The site itself is plain PHP with SQLite, with no framework and no build step. Every find gets a stable ID derived from its content, a permalinked page with its own social-sharing card, and a place in the RSS feed. There is also a stats page, and visitors can like finds to float the best ones to the top.

The hard part isn't finding comments, it's that roughly 99% of them are boilerplate: IE conditionals, WordPress block markers, analytics and SEO-plugin notes. A hard-skip list removes machine-generated plumbing outright, and a scoring pass ranks whatever is left, so the wall shows jokes and confessions instead of "end header".

Publishing someone else's page source safely took more care than collecting it. Before anything goes public, emails are masked to their domain and IP addresses are stripped. Any find that contains a real credential or key is dropped entirely rather than redacted, because I don't want to spotlight anyone's accidental leak, even in masked form. Every find links back to its source, and site owners can ask for removal.

Likes without accounts needed a little thought. A coarse hashed fingerprint plus a per-browser flag blunts refresh-spamming, and votes are only accepted for IDs that exist in the dataset. Because each find's ID is a hash of its content, likes survive every data refresh. The same export also had to stay bounded, so it keeps only the top finds by score instead of growing forever as crawls pile up.

ASCII art needed its own handling from end to end. Detection has to tell real drawings from ordinary text, and whitespace has to survive normalization and display intact, or the art collapses into noise.

Visit Site