Sammy's Blog

Table of Contents 1. I posted to HN, and then my website was flooded with AI BOT traffic 2. Using LLMs to Make the Emacs Web Browser Great Again 3. If technology got better, why did everything get worse? 4. Deft version 0.2.0 5. How much LLMs is too much LLMs? 6. Building Chess in about 350 lines of Clojure 7. Announcing Deft: A class and interface system for Clojure and ClojureScript 8. My Blog is Now on RSS 9. How to Win at Wordle 10. The Power Of Habit (Book Review Pt 1) 11. Modern Monetary Theory and You 12. The Perfect Camera 13. Camera Lenses: What's in My Bag 14. Why Many Primary Forecasts are Wrong 15. The Wonky Math behind the Democratic Primary 16. What's Going on With The Impeachment 17. Treating Cancer with Chemo (Part 3) 18. Treating Cancer with Chemo (Part 2) 19. Treating Cancer with Chemo (Part 1) 20. The Importance of Doing Nothing 21. Why I Support Citizens United 22. Camera Settings 1. I posted to HN, and then my website was flooded with AI BOT traffic For the past few years, I've been working on sprinklz.io, a customizable feed reader that can, among other things, deprioritize certain topics from your news feed. A month ago, I posted it to HN, and it got a hundred or so upvotes, nice! It seems like people are really tired of hearing about LLMs, and they enjoyed being able to deprioritize them from their feed. It was a good demo of why a tool like this is useful. Everything was looking up, and then… My website got slow. Like, really, painfully slow. At first I chalked it up to the extra attention of a suddenly popular post. Like any developer I both celebrated, and vowed to make my website more resiliant in the future. Strangely though, the slowness continued, and even started to get worse several days after my post went live, and the metrics I did log did not make sense. Like, you know, a good handful of genuinely active, logged-in users, and a sever exhibiting exhorbatantly high load. So I took a look at some data. I added top-level logging for every endpoint (how many times was a request made to each url path), and I found that a majority of the requests did not even make an attempt to fetch news articles! For reference, my website is built as a single-page-application, and what this means is that a majority of the requests did not even run the javascript necessary to render the page (!). Digging further, I looked at aggregate NGINX logs to find abusers, and, you're never going to believe this, but it turns out that nearly half of our traffic was coming from one specific user. I won't name names, but I looked them up, and they describe themselves as a service that listens for changes to a website, and then summarizes them into an AI slop news letter. And by 'listens to changes' I mean query our website many many times per-minute. Blocked. Several of the others were your standard known crawlers, and kindly identified themselves as such. Claude bot, OpenAI bot, google's search indexer, and several others. I even picked up a few vulnerability scanners along the way. At least, somebody is trying to access -/wp-content/phpinfo.php (my website is not a wordpress site). I added robots.txt to discourage most of the AI scanners, but allow google-bot and other search-related traffic, and asked claude to query all the vulnerability paths and mimic the scanners to make sure I wasn't leaking anything. I also added reasonable rate limits, etc., etc. The most important lesson though, is that it's time to make my website faster. I want to be the website that can day handle xxx QPS of real traffic, and it should scale better than it did, even under heavy load. Inspired by this encounter, I've made quite a few improvements to the underlying infrastructure over the past few weeks, and these should result in pretty substantial performance gains for all users. P.S. if you'd like to check it out, the website is fully free, and available here The TLDR is that it lets you define your own customizations, in addition to things like the points totals from hacker news. You can use it to filter out topics, like AI or LLMs, but the topics are completely arbitrary and you can program your own customizations. 2. Using LLMs to Make the Emacs Web Browser Great Again When I tell people about the emacs web browser, their first reaction is 'Wait emacs has a web browser?!', followed by, 'yeah… I guess that's kind of a web browser'. The emacs web browser is designed to render webpages in a terminal-based, textonly environment and, well, it goes about exactly the way you'd expect. It works well on HTML-only, small-content websites, but not so good for anything else. The problem is that emacs is primarily a text-only editor, but the modern internet is very much not plain-text based. Simple HTML, designed in the simple style of the 1990s has a clear translation into plain text, but modern UI design, with it's visual styling, heavy javascript, and just, overall chaos, not so much. The emacs web browser would be great, if only we could find a way to go back in time… Ok, so the plan is simple. We take modern webpages, rewrite their source code into old-school simply-styled HTML, and then pipe it through to the emacs web browser. We just need to find a way to do this easily and at scale. So um, you didn't forget about LLMs did you? It's nice to be using them in a way that will hopefully reduce the amount of spam content in the world. Basically, the plan is we write a program that: Opens a webpage in firefox Loads the page and runs all the javascript Asks an LLM to generate a re-usable program that transforms the current-page's contents into simplified text Inject the generated javascript into the firefox browser Grab the new page source and forward it to emacs web browser. Yeah, and so, I've been using it, and it runs very well. Like realistically I can use this on many webpages where eww (emacs web wowser) did not run previously. Below are some examples of before and after: The docs for Clojure's mapcat Figure 1: (before) Using eww-readable Figure 2: (after) Using simpleweb-simplify-page It takes a moment to generate the initial modifier program, but once it's generated, it runs pretty fast, and during day-to-day browsing, it mostly doesn't need any extra LLM calls. I have been using it myself and enjoying the experience. The source code lives here: https://github.com/sstraust/simpleweb and is open-source, and free to install, etc., etc. 2. If technology got better, why did everything get worse? I think we all know the answer to this: bad incentives. Technology makes systems more productive, but it does not, in fact, make you in particular more productive. The tools that enable computers to be more efficient also enable people to be more efficient at exploiting your time, attention, and wellbeing. I'm talking neural nets babyyy. I first noticed the problem in 2018. Or rather, I noticed that I had a problem, and that problem was spendinng 3 hours a day watching starcraft 2 replays and wondering where the time went, and how I managed to stay up til 2am again. What's interesting to me is that I didn't have any of these problems until the late 2010s. I had a computer with an internet connection since at least the mid 2000s, and a smartphone with twitter and reddit and the like not long after. It's possible that I've gotten less resiliant over the years, but I have a sneaking suspiscion that it's the computers who changed. I can't quite put my finger on it, but I have this sensation that every time I interact with a system or website, I'm playing this kind of adversarial game, where I'm trying to just go about my day, and maybe read a little article about computers, and they're trying to figure out a way to keep me on their site for just a little bit longer. I also have this feeling that as software engineering has moved from a discipline of hobbyists and enthusiasts to leaders who increasingly developed through finance or business-minded tracks, we've found ourselves with these major tools that we used to love become both extremely bland and extremely profitable. So what can we do it make it stop? touch grass. really when the world is like this, i think the best solution is to just leave. it's not easy, especially when the planning is often online, but if you can set it up, I promise that a day in the park playing chess with your friends is better than any amount of time you can spend on the internet. take to the streets. i think that protesting is effective, and more people should do it. if you're willing to take action, it models to others that they can take action. stop coming back to the same machines. I think we all have this gamblers fallacy in us, where if it didn't pay out the first time, but it pays out sometimes, we try the machine again, but it doesn't really work like that. if you're losing money the lesson is to stop putting coins in the machine. editor's note: I've been working on a free customizable news reader as a way to take out some of my frustrations with the modern internet. if you don't use this one, I recommend following individual blogs and websites, because it's so much better to just get off these big platforms entirely. editors note 2: i haven't yet posted this latest article to my rss xml, because i'm not ready to send notifications for it yet. will add in the coming weeks. 3. Deft version 0.2.0 Now that deft 0.2.0 is released, I figure it's a good time to talk about what I built, and why I built it. The point of Clojure's everything-is-a-map toolkit is that getters and setters are a waste of time. Clojure was built, in large part, in response to the crappy 2008 Java workflow, where you had to write getters, setters, and builders, and you took on all this bloat that nobody enjoyed. The problem is that if you want to represent your data using the built-in defprotocol and defrecord, and you want to define functions that depend on the presence of a certain field, then the only way to encode that information