How 2004 RuneScape fit a multiplayer RPG into 56k dial-up · jkm.dev

In 2004 I played too much RuneScape on a 56k modem that died the moment Mum picked up the phone. A 3D world, up to a couple of thousand players on a server, dozens on screen at once - in the browser, on 5 kilobytes per second. It worked. Let’s follow a single step and see how.As a child I was too preoccupied with picking flax and killing goblins to think about how this worked. The answer, however, is a sustained, almost obsessive exercise in not wasting bytes. So, let’s click one tile north of where we’re standing, and trace every byte that crosses the wire from that click, to the server, to the screen of another player.Central fountain, Varrock SquareMethodology#The detail in this post comes from a decompiled 2004 RuneScape 2 client. Snippets are rough translations from that decompile, tidied up in places for readability but with the logic intact.The core principles aren’t identical across versions, but most of them run all the way from RuneScape Classic (2001) to present-day RuneScape 3 and, of course, Old School RuneScape.Constraints#Let’s look at some of the constraints that Jagex were working with at the time.Bandwidth. A 56k modem syncs at 56 kilobits per second downstream, and less upstream, minus any protocol overheads and line noise. Call it 5 KB/s down and a lot less up. Broadband was available in British homes by 2000, but it wasn’t until the late 2000s that the majority of UK households had a broadband connection, so plenty of players were on dial-up.Java applet, in a browser, in 2004. Java applets ran in a security sandbox, which meant no raw native sockets and no UDP. Every byte travelled over a single TCP connection, in-order and with per-segment overhead.A 600ms server cycle. The RuneScape game server advances in discrete cycles (or ticks) of roughly 600 milliseconds. Every cycle, for every player, the server has to work out everything that player can now see and ship it before the next one.The cipher layer, briefly#After the login handshake completes, before any game packets are sent, a small encryption layer is set up. This one’s not about saving bytes; it’s the only encryption in the stack (outside of some RSA encryption in the login handshake), and it’s here because the opcode it protects is the very thing every later section depends on.Every packet begins with an “opcode” byte: a small integer saying what kind of packet this is. That opcode (and only that opcode) is enciphered with a stream cipher called ISAAC. There are two streams in play - one for traffic from client to server, and one for the reverse direction. Both sides need both streams: the client enciphers what it’s about to send and deciphers what just arrived, and the server does the same in mirror image (per connected player).Both streams are seeded from a shared four-integer key. The client generates two of those integers itself; the other two come from the server as part of the handshake. The server-to-client stream then uses the same seed with 50 added to each word - enough to keep the two directions from sharing a keystream:this.outboundCipher = new ISAAC(seed); for (int index = 0; index < 4; index++) { seed[index] += 50; } this.inboundCipher = new ISAAC(seed); Enciphering on the way out is one line:public void putOpcode(int opcode) { this.putByte(opcode + this.outboundCipher.value()); } And on the way in, the mirror image:this.currentOpcode = (this.currentOpcode - this.inboundCipher.value()) & 0xFF; So the packet body isn’t encrypted, only the opcode. As we’ll see later, the opcode is what tells you how to read the rest of the packet, and where one packet ends and the next begins. Without it, the body is just a wall of bytes, so enciphering that one byte was the cheapest possible defence against third-party packet parsers.Sending a walk request#We’re going to look at what happens when you click on a tile one square north, and how that gets transmitted to the server.Before any networking occurs, the client runs a breadth-first search using the local collision map to build a path from where you are to where you clicked (an easy search, in this case), and then writes the packet for the server to read. The pathfinding is standard so I won’t go into it here.The first part of the packet is the opcode, followed by a single byte containing the length of the packet body. As you’ll see, the number of bytes contained in the packet is dependent on the size of the path, so this “length” byte allows the server to know how far to read. Not all packets have this length byte, only packets which contain some variably sized body.The start position takes 4 bytes (two shorts), each subsequent waypoint delta takes 2 bytes, and there’s a final byte for whether the Ctrl key is held. So the body length is 4 + 2 * (pathLength - 1) + 1.this.outboundStream.putOpcode(ClientToServerOpcodes.WALK_TILE); this.outboundStream.putByte(4 + 2 * (pathLength - 1) + 1); The packet contains the absolute position of the first waypoint in the path (x and z sent as a two-byte “short” each), followed by the delta of each waypoint in the path against the first one - one signed byte per axis, which fits comfortably within the byte’s range of -128 to 127, as a single click can only ever land so far away.The decision to send only a delta here, as 2 bytes per step, rather than absolute coordinates as 4 bytes per step is the first example we’ve seen of Jagex’s networking frugality. In absolute terms it only saves a few bytes for a single walk packet, but every additional waypoint costs 2 bytes instead of 4 - a 50% saving per waypoint.int firstX = pathX[0]; int firstZ = pathZ[0]; this.outboundStream.putShort(this.playerPositionX + firstX); this.outboundStream.putShort(this.playerPositionZ + firstZ); for (int i = 1; i < pathLength; i++) { this.outboundStream.putByte(this.pathX[i] - firstX); this.outboundStream.putByte(this.pathZ[i] - firstZ); } Another frugal decision here is that pathX and pathZ do not contain every tile in the path, just the corners. Walking ten tiles in a straight line only sends one waypoint: the destination. The server already knows where you started, so it walks the line itself and validates against its own collision map.The last part of this packet is a single byte to indicate whether the Ctrl key is held. In early versions of the game, this was used to force “run mode”, in later versions it inverts the current movement mode (runs to your clicked destination if “run” is off, or walks if it’s on):this.outboundStream.putByte(this.keyStatus[Keys.CTRL] == 1 ? 1 : 0); So we can see that our single step north takes seven bytes, including our opcode and length marker:WALK_TILE packet byte layoutA seven-byte client-to-server walk packet for a single step: one opcode byte, one length byte (value 5), a two-byte destination x short, a two-byte destination z short, and one run-toggle byte. The opcode and length form the header; the remaining five bytes form the body, whose size equals the length byte.0123456opcodeencipheredlength= 5xxzzCtrlrun toggledestination x · 2-byte shortdestination z · 2-byte shortheaderbody ·5bytesWALK_TILE packet byte layoutThe seven bytes of a single-step walk packet, stacked top to bottom: byte 0 opcode (enciphered), byte 1 length (value 5), bytes 2 and 3 a destination x two-byte short, bytes 4 and 5 a destination z two-byte short, and byte 6 a run-toggle byte. Bytes 0 and 1 are the header; bytes 2 to 6 are the body, whose size equals the length byte.0123456opcodeencipheredlength= 5x2-byte shortxz2-byte shortzCtrlrun toggleheaderbodyAs our path only contained a single step, we don’t enter the loop to send the “delta” waypoints, so we can cross-check our 5-byte payload against the length marker:4 + 2 * (pathLength - 1) + 1 = 4 + 2 * 0 + 1 = 5Once the snippets above have run, the packet is in the client’s outbound stream. That stream is drained to the network roughly every 20ms.Server receives the request#The server’s main loop wakes roughly once every 600ms. On each wake, it drains every player’s inbound buffer, runs whatever handlers the packets call for, and composes the outbound player updates that we’ll look at next. A packet that arrives just before a cycle is processed almost instantly; one that arrives just after waits nearly a full 600ms.That 600ms cycle time sets the granularity for latency. The 20ms client flush and any other networking overheads all swim well under this time. That’s why the rest of this post is about bytes, not time: there is no latency to save.Once the inbound buffer has been drained by the server, reading the packet is roughly the process above, but in reverse:int opcode = player.inboundStream.takeOpcode(); if (opcode == ClientToServerOpcodes.WALK_TILE) { int length = player.inboundStream.takeByte(); int deltaCount = (length - 4 - 1) / 2; int[] firstWaypoint = new int[2]; firstWaypoint[0] = player.inboundStream.takeShort(); firstWaypoint[1] = player.inboundStream.takeShort(); int[][] waypointDeltas = new int[deltaCount][2]; for (int i = 0; i < deltaCount; i++) { waypointDeltas[i][0] = player.inboundStream.takeByte(); waypointDeltas[i][1] = player.inboundStream.takeByte(); } boolean holdingCtrl = player.inboundStream.takeByte() == 1; player.processWalkTile(firstWaypoint, waypointDeltas, holdingCtrl); } As you can see, once we’ve identified the opcode, we can read the length byte and reverse the write logic to extract the number of deltas.I mentioned earlier that not all packets contain this length byte. In fact, most don’t; the majority of packets have a fixed-length body. Reading those is even simpler. Take, for instance, the “item on item” packet - sent when a player “uses” one item in their inventory with another:if (opcode == ClientToServerOpcodes.USE_ITEM_ON_ITEM) { int sourceItemId = player.inboundStream.takeShort(); int sourceInterfaceId = player.inboundStream.takeShort(); int sourceInterfaceSlot = player.inboundStream.takeShort(); int targetItemId = player.inboundStream.takeShort(); int