Developing a Browser Game: Managing Persistent Data

After five years as a full-stack developer, I decided a few months ago to embark on a browser game project, an opportunity to deepen my understanding of web development. I share one article per week on the technical challenges I encounter, which enhance my understanding and knowledge of web development. The first topic of interest: data storage.

As a browser game developer, data persistence is a problem that arises very quickly. A simple example is needing to keep track of the player's current score at all times and being able to update it. Working on a React application, data stored in the game state is by default erased every time the page is refreshed or the site is exited, making development impossible beyond a few minutes of gameplay.

Here are the constraints I face:#[#[- As a player, when I perform a significant action, that action is stored in the game's memory.]##[- As a player, if I refresh the page, the data remains accessible after the refresh.]##[- As a player, if I exit the browser and return later, the game retains my progress.]#]#

In a typical web project, this problem is common and is often solved by adding a database and a server with an API to access it. A database allows data to be stored in a structured way, and adding a server layer protects the data by permitting only specific operations necessary for the product. Additionally, the server enables data sharing among multiple users, which is a basic feature of most websites.

https://storage.googleapis.com/hocbar/600_sthinc-data-management-typical-web-application.png

In a typical web application, data is stored in a database, and a backend layer must be developed to serve this data to the frontend.

The main difficulty with this approach is the increased complexity of the project, as it requires maintaining two software components: frontend code + backend server, along with a database. A database also introduces significant rigidity in development due to the need to define the database schema (tables and columns). Revising database models often necessitates refactoring the code at multiple levels, which is very time-consuming.

Specifics of Video Game Development and Benchmarking

In the context of a video game, some constraints of a typical web application no longer apply:#[#[- Data sharing among players (for a single-player game, which is my case).]##[- Preventing malicious users from tampering with data. If a player only has access to their own data, it doesn't matter if they break the game as long as it doesn't impact others.]#]#

An analysis of known browser games like Universal Paperclip or Cookie Clicker shows that data is stored in local storage. For Cookie Clicker, it is moderately obfuscated with Base64 encoding, while for Universal Paperclip, it is stored in plain text. Allowing players to manipulate data at will—even cheat—did not hinder the success of these two projects because security is not a priority for this type of project.

https://storage.googleapis.com/hocbar/600_universal-paperclip-local-storage.png

In Universal Paperclip, all data is stored unencrypted in local storage.

Given the specifics of browser game development and existing examples, using browser memory seemed the easiest solution to implement at the start of my game's development. Indeed, it offers several advantages: #[#[- Synchronous manipulation vs. asynchronous manipulation when using a server and an API. Synchronous code is much simpler and more maintainable than asynchronous code, which must account for data retrieval delays from the server.]##[- No need to add a server layer or a database, significantly simplifying development. This also facilitates development tools such as resetting data to restart the game or saving the game's state to test key moments without replaying the entire game during each test.]#]#

https://storage.googleapis.com/hocbar/600_sthinc-data-management-simplified-game-application.png

Simplified version of persistent data management using local storage.

Several additional decisions helped me avoid putting all my eggs in one basket and remain flexible enough to revise this choice if needed: #[#[- Isolating browser memory manipulation functions in a dedicated service. If necessary, this service can be moved to backend code if unforeseen issues arise.]##[- Having an extremely simple interface for manipulating saved data that is agnostic of the technology used (except for being synchronous for now to benefit from frontend memory advantages).]#]#

For the choice of frontend memory type, I naturally leaned towards local storage, which remains persistent despite “hard refreshes” or browser closures.

In practice, I made this choice two months ago, and despite increasing the data stored in local storage and the complexity of my game, I have yet to encounter any issues that challenge this solution!

Implementing Delays in a Dialogue System for SMTH inc.

One of the main features of my game, SMTH inc., is the dialogue system, where players can interact with colleagues via a messaging interface. To enhance immersion, I quickly sought to implement a delay between the moment the player sends a message and the moment the character responds.

https://storage.googleapis.com/hocbar/600_smthinc-event-article-1.png

The three steps of the feature I want to implement

Version 1: Using setTimeout and Encountered Issues

The initial idea for implementing this feature was to use **setTimeout**, a JavaScript function that adds a delay before executing instructions. The advantage of setTimeout is its simplicity: it only takes two arguments:#[#[- The function to call after the delay.]##[- The delay in milliseconds.]#]#

With this, I quickly implemented a functional chat system with delays !

However, I soon encountered **a series of unexpected bugs**:#[#[- When multiple characters responded to the player, some messages would disappear.]##[- If the webpage was refreshed during the wait, the message would never arrive, leaving the player stuck.]#]#

Additionally, I wanted to **enhance the feature further** by adding an intermediate 'isTyping' state, where a small message like 'So-and-so is typing' would appear for a few seconds under the chat, similar to modern messaging systems. The functionality's design then evolved as follows:

https://storage.googleapis.com/hocbar/600_smthinc-event-article-2.png

The new feature schema with a new isTyping step

Revisiting the Issues and Challenges to Evolving the Feature

A fundamental problem with setTimeout is that it **does not easily handle concurrency**. The program's state and the values used during function execution are frozen at the start of execution and do not update if something changes during the waiting period.

https://storage.googleapis.com/hocbar/600_smthinc-event-article-3.png

With setTimeout, the state used for the update is the value when the function was first called, not when the callback is called after the delay

Imagine a situation where the player simultaneously responded to Emily and Lance and is waiting for their answers. The game's state used for both conversations is frozen by setTimeout when the function is first called. #[#[- Since Lance’s response time is shorter, the callback function updates this state at the end of its execution.]##[- However, when Emily responds, the game state used is still the initial state and not the updated one with Lance’s response.]##[- As a result, Lance’s message **remains visible for a brief period before disappearing** when Emily’s response is created.]#]#

**Second Problem: Messages Never Appear When setTimeout is called**

JavaScript execution must continue smoothly until the delay expires for the callback function to be executed. However, in a web application, this is not guaranteed. At any moment, the user could refresh the page or simply leave it. In such cases, with my naive implementation, the player never receives the message and becomes stuck if the response is necessary to progress in the game.

**Third Problem: Iterating for the New Feature**

The new feature involving two successive states (isWaiting and isTyping) during the NPC’s message wait time risks exacerbating the issues mentioned above. If we keep the same design with setTimeout, it means nesting two setTimeout calls! 

At this point, I decided these bugs were too significant to ignore and opted to completely redesign the feature.

If I summarize the critical needs of the feature in terms of acceptance criteria:#[#[- The event management system must know the current, up-to-date game state at the moment it updates it.]##[- In case of a page refresh or if the user leaves the site, the pending message must not be lost.]##[- To maintain a simple design, the system should function iteratively rather than recursively, avoiding nested operations.]#]#

Based on these criteria, I developed **a new design based on a timestamp system** with a straightforward operation.

Each planned state change is added to an array with an associated timestamp. #[#[- A loop checks every second whether any of the timestamps have been exceeded.]##[- If so, the change is applied, considering the current game state rather than the initial state.]#]#

https://storage.googleapis.com/hocbar/600_smthinc-event-article-4.png

The new event management system design with the storage of event in the local storage and a check every second

**Knowing the Current State at the Moment of the State Change** #[At the time of the timestamp check, if the current date exceeds the timestamp, the program consults the current state to update it. If two events simultaneously have surpassed timestamps, the second event will take into account the state changes from the first. This is achieved by synchronously updating localStorage during the first check and before the second.]#

**Protecting Against Page Refresh** #[ The queue of pending events is stored alongside the game state in localStorage. A page refresh or site exit does not clear localStorage, so the player’s progress is preserved without losing any information.]#

**Maintaining a Simple Iterative Design** #[#[This new system does not involve nested events. It doesn’t even need to know whether the messages belong to the same conversation or what the previous state was because that is outside its scope. It applies state changes agnostically, greatly simplifying the system.]#

- Rendering Management. Rigorous management of renders is necessary to avoid infinite render loops caused by frequent state-change checks.

- Slight Latency. The timestamp will never be perfectly respected, as the event will always trigger at the first check after the delay expires. In my case, this isn’t an issue but could be for other functionalities.

I’m very satisfied with this new system, which efficiently manages the game’s events. Its agnostic implementation allowed me to use the same system for dialogues, interface appearance changes, player score updates, and more. Today, I have a modular event management system for my entire game!

Accueil