Skip to main content

1. Canister upgrades, storage, and persistence

The Internet Computer handles persistent data storage using a feature known as stable memory. Stable memory is a unique feature of the Internet Computer that defines a data store separate from the canister's regular Wasm memory data store, which is known as heap memory. A canister's heap memory is not persistent storage and does not persist across canister upgrades; canister data and state stored in heap memory is removed when a canister is upgraded or reinstalled. For immutable canisters that use less than the heap memory limit of 4GiB, heap memory can be used. For larger canisters, and especially those that intend to be upgraded and changed over time, stable memory is important since it persists across canister upgrades, has a much larger storage capacity than heap memory, and is very beneficial for vertically scaling a dapp.

To use stable memory requires anticipating and indicating which canister data you want to be retained after a canister upgrade. For some canisters, this data might be all of the canister's data, while for others it may be only certain parts or none. By default, stable memory starts empty, and can hold up to 400GiB as long as the subnet the canister is running on has the available space. If a canister uses more stable memory than 400GiB, the canister will trap and become unrecoverable.

Memory types and terms​

There are several terms associated with memory and storage on ICP. To avoid confusion between them, let's define them here.

  • Stable memory: Stable memory refers to the Internet Computer's long-term data storage feature. Stable memory is not language specific, and can be utilized by canisters written in Typescript, Motoko, Rust, or any other language. Stable memory can hold up to 400GiB if the subnet can accommodate it. Stable memory persists automatically across upgrades, When a canister is upgraded (its code is changed after being initially deployed) anything stored in stable memory will be preserved.

  • Heap memory: Heap memory refers to the regular Wasm memory data store for each canister. This storage is temporary and is not persisted across canister upgrades. Data stored in heap memory is removed when the canister is upgraded or reinstalled. Heap memory is limited to 4GiB.

  • Stable Structures: Stable structures is an Azle-specific term that refers to a data structures with familiar APIs that allow write and read access to stable memory. Writing and reading to and from stable memory can be done with a low-level API, but it is generally easier and preferable to use stable structures.

To further understand stable memory and how to use it, let's learn about upgrading canisters.

Upgrading canisters​

Once a canister has been deployed and is running, there may need to be changes made to the canister's code to fix bugs or introduce new features. To make these changes, the canister must be upgraded.

Azle stable structures​

Azle currently provides one stable structure called StableBTreeMap. It's similar to a JavaScript Map and has most of the common operations you'd expect such as reading, inserting, and removing values.

Here's how to define a simple StableBTreeMap:

import { StableBTreeMap } from "azle";

let map = StableBTreeMap<string, string>(0);

This is a StableBTreeMap with a key of type string and a value of type string. Unless you want a default type of any for your key and value, then you must explicitly type your StableBTreeMap with type arguments.

StableBTreeMap works by encoding and decoding values under-the-hood, storing and retrieving these values in bytes in stable memory. When writing to and reading from a StableBTreeMap, by default the stableJson Serializable object is used to encode JS values into bytes and to decode JS values from bytes. stableJson uses JSON.stringify and JSON.parse with a custom replacer and reviver to handle many Candid and other values that you will most likely use in your canisters.

Interactive example​

Let's take a look at creating an Azle project and using StableBTreeMap to store data in stable memory.

Prerequisites​

Before you start, verify that you have set up your developer environment according to the instructions in 3 Developer environment setup.

Creating a new project​

To get started, create a new project in your working directory. Open a terminal window, navigate into your working directory (developer_journey), then use the commands:

use npx azle new <project_name> to create a new project

npx azle new counter

Then, navigate into the new project directory and install the dependencies

cd counter
npm install

Defining a stable structure​

In the following code example, we'll define a function called incrementCounter. This function is designed to manage a counter value stored within a StableBTreeMap, ensuring its stability across updates. Paste this code into the src/counter_backend/index.ts file:

import { StableBTreeMap } from "azle";
import express, { Request } from "express";

let counter = 0;
let map = StableBTreeMap<string, number>(0);

const incrementCounter = () => {
  const currentCounter = map.get("counter");
  if ("None" in currentCounter) {
    map.insert("counter", counter);
    return { counter: map.get("counter") };
  }
  const incrementedValue = currentCounter.Some + 1;
  map.insert("counter", incrementedValue);

  return { counter: incrementedValue };
};
const app = express();

app.use(express.json());

app.post("/db/update", (req: Request<any, any>, res) => {
  const result = incrementCounter();
  return res.json(result);
});

app.listen();
info

By default, GET requests are query calls and are not intended to update the canister's state. POST, PUT, PATCH, DELETE, etc., are used for update calls, modifying the state of the canister.

Save this file.

Deploying your counter dapp​

First step, make sure you initialize your replica by running the command:

dfx start --clean

To upgrade your canister, first you need to deploy the initial version of the canister. Deploy the canister with the command:

dfx deploy
info

For this tutorial, you're using the local replica environment to deploy the canisters. You can deploy yours on the mainnet with the flag --network ic. Remember that deploying to the mainnet will cost cycles.

You can interact with the counter dapp by sending requests through Postman, utilizing the Candid UI URL provided in the output of the dfx deploy command, like this:

Deployed canisters.
URLs:
  Backend canister via Candid interface:
    backend: http://127.0.0.1:4943/?canisterId=bd3sg-teaaa-aaaaa-qaaba-cai&id=bkyz2-fmaaa-aaaaa-qaaaq-cai

To interact with the counter, open Postman and make a POST request using the provided URL followed by /db/update, the counter will be 1.

Since this value is being stored in a StableBTreeMap, this value of 1 will persist through a canister upgrade. This is a great example of persistent storage using Stable structures.

Stable structures in action​

Now, let's make some changes to upgrade the canister. Alter the code of your canister, To keep things simple, you're going to change the counter increment value from 1 to 3. Your altered code looks like this:

import { StableBTreeMap } from "azle";
import express, { Request } from "express";

let counter = 0;
let map = StableBTreeMap<string, number>(0);

const incrementCounter = () => {
  const currentCounter = map.get("counter");
  if ("None" in currentCounter) {
    map.insert("counter", counter);
    return { counter: map.get("counter") };
  }
  const incrementedValue = currentCounter.Some + 3;
  map.insert("counter", incrementedValue);

  return { counter: incrementedValue };
};
const app = express();

app.use(express.json());

app.post("/db/update", (req: Request<any, any>, res) => {
  const result = incrementCounter();
  return res.json(result);
});

app.listen();

Save the file. Now, to confirm the canister's code has been upgraded, re-deploy the canister again with the command:

dfx deploy

This time, the output should include information about the canister's upgrade, such as:


🎉 Canister backend will be available at http://bkyz2-fmaaa-aaaaa-qaaaq-cai.localhost:4943

Installing canisters...
Upgrading code for canister backend, with canister ID bkyz2-fmaaa-aaaaa-qaaaq-cai
Deployed canisters.
URLs:
  Backend canister via Candid interface:
    backend: http://127.0.0.1:4943/?canisterId=bd3sg-teaaa-aaaaa-qaaba-cai&id=bkyz2-fmaaa-aaaaa-qaaaq-cai

Now, navigate back to Postman and make a POST request using the provided URL followed by /db/update again. This time, the counter should increment by 3, maintaining consistency with the value stored prior to the upgrade.

Need help?​

Did you get stuck somewhere in this tutorial, or feel like you need additional help understanding some of the concepts? The ICP community has several resources available for developers, like working groups and bootcamps, along with our Discord community, forum, and events such as hackathons. Here are a few to check out:

Next steps​

Next, you'll look at advanced canister calls, such as inter-canister calls and canister query methods.